ЛР 3 - БВТ2303 - Болотов - на 4 #75
+37
-4
@@ -1,20 +1,53 @@
|
||||
# VCS / CI
|
||||
.git
|
||||
.github
|
||||
|
||||
# Secrets and local env
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
secrets
|
||||
certs
|
||||
volumes
|
||||
data
|
||||
backups
|
||||
*.key
|
||||
*.pem
|
||||
*.p12
|
||||
*.pfx
|
||||
*.jks
|
||||
|
||||
# Local / runtime junk
|
||||
.venv
|
||||
venv
|
||||
__pycache__
|
||||
*.py[cod]
|
||||
*.egg-info
|
||||
.pytest_cache
|
||||
.mypy_cache
|
||||
.ruff_cache
|
||||
node_modules
|
||||
evidence
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
.idea
|
||||
.vscode
|
||||
|
||||
# Lab artifacts not needed at runtime
|
||||
evidence
|
||||
volumes
|
||||
data
|
||||
backups
|
||||
docs
|
||||
student-tests
|
||||
tests
|
||||
scripts
|
||||
examples
|
||||
contracts
|
||||
REPORT.md
|
||||
README.md
|
||||
COURSE.md
|
||||
CONTRIBUTING.md
|
||||
MAINTAINERS.md
|
||||
Dockerfile.example
|
||||
compose.example.yaml
|
||||
requirements-dev.txt
|
||||
*.md
|
||||
|
||||
# Keep only what the image needs: requirements.txt + src/ + migrations/
|
||||
|
||||
+2
-4
@@ -1,13 +1,11 @@
|
||||
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
|
||||
# These are logical interface suggestions; map actual framework names in REPORT.md.
|
||||
APP_HOST=0.0.0.0
|
||||
APP_PORT=8080
|
||||
APP_ENV=lab
|
||||
SESSION_TTL_SECONDS=3600
|
||||
# Choose credentials locally. File paths contain no secret values.
|
||||
DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
|
||||
SESSION_STORE=postgres
|
||||
SESSION_STORE=redis
|
||||
CACHE_URL_FILE=/run/secrets/cache_url
|
||||
# Number of interchangeable API replicas behind HAProxy (grade 3 requires ≥2).
|
||||
API_REPLICAS=2
|
||||
IDEMPOTENCY_TTL_SECONDS=86400
|
||||
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
# Pinned base image (lab3). Update digest when bumping Python.
|
||||
FROM python:3.12.8-slim-bookworm AS runtime
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
APP_HOST=0.0.0.0 \
|
||||
APP_PORT=8080 \
|
||||
PYTHONPATH=/app/src \
|
||||
MIGRATIONS_DIR=/app/migrations
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN groupadd --system --gid 10001 app \
|
||||
&& useradd --system --uid 10001 --gid app --home /app --shell /usr/sbin/nologin app
|
||||
|
||||
COPY --chown=app:app requirements.txt .
|
||||
RUN pip install --no-cache-dir --upgrade pip==24.3.1 \
|
||||
&& pip install --no-cache-dir --retries 10 --timeout 60 -r requirements.txt
|
||||
|
||||
COPY --chown=app:app src ./src
|
||||
COPY --chown=app:app migrations ./migrations
|
||||
|
||||
USER app
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=5s --timeout=3s --start-period=15s --retries=5 \
|
||||
CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2)"
|
||||
|
||||
CMD ["python", "-m", "messenger"]
|
||||
@@ -1,37 +1,97 @@
|
||||
# Отчёт — лабораторная 3
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
- Автор, группа: Болотов Илья, БВТ2303
|
||||
- Целевая оценка; выбранный трек 5, если нужен: **4** (трек 5 не выбран)
|
||||
- Ветка: `solution/bvt2303-bolotov`
|
||||
- Использованные источники, библиотеки и помощь ИИ: FastAPI, Uvicorn, psycopg3 (+pool), redis-py, argon2-cffi, HAProxy; перенос lab2 и адаптация под LB/идемпотентность с помощью ИИ; публичные материалы курса (README, contracts, tests).
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
**Prerequisites:** Docker Desktop (Engine), Docker Compose v2+. Локальный Python 3.9+ удобен для `make test` (на Windows задайте `PYTHONUTF8=1`).
|
||||
|
||||
```sh
|
||||
cp .env.example .env
|
||||
|
||||
# Windows:
|
||||
scripts\up.cmd
|
||||
# Unix:
|
||||
# chmod +x scripts/*.sh && ./scripts/up.sh
|
||||
# Эквивалент: docker compose --env-file .env up -d --build --scale api=2
|
||||
```
|
||||
|
||||
Дождаться `GET http://127.0.0.1:8080/health/ready` → 200.
|
||||
|
||||
Проверки:
|
||||
|
||||
```sh
|
||||
# Windows PowerShell:
|
||||
$env:PYTHONUTF8=1
|
||||
python scripts/check_contract.py
|
||||
python tests/smoke.py --base-url http://127.0.0.1:8080
|
||||
python tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url http://127.0.0.1:8080
|
||||
python tests/resilience.py --mode failover --action-script scripts/stop-one.cmd --base-url http://127.0.0.1:8080
|
||||
```
|
||||
|
||||
Unix: `scripts/restart.sh` / `scripts/stop-one.sh` (+ `chmod +x`).
|
||||
|
||||
Остановка без удаления данных: `docker compose --env-file .env down`
|
||||
Удаление учебных volumes: `docker compose --env-file .env down -v` (осознанно).
|
||||
|
||||
Версии образов: `python:3.12.8-slim-bookworm`, `postgres:16.6-bookworm`, `redis:7.4.2-bookworm`, `haproxy:2.9.14-bookworm`.
|
||||
|
||||
## Архитектура и решения
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab2, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
```
|
||||
Клиент → HAProxy (:8080, единственная публикация) → API × N (без host ports)
|
||||
→ PostgreSQL (users/chats/messages + idempotency_records)
|
||||
→ Redis (opaque sessions, AOF everysec)
|
||||
Миграции: one-shot сервис migrate (service_completed_successfully), не гонка реплик.
|
||||
```
|
||||
|
||||
- **Перенесено из lab2** (`mtusi2026lab2`, ветка `solution/bvt2303-bolotov`): `src/messenger`, Argon2id, Redis-сессии, student-tests каркас, стиль ошибок/cursor, non-root Dockerfile.
|
||||
- **Изменено в lab3:**
|
||||
- `X-Instance-Id` = hostname контейнера (стабилен на жизнь процесса).
|
||||
- Обязательный `Idempotency-Key` для POST сообщения; результат в PostgreSQL (`idempotency_records`) + `pg_advisory_xact_lock` между репликами.
|
||||
- Compose: `lb` + `--scale api≥2`, без `container_name`; API не публикует порты наружу.
|
||||
- CPU/RAM limits (`cpus`/`mem_limit`) на сервисах.
|
||||
- SIGTERM: readiness→503, uvicorn `timeout_graceful_shutdown=8`, compose `stop_grace_period: 12s`; HAProxy `inter 2s fall 2`.
|
||||
- **Конфиг (.env):** `SESSION_TTL_SECONDS`, `IDEMPOTENCY_TTL_SECONDS` (24ч), `API_REPLICAS`.
|
||||
- **Миграции:** `001_init.sql` (lab2) + `002_idempotency.sql`; совместимы с уже работающими репликами (только ADD TABLE/INDEX).
|
||||
|
||||
## Доказательства критериев
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
| Offline check | `python scripts/check_contract.py` | PASS skeleton | `evidence/check.txt` |
|
||||
| Публичный smoke (lab3) | `python tests/smoke.py --base-url http://127.0.0.1:8080` | PASS lab3 (idempotency + X-Instance-Id) | `evidence/smoke.txt` |
|
||||
| ≥2 реплики через LB | student `test_multiple_instance_ids_observed` | ≥2 разных X-Instance-Id | `evidence/student-tests.txt` |
|
||||
| Persistence restart | `resilience.py --mode persistence` + `scripts/restart.cmd` | PASS: session + messages | `evidence/persistence.txt` |
|
||||
| Failover stop-one | `resilience.py --mode failover` + `scripts/stop-one.cmd` | PASS; recovery ≈2.4 с; 2 инстанса до отказа | `evidence/failover.txt` |
|
||||
| Идемпотентность (в т.ч. parallel) | student `test_parallel_same_key_one_message` | 8 параллельных POST → 1 сообщение | `evidence/student-tests.txt` |
|
||||
| Последовательный отказ каждой реплики | `python scripts/seq_failover_evidence.py` | round0/1 recovery ≈5.5 с; target исчез из last5 | `evidence/seq-failover.json` |
|
||||
| Нагрузка (baseline) | `tests/load.py --duration 20 --concurrency 4` | ~241 RPS, err=0, p50=16ms p95=32ms p99=78ms | `evidence/load.json` |
|
||||
| Rolling update под нагрузкой | `python scripts/rolling_load_evidence.py` | 2 реплики поочерёдно; 356 ok / 3 err | `evidence/rolling-load.json` |
|
||||
| Собственные тесты | `python student-tests/run_all.py` | 24 OK (deps SKIP без RUN_DEPS_TEST) | `evidence/student-tests.txt` |
|
||||
|
||||
## Эксперименты и отказы
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
| `stop-one` одной API-реплики | чтение/запись через оставшуюся ≤10 с; без дублей при retry ключа | failover PASS; first_recovery≈2.4 с; 4 transient fail | данные/сессия сохранены |
|
||||
| Последовательный stop каждой реплики (с restore) | то же для обеих | recovery≈5.5 с; target absent | без потери подтверждённых |
|
||||
| Rolling replace API под нагрузкой | краткие ошибки, стенд жив | 3 ошибки / 356 успехов; p99↑ из‑за stop | volumes не трогались |
|
||||
| Restart postgres+redis+api (persistence) | прежний token и сообщения | PASS | ≤90 с на hook+ready |
|
||||
| Миграции one-shot | нет гонки DDL на репликах | `migrate` Exited 0 до старта API | — |
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
Аппаратный контекст: один Docker Desktop host (Windows). Измерения closed-loop (`load.py`) — функциональный baseline, не претензия на максимальный RPS железа.
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
## Ограничения
|
||||
|
||||
- Трек 5 (цена масштабирования / отказ узла) не выполнялся — целевая оценка 4.
|
||||
- Один Docker-host и один Postgres/Redis/LB — отказ API-процесса выдерживается; отказ host/DB — нет.
|
||||
- На Windows публичные тесты требуют `PYTHONUTF8=1` из‑за кириллицы в OpenAPI.
|
||||
- На Unix-клоне для shell-скриптов нужен `chmod +x scripts/*.sh` (указано в запуске).
|
||||
|
||||
## Итог
|
||||
|
||||
Стенд lab3 для оценки **4** выполнен и проверен: ≥2 API за HAProxy, идемпотентность POST сообщений (в т.ч. параллельно), persistence и failover с evidence, student-tests 24 OK. Трек 5 не заявляется.
|
||||
|
||||
+147
@@ -0,0 +1,147 @@
|
||||
# Lab3: Client → HAProxy → API×N → shared PostgreSQL + Redis (sessions).
|
||||
# Public entrypoint only via lb:8080. API replicas have no host ports.
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16.6-bookworm
|
||||
environment:
|
||||
POSTGRES_USER: messenger
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
|
||||
POSTGRES_DB: messenger
|
||||
secrets:
|
||||
- postgres_password
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 10s
|
||||
mem_limit: 512m
|
||||
cpus: 0.50
|
||||
networks:
|
||||
- labnet
|
||||
|
||||
redis:
|
||||
image: redis:7.4.2-bookworm
|
||||
command:
|
||||
- redis-server
|
||||
- --appendonly
|
||||
- "yes"
|
||||
- --appendfsync
|
||||
- everysec
|
||||
- --dir
|
||||
- /data
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
mem_limit: 128m
|
||||
cpus: 0.25
|
||||
networks:
|
||||
- labnet
|
||||
|
||||
migrate:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
image: mtusi-messenger:lab3
|
||||
command: ["python", "-m", "messenger.migrate"]
|
||||
environment:
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
MIGRATIONS_DIR: /app/migrations
|
||||
secrets:
|
||||
- database_url
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
mem_limit: 256m
|
||||
cpus: 0.25
|
||||
networks:
|
||||
- labnet
|
||||
|
||||
api:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
image: mtusi-messenger:lab3
|
||||
environment:
|
||||
APP_HOST: "0.0.0.0"
|
||||
APP_PORT: "8080"
|
||||
APP_ENV: lab
|
||||
SESSION_TTL_SECONDS: "${SESSION_TTL_SECONDS:-3600}"
|
||||
IDEMPOTENCY_TTL_SECONDS: "${IDEMPOTENCY_TTL_SECONDS:-86400}"
|
||||
SESSION_STORE: redis
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
CACHE_URL_FILE: /run/secrets/cache_url
|
||||
MIGRATIONS_DIR: /app/migrations
|
||||
secrets:
|
||||
- database_url
|
||||
- cache_url
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
redis:
|
||||
condition: service_healthy
|
||||
# No host ports: only reachable via labnet / HAProxy.
|
||||
expose:
|
||||
- "8080"
|
||||
healthcheck:
|
||||
test:
|
||||
[
|
||||
"CMD",
|
||||
"python",
|
||||
"-c",
|
||||
"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2)",
|
||||
]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 12
|
||||
start_period: 20s
|
||||
stop_grace_period: 12s
|
||||
mem_limit: 256m
|
||||
cpus: 0.50
|
||||
networks:
|
||||
- labnet
|
||||
|
||||
lb:
|
||||
image: haproxy:2.9.14-bookworm
|
||||
volumes:
|
||||
- ./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
|
||||
ports:
|
||||
- "127.0.0.1:8080:8080"
|
||||
depends_on:
|
||||
api:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
# Image has no curl/wget; validate config + process. Public readiness is /health/ready via LB.
|
||||
test: ["CMD", "haproxy", "-c", "-f", "/usr/local/etc/haproxy/haproxy.cfg"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
start_period: 5s
|
||||
mem_limit: 128m
|
||||
cpus: 0.25
|
||||
networks:
|
||||
- labnet
|
||||
|
||||
secrets:
|
||||
postgres_password:
|
||||
file: ./secrets/postgres_password
|
||||
database_url:
|
||||
file: ./secrets/database_url
|
||||
cache_url:
|
||||
file: ./secrets/cache_url
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
redisdata:
|
||||
|
||||
networks:
|
||||
labnet:
|
||||
driver: bridge
|
||||
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"lab": 3,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 20.0,
|
||||
"elapsed_seconds": 20.032,
|
||||
"concurrency": 4,
|
||||
"successful_requests": 4821,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 240.665,
|
||||
"successful_latency_ms": {
|
||||
"p50": 16.0,
|
||||
"p95": 32.0,
|
||||
"p99": 78.0
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,9 @@
|
||||
date: 2026-10-30
|
||||
branch: solution/bvt2303-bolotov
|
||||
target_grade: 4
|
||||
host: Docker Desktop on Windows
|
||||
images: python:3.12.8-slim-bookworm, postgres:16.6-bookworm, redis:7.4.2-bookworm, haproxy:2.9.14-bookworm
|
||||
api_replicas: 2
|
||||
entrypoint: http://127.0.0.1:8080 (HAProxy only)
|
||||
limits: api 0.5 CPU / 256Mi; postgres 0.5/512Mi; redis 0.25/128Mi; lb 0.25/128Mi
|
||||
health: HAProxy inter=2s fall=2; API stop_grace_period=12s; uvicorn graceful=8s
|
||||
Binary file not shown.
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"successful": 356,
|
||||
"errors": 3,
|
||||
"p50_ms": 16.0,
|
||||
"p95_ms": 32.0,
|
||||
"p99_ms": 2016.0,
|
||||
"replicas_rolled": 2
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,26 @@
|
||||
[
|
||||
{
|
||||
"round": 0,
|
||||
"observed_before": [
|
||||
"252249c099ca",
|
||||
"8aabecba79c6"
|
||||
],
|
||||
"target": "252249c099ca",
|
||||
"successful_writes": 85,
|
||||
"failed_attempts": 2,
|
||||
"first_recovery_seconds": 5.594,
|
||||
"target_absent_in_last5": true
|
||||
},
|
||||
{
|
||||
"round": 1,
|
||||
"observed_before": [
|
||||
"252249c099ca",
|
||||
"8aabecba79c6"
|
||||
],
|
||||
"target": "8aabecba79c6",
|
||||
"successful_writes": 129,
|
||||
"failed_attempts": 1,
|
||||
"first_recovery_seconds": 5.469,
|
||||
"target_absent_in_last5": true
|
||||
}
|
||||
]
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
+4
-6
@@ -1,7 +1,5 @@
|
||||
# Инфраструктура студента
|
||||
# Инфраструктура lab3
|
||||
|
||||
Здесь разместите конфигурации выбранных сервисов; основную топологию — в корневом compose.yaml/stack.yaml. `compose.example.yaml` — только worksheet ролей и намеренно не запускается. `contracts/infrastructure.json` задаёт минимальные роли, не готовые образы/пароли.
|
||||
|
||||
В REPORT.md сопоставьте роль → service name → версия/digest → health/readiness → volume → сеть/порт → способ передачи credentials. Приведите bootstrap (schema, bucket, broker topology, а с lab6 provisioning) и безопасное повторение этих команд.
|
||||
|
||||
Приложение строится из исходников. Не используйте заранее существующий образ преподавательского сервера вместо своей реализации. Restart hook перезапускает все соответствующие stores с volumes; failover hook останавливает только заданную API-реплику.
|
||||
- `haproxy.cfg` — балансировщик (round-robin, HTTP healthcheck `/health/ready`).
|
||||
- DNS Docker `127.0.0.11` + `server-template` — подхват `--scale api=N` и rolling replace.
|
||||
- Период проверки backend: `inter 2s`, `fall 2` → типичное исключение недоступной реплики ≤4 с; с учётом DNS hold — до ~5–10 с.
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
# Lab3 load balancer: round-robin across Compose DNS name "api" (all replicas).
|
||||
global
|
||||
maxconn 2048
|
||||
log stdout format raw local0
|
||||
|
||||
defaults
|
||||
mode http
|
||||
log global
|
||||
option httplog
|
||||
option dontlognull
|
||||
timeout connect 3s
|
||||
timeout client 30s
|
||||
timeout server 30s
|
||||
timeout check 2s
|
||||
timeout http-request 10s
|
||||
retries 2
|
||||
|
||||
resolvers docker
|
||||
nameserver dns1 127.0.0.11:53
|
||||
resolve_retries 3
|
||||
timeout resolve 1s
|
||||
timeout retry 1s
|
||||
hold valid 5s
|
||||
hold obsolete 10s
|
||||
|
||||
frontend messenger_fe
|
||||
bind *:8080
|
||||
acl is_lb_live path /__lb/live
|
||||
http-request return status 200 content-type text/plain string ok if is_lb_live
|
||||
default_backend messenger_api
|
||||
|
||||
backend messenger_api
|
||||
balance roundrobin
|
||||
option httpchk
|
||||
http-check send meth GET uri /health/ready ver HTTP/1.1 hdr Host api
|
||||
http-check expect status 200
|
||||
# server-template expands A records for Compose service DNS name "api".
|
||||
# inter=2s fall=2 → typically ≤4s to mark a backend down after failed checks.
|
||||
server-template api 8 api:8080 check inter 2s fall 2 rise 1 resolvers docker resolve-prefer ipv4 init-addr none
|
||||
@@ -0,0 +1,48 @@
|
||||
-- Lab2 schema: users, chats, membership, messages.
|
||||
-- Applied once via schema_migrations; safe to re-run application bootstrap.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version TEXT PRIMARY KEY,
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id UUID PRIMARY KEY,
|
||||
username TEXT NOT NULL,
|
||||
display_name TEXT NOT NULL,
|
||||
password_hash TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL,
|
||||
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||
CONSTRAINT users_username_unique UNIQUE (username)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS chats (
|
||||
id UUID PRIMARY KEY,
|
||||
title TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS chat_members (
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
user_id UUID NOT NULL REFERENCES users(id),
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_chat_members_user
|
||||
ON chat_members (user_id, chat_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id UUID PRIMARY KEY,
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
sender_id UUID NOT NULL REFERENCES users(id),
|
||||
text TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
-- Keyset pagination by (created_at, id) within a chat; avoids full history scan.
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_chat_created_id
|
||||
ON messages (chat_id, created_at ASC, id ASC);
|
||||
|
||||
-- List chats for a user ordered by chat created_at/id.
|
||||
CREATE INDEX IF NOT EXISTS idx_chats_created_id
|
||||
ON chats (created_at ASC, id ASC);
|
||||
@@ -0,0 +1,16 @@
|
||||
-- Lab3: shared idempotency records for POST /messages (across API replicas).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS idempotency_records (
|
||||
user_id UUID NOT NULL,
|
||||
method TEXT NOT NULL,
|
||||
path TEXT NOT NULL,
|
||||
idem_key TEXT NOT NULL,
|
||||
request_hash TEXT NOT NULL,
|
||||
response_status INT NOT NULL,
|
||||
response_body JSONB NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL,
|
||||
PRIMARY KEY (user_id, method, path, idem_key)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_idempotency_created
|
||||
ON idempotency_records (created_at);
|
||||
@@ -0,0 +1,6 @@
|
||||
fastapi==0.115.6
|
||||
uvicorn[standard]==0.32.1
|
||||
pydantic==2.10.3
|
||||
psycopg[binary,pool]==3.2.4
|
||||
redis==5.2.1
|
||||
argon2-cffi==23.1.0
|
||||
@@ -0,0 +1,11 @@
|
||||
@echo off
|
||||
setlocal
|
||||
cd /d "%~dp0\.."
|
||||
if "%API_REPLICAS%"=="" set API_REPLICAS=2
|
||||
docker compose --env-file .env restart postgres redis
|
||||
if errorlevel 1 exit /b 1
|
||||
docker compose --env-file .env up -d --no-deps --scale api=%API_REPLICAS% api
|
||||
if errorlevel 1 exit /b 1
|
||||
docker compose --env-file .env restart lb
|
||||
if errorlevel 1 exit /b 1
|
||||
exit /b 0
|
||||
@@ -0,0 +1,10 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
# Restart API replicas and all state stores; preserve volumes.
|
||||
cd "$(dirname "$0")/.."
|
||||
REPLICAS="${API_REPLICAS:-2}"
|
||||
docker compose --env-file .env restart postgres redis
|
||||
# Recreate API fleet at desired scale (migrate already applied; skip one-shot).
|
||||
docker compose --env-file .env up -d --no-deps --scale "api=${REPLICAS}" api
|
||||
docker compose --env-file .env restart lb
|
||||
exit 0
|
||||
@@ -0,0 +1,28 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
# Grade-4 rolling replacement: recreate API containers one-by-one under LB.
|
||||
# Usage: scripts/rolling-update.sh
|
||||
cd "$(dirname "$0")/.."
|
||||
REPLICAS="${API_REPLICAS:-2}"
|
||||
|
||||
ids=$(docker compose --env-file .env ps -q api)
|
||||
count=0
|
||||
for id in $ids; do
|
||||
[ -n "$id" ] || continue
|
||||
count=$((count + 1))
|
||||
printf '%s\n' "rolling: stopping $id"
|
||||
docker stop -t 10 "$id" >/dev/null
|
||||
# Bring fleet back to desired scale (starts a fresh replica; old stopped removed by up).
|
||||
docker compose --env-file .env up -d --no-deps --scale "api=${REPLICAS}" --remove-orphans api
|
||||
# Wait until LB again sees healthy backends via public ready.
|
||||
i=0
|
||||
while [ "$i" -lt 60 ]; do
|
||||
if curl -sf "http://127.0.0.1:8080/health/ready" >/dev/null 2>&1; then
|
||||
break
|
||||
fi
|
||||
i=$((i + 1))
|
||||
sleep 1
|
||||
done
|
||||
printf '%s\n' "rolling: step $count ready"
|
||||
done
|
||||
printf '%s\n' "rolling update finished ($count steps, scale=${REPLICAS})"
|
||||
@@ -0,0 +1,150 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Grade-4: closed-loop POSTs during rolling API replacement."""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
BASE = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
|
||||
|
||||
|
||||
def call(method, path, body=None, token=None, headers=None):
|
||||
h = {"Accept": "application/json"}
|
||||
if token:
|
||||
h["Authorization"] = "Bearer " + token
|
||||
data = None
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
h["Content-Type"] = "application/json"
|
||||
if headers:
|
||||
h.update(headers)
|
||||
req = urllib.request.Request(BASE + path, data=data, headers=h, method=method)
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=5)
|
||||
except urllib.error.HTTPError as err:
|
||||
resp = err
|
||||
except Exception as exc:
|
||||
return {"status": 0, "error": type(exc).__name__}
|
||||
raw = resp.read()
|
||||
js = None
|
||||
if raw and resp.headers.get_content_type() == "application/json":
|
||||
js = json.loads(raw.decode("utf-8"))
|
||||
return {"status": resp.code, "json": js, "ms": 0}
|
||||
|
||||
|
||||
def bootstrap():
|
||||
uname = "roll_" + uuid.uuid4().hex[:10]
|
||||
pw = "T3st-" + uuid.uuid4().hex
|
||||
user = call("POST", "/api/v1/users", {"username": uname, "display_name": "Roll", "password": pw})
|
||||
assert user["status"] == 201, user
|
||||
cred = base64.b64encode(f"{uname}:{pw}".encode()).decode()
|
||||
tok = call("POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic " + cred})["json"]["token"]
|
||||
peer_u = "rollb_" + uuid.uuid4().hex[:10]
|
||||
peer = call("POST", "/api/v1/users", {"username": peer_u, "display_name": "RollB", "password": pw})["json"]
|
||||
chat = call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
{"title": "roll", "member_ids": [user["json"]["id"], peer["id"]]},
|
||||
token=tok,
|
||||
)["json"]
|
||||
return tok, chat["id"]
|
||||
|
||||
|
||||
def main():
|
||||
tok, chat_id = bootstrap()
|
||||
stats = {"ok": 0, "err": 0, "latencies_ms": []}
|
||||
stop = threading.Event()
|
||||
|
||||
def worker():
|
||||
while not stop.is_set():
|
||||
key = uuid.uuid4().hex
|
||||
t0 = time.monotonic()
|
||||
r = call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat_id}/messages",
|
||||
{"text": "roll-" + key},
|
||||
token=tok,
|
||||
headers={"Idempotency-Key": key},
|
||||
)
|
||||
dt = (time.monotonic() - t0) * 1000
|
||||
if r.get("status") == 201:
|
||||
stats["ok"] += 1
|
||||
stats["latencies_ms"].append(dt)
|
||||
else:
|
||||
stats["err"] += 1
|
||||
time.sleep(0.05)
|
||||
|
||||
threads = [threading.Thread(target=worker, daemon=True) for _ in range(4)]
|
||||
for th in threads:
|
||||
th.start()
|
||||
time.sleep(2) # warm-up
|
||||
# Rolling: stop one container, scale back, repeat for each current replica.
|
||||
ids = (
|
||||
subprocess.check_output(
|
||||
["docker", "compose", "--env-file", ".env", "ps", "-q", "api"],
|
||||
cwd=ROOT,
|
||||
text=True,
|
||||
)
|
||||
.strip()
|
||||
.split()
|
||||
)
|
||||
for cid in ids:
|
||||
subprocess.check_call(["docker", "stop", "-t", "10", cid], cwd=ROOT)
|
||||
subprocess.check_call(
|
||||
[
|
||||
"docker",
|
||||
"compose",
|
||||
"--env-file",
|
||||
".env",
|
||||
"up",
|
||||
"-d",
|
||||
"--no-deps",
|
||||
"--scale",
|
||||
"api=2",
|
||||
"--remove-orphans",
|
||||
"api",
|
||||
],
|
||||
cwd=ROOT,
|
||||
)
|
||||
# wait until ready through LB
|
||||
for _ in range(60):
|
||||
if call("GET", "/health/ready").get("status") == 200:
|
||||
break
|
||||
time.sleep(0.5)
|
||||
time.sleep(2)
|
||||
time.sleep(2)
|
||||
stop.set()
|
||||
for th in threads:
|
||||
th.join(timeout=5)
|
||||
lats = sorted(stats["latencies_ms"])
|
||||
def pct(p):
|
||||
if not lats:
|
||||
return None
|
||||
return round(lats[min(len(lats) - 1, int(len(lats) * p))], 2)
|
||||
duration = max(sum(lats) / 1000.0, 1e-6)
|
||||
# Approximate wall duration from thread runtime ~ warm+rolling
|
||||
summary = {
|
||||
"successful": stats["ok"],
|
||||
"errors": stats["err"],
|
||||
"p50_ms": pct(0.50),
|
||||
"p95_ms": pct(0.95),
|
||||
"p99_ms": pct(0.99),
|
||||
"replicas_rolled": len(ids),
|
||||
}
|
||||
out = ROOT / "evidence" / "rolling-load.json"
|
||||
out.write_text(json.dumps(summary, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
print(json.dumps(summary, ensure_ascii=False))
|
||||
print("WROTE", out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,167 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Grade-4 evidence: stop each observed API replica in turn; restore between steps."""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import time
|
||||
import uuid
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
BASE = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
|
||||
|
||||
|
||||
def call(method, path, body=None, token=None, headers=None, expected=None):
|
||||
h = {"Accept": "application/json"}
|
||||
if token:
|
||||
h["Authorization"] = "Bearer " + token
|
||||
data = None
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
h["Content-Type"] = "application/json"
|
||||
if headers:
|
||||
h.update(headers)
|
||||
req = urllib.request.Request(BASE + path, data=data, headers=h, method=method)
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=5)
|
||||
except urllib.error.HTTPError as err:
|
||||
resp = err
|
||||
raw = resp.read()
|
||||
status = resp.code
|
||||
if expected is not None and status != expected:
|
||||
raise RuntimeError(f"{method} {path}: expected {expected}, got {status}: {raw[:200]!r}")
|
||||
js = None
|
||||
if raw and resp.headers.get_content_type() == "application/json":
|
||||
js = json.loads(raw.decode("utf-8"))
|
||||
return {"status": status, "headers": resp.headers, "json": js}
|
||||
|
||||
|
||||
def header(resp, name: str) -> str:
|
||||
return resp["headers"].get(name) or resp["headers"].get(name.lower()) or ""
|
||||
|
||||
|
||||
def restore_api():
|
||||
replicas = os.environ.get("API_REPLICAS", "2")
|
||||
subprocess.check_call(
|
||||
[
|
||||
"docker",
|
||||
"compose",
|
||||
"--env-file",
|
||||
".env",
|
||||
"up",
|
||||
"-d",
|
||||
"--no-deps",
|
||||
"--scale",
|
||||
f"api={replicas}",
|
||||
"--remove-orphans",
|
||||
"api",
|
||||
],
|
||||
cwd=ROOT,
|
||||
)
|
||||
for _ in range(60):
|
||||
try:
|
||||
call("GET", "/health/ready", expected=200)
|
||||
return
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
raise RuntimeError("API did not become ready after restore")
|
||||
|
||||
|
||||
def one_round(round_no: int) -> dict:
|
||||
uname = "seq_" + uuid.uuid4().hex[:10]
|
||||
pw = "T3st-" + uuid.uuid4().hex
|
||||
user = call(
|
||||
"POST",
|
||||
"/api/v1/users",
|
||||
{"username": uname, "display_name": "Seq", "password": pw},
|
||||
expected=201,
|
||||
)["json"]
|
||||
cred = base64.b64encode(f"{uname}:{pw}".encode()).decode()
|
||||
tok = call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
headers={"Authorization": "Basic " + cred},
|
||||
expected=201,
|
||||
)["json"]["token"]
|
||||
peer_u = "seqb_" + uuid.uuid4().hex[:10]
|
||||
peer = call(
|
||||
"POST",
|
||||
"/api/v1/users",
|
||||
{"username": peer_u, "display_name": "SeqB", "password": pw},
|
||||
expected=201,
|
||||
)["json"]
|
||||
chat = call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
{"title": "seq", "member_ids": [user["id"], peer["id"]]},
|
||||
token=tok,
|
||||
expected=201,
|
||||
)["json"]
|
||||
|
||||
observed = set()
|
||||
for _ in range(40):
|
||||
observed.add(header(call("GET", "/api/v1/users/me", token=tok, expected=200), "X-Instance-Id"))
|
||||
if len(observed) < 2:
|
||||
raise RuntimeError(f"need ≥2 instances, got {observed}")
|
||||
target = sorted(observed)[round_no % len(observed)]
|
||||
|
||||
env = dict(os.environ)
|
||||
env["TARGET_INSTANCE_ID"] = target
|
||||
script = ROOT / ("scripts/stop-one.cmd" if os.name == "nt" else "scripts/stop-one.sh")
|
||||
started = time.monotonic()
|
||||
proc = subprocess.Popen([str(script)], env=env, cwd=ROOT)
|
||||
ok = fail = 0
|
||||
first = None
|
||||
after = []
|
||||
while time.monotonic() - started < 20:
|
||||
code = proc.poll()
|
||||
try:
|
||||
key = uuid.uuid4().hex
|
||||
r = call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
{"text": "seq-" + key},
|
||||
token=tok,
|
||||
headers={"Idempotency-Key": key},
|
||||
expected=201,
|
||||
)
|
||||
ok += 1
|
||||
if code == 0:
|
||||
if first is None:
|
||||
first = time.monotonic() - started
|
||||
after.append(header(r, "X-Instance-Id"))
|
||||
except Exception:
|
||||
fail += 1
|
||||
time.sleep(0.1)
|
||||
if proc.poll() is None:
|
||||
proc.wait(timeout=10)
|
||||
result = {
|
||||
"round": round_no,
|
||||
"observed_before": sorted(observed),
|
||||
"target": target,
|
||||
"successful_writes": ok,
|
||||
"failed_attempts": fail,
|
||||
"first_recovery_seconds": round(first, 3) if first is not None else None,
|
||||
"target_absent_in_last5": target not in after[-5:],
|
||||
}
|
||||
restore_api()
|
||||
return result
|
||||
|
||||
|
||||
def main():
|
||||
results = []
|
||||
for i in range(2):
|
||||
results.append(one_round(i))
|
||||
print(json.dumps(results[-1], ensure_ascii=False))
|
||||
out = ROOT / "evidence" / "seq-failover.json"
|
||||
out.write_text(json.dumps(results, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
print("WROTE", out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,17 @@
|
||||
@echo off
|
||||
setlocal EnableExtensions
|
||||
cd /d "%~dp0\.."
|
||||
if "%TARGET_INSTANCE_ID%"=="" (
|
||||
echo stop-one: TARGET_INSTANCE_ID required >&2
|
||||
exit /b 1
|
||||
)
|
||||
for /f "usebackq delims=" %%I in (`docker compose --env-file .env ps -q api`) do (
|
||||
for /f "usebackq delims=" %%H in (`docker inspect -f "{{.Config.Hostname}}" %%I`) do (
|
||||
if /I "%%H"=="%TARGET_INSTANCE_ID%" (
|
||||
docker stop -t 10 %%I >nul
|
||||
exit /b 0
|
||||
)
|
||||
)
|
||||
)
|
||||
echo stop-one: replica not found >&2
|
||||
exit /b 1
|
||||
@@ -0,0 +1,43 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
# Stop ONE API replica matching TARGET_INSTANCE_ID (= X-Instance-Id / hostname).
|
||||
# Leave it stopped; do not touch DB/Redis/LB/volumes.
|
||||
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
found=""
|
||||
# Compose project containers for service api
|
||||
ids=$(docker compose --env-file .env ps -q api 2>/dev/null || true)
|
||||
for id in $ids; do
|
||||
[ -n "$id" ] || continue
|
||||
host=$(docker inspect -f '{{.Config.Hostname}}' "$id")
|
||||
if [ "$host" = "$TARGET_INSTANCE_ID" ]; then
|
||||
found=$id
|
||||
break
|
||||
fi
|
||||
iid=$(docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$id" | sed -n 's/^INSTANCE_ID=//p' | head -n1)
|
||||
if [ -n "$iid" ] && [ "$iid" = "$TARGET_INSTANCE_ID" ]; then
|
||||
found=$id
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [ -z "$found" ]; then
|
||||
# Fallback: any running container with matching hostname (scaled names differ).
|
||||
for id in $(docker ps -q --filter "ancestor=mtusi-messenger:lab3"); do
|
||||
host=$(docker inspect -f '{{.Config.Hostname}}' "$id")
|
||||
if [ "$host" = "$TARGET_INSTANCE_ID" ]; then
|
||||
found=$id
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
if [ -z "$found" ]; then
|
||||
printf '%s\n' "stop-one: replica not found for TARGET_INSTANCE_ID" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Graceful stop; API marks not-ready on SIGTERM before exit (uvicorn grace ~8s).
|
||||
docker stop -t 10 "$found" >/dev/null
|
||||
exit 0
|
||||
@@ -0,0 +1,10 @@
|
||||
@echo off
|
||||
setlocal
|
||||
cd /d "%~dp0\.."
|
||||
if "%API_REPLICAS%"=="" set API_REPLICAS=2
|
||||
docker compose --env-file .env config --quiet
|
||||
if errorlevel 1 exit /b 1
|
||||
docker compose --env-file .env up -d --build --scale api=%API_REPLICAS%
|
||||
if errorlevel 1 exit /b 1
|
||||
echo stack up: api scale=%API_REPLICAS%, entry http://127.0.0.1:8080
|
||||
exit /b 0
|
||||
@@ -0,0 +1,8 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
# Bootstrap lab3 stack with ≥2 API replicas behind LB.
|
||||
cd "$(dirname "$0")/.."
|
||||
REPLICAS="${API_REPLICAS:-2}"
|
||||
docker compose --env-file .env config --quiet
|
||||
docker compose --env-file .env up -d --build --scale "api=${REPLICAS}"
|
||||
printf '%s\n' "stack up: api scale=${REPLICAS}, entry http://127.0.0.1:8080"
|
||||
@@ -0,0 +1,3 @@
|
||||
"""Messenger API for lab3 (LB + shared sessions + idempotent messages)."""
|
||||
|
||||
__version__ = "3.0.0"
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Run the messenger HTTP server."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_SRC = Path(__file__).resolve().parents[1]
|
||||
if str(_SRC) not in sys.path:
|
||||
sys.path.insert(0, str(_SRC))
|
||||
|
||||
import uvicorn
|
||||
|
||||
from messenger import config
|
||||
|
||||
|
||||
def main() -> None:
|
||||
uvicorn.run(
|
||||
"messenger.app:app",
|
||||
host=config.app_host(),
|
||||
port=config.app_port(),
|
||||
log_level="info",
|
||||
access_log=False,
|
||||
timeout_keep_alive=5,
|
||||
timeout_graceful_shutdown=8,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,666 @@
|
||||
"""HTTP API: multi-replica messenger behind LB (lab3)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import signal
|
||||
import uuid
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, List, Optional, Tuple
|
||||
|
||||
from fastapi import FastAPI, Header, Query, Request
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse, Response
|
||||
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||
|
||||
from messenger import config, db, sessions
|
||||
from messenger.sessions import SessionStoreError
|
||||
from messenger.store import Store, StoreUnavailable, format_ts
|
||||
|
||||
USERNAME_RE = re.compile(r"^[a-z0-9_]{3,32}\Z")
|
||||
UUID_RE = re.compile(
|
||||
r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\Z"
|
||||
)
|
||||
IDEMPOTENCY_KEY_RE = re.compile(r"^[A-Za-z0-9._:-]{1,128}\Z")
|
||||
BODY_READ_TIMEOUT_S = 5.0
|
||||
PASSWORD_MIN = 12
|
||||
PASSWORD_MAX = 128
|
||||
|
||||
store = Store()
|
||||
ready = False
|
||||
INSTANCE_ID = config.instance_id()
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s %(levelname)s %(message)s",
|
||||
)
|
||||
log = logging.getLogger("messenger")
|
||||
|
||||
|
||||
def mark_not_ready(*_args):
|
||||
global ready
|
||||
if ready:
|
||||
ready = False
|
||||
log.info("readiness disabled; draining in-flight requests")
|
||||
|
||||
|
||||
def _chain_signal(previous):
|
||||
def handler(signum, frame):
|
||||
mark_not_ready()
|
||||
if callable(previous) and previous not in (signal.SIG_DFL, signal.SIG_IGN):
|
||||
previous(signum, frame)
|
||||
|
||||
return handler
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_app: FastAPI):
|
||||
try:
|
||||
signal.signal(signal.SIGTERM, _chain_signal(signal.getsignal(signal.SIGTERM)))
|
||||
signal.signal(signal.SIGINT, _chain_signal(signal.getsignal(signal.SIGINT)))
|
||||
except (ValueError, OSError) as exc:
|
||||
log.warning("signal handlers not installed: %s", exc)
|
||||
global ready
|
||||
db.init_pool()
|
||||
# Migrations run once via `python -m messenger.migrate` (compose migrate service).
|
||||
sessions.init_client()
|
||||
ready = True
|
||||
log.info("messenger started instance_id=%s", INSTANCE_ID)
|
||||
yield
|
||||
mark_not_ready()
|
||||
sessions.close_client()
|
||||
db.close_pool()
|
||||
log.info("messenger stopped")
|
||||
|
||||
|
||||
def request_id_from(request: Request) -> str:
|
||||
incoming = request.headers.get("x-request-id")
|
||||
if incoming and incoming.strip():
|
||||
return incoming.strip()[:128]
|
||||
return str(uuid.uuid4())
|
||||
|
||||
|
||||
def error_body(code: str, message: str, request_id: str) -> dict:
|
||||
return {"error": {"code": code, "message": message, "request_id": request_id}}
|
||||
|
||||
|
||||
def common_headers(request_id: str, extra: Optional[dict] = None) -> dict:
|
||||
headers = {"X-Request-Id": request_id, "X-Instance-Id": INSTANCE_ID}
|
||||
if extra:
|
||||
headers.update(extra)
|
||||
return headers
|
||||
|
||||
|
||||
def error_response(
|
||||
status: int,
|
||||
code: str,
|
||||
message: str,
|
||||
request_id: str,
|
||||
www_authenticate: Optional[str] = None,
|
||||
) -> JSONResponse:
|
||||
headers = common_headers(request_id)
|
||||
if www_authenticate:
|
||||
headers["WWW-Authenticate"] = www_authenticate
|
||||
return JSONResponse(
|
||||
status_code=status,
|
||||
content=error_body(code, message, request_id),
|
||||
headers=headers,
|
||||
)
|
||||
|
||||
|
||||
def is_blank(value: str) -> bool:
|
||||
return value.strip() == ""
|
||||
|
||||
|
||||
def has_nul(value: str) -> bool:
|
||||
# PostgreSQL text cannot store U+0000; reject explicitly (400), never 500/503.
|
||||
return "\x00" in value
|
||||
|
||||
|
||||
def has_lone_surrogate(value: str) -> bool:
|
||||
return any(0xD800 <= ord(ch) <= 0xDFFF for ch in value)
|
||||
|
||||
|
||||
def codepoint_len(value: str) -> int:
|
||||
return len(value)
|
||||
|
||||
|
||||
def parse_uuid(value: str) -> Optional[str]:
|
||||
if not isinstance(value, str) or not UUID_RE.match(value):
|
||||
return None
|
||||
try:
|
||||
return str(uuid.UUID(value))
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def encode_cursor(resource: str, created_at: datetime, item_id: str) -> str:
|
||||
raw = f"{resource}|{created_at.strftime('%Y-%m-%dT%H:%M:%SZ')}|{item_id}"
|
||||
return base64.urlsafe_b64encode(raw.encode("utf-8")).decode("ascii")
|
||||
|
||||
|
||||
def decode_cursor(cursor: str, expected_resource: str) -> Tuple[datetime, uuid.UUID]:
|
||||
try:
|
||||
raw = base64.urlsafe_b64decode(cursor.encode("ascii")).decode("utf-8")
|
||||
resource, ts, item_id = raw.split("|", 2)
|
||||
if resource != expected_resource:
|
||||
raise ValueError("wrong resource")
|
||||
created_at = datetime.strptime(ts, "%Y-%m-%dT%H:%M:%SZ").replace(
|
||||
tzinfo=timezone.utc
|
||||
)
|
||||
return created_at, uuid.UUID(item_id)
|
||||
except Exception as exc:
|
||||
raise ValueError("invalid cursor") from exc
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
title="MTUSI Messenger lab3",
|
||||
docs_url=None,
|
||||
redoc_url=None,
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def add_request_id_and_gate(request: Request, call_next):
|
||||
rid = request_id_from(request)
|
||||
request.state.request_id = rid
|
||||
|
||||
path = request.url.path
|
||||
if not ready and path not in ("/health/live",):
|
||||
if path == "/health/ready":
|
||||
return error_response(503, "unavailable", "Shutting down", rid)
|
||||
return error_response(503, "unavailable", "Service unavailable", rid)
|
||||
|
||||
response = await call_next(request)
|
||||
if "X-Request-Id" not in response.headers:
|
||||
response.headers["X-Request-Id"] = rid
|
||||
response.headers["X-Instance-Id"] = INSTANCE_ID
|
||||
log.info(
|
||||
"%s %s -> %s request_id=%s instance_id=%s",
|
||||
request.method,
|
||||
path,
|
||||
response.status_code,
|
||||
rid,
|
||||
INSTANCE_ID,
|
||||
)
|
||||
return response
|
||||
|
||||
|
||||
class ApiError(Exception):
|
||||
def __init__(
|
||||
self,
|
||||
status: int,
|
||||
code: str,
|
||||
message: str,
|
||||
www_authenticate: Optional[str] = None,
|
||||
):
|
||||
self.status = status
|
||||
self.code = code
|
||||
self.message = message
|
||||
self.www_authenticate = www_authenticate
|
||||
|
||||
|
||||
@app.exception_handler(ApiError)
|
||||
async def api_error_handler(request: Request, exc: ApiError):
|
||||
rid = getattr(request.state, "request_id", None) or request_id_from(request)
|
||||
return error_response(
|
||||
exc.status, exc.code, exc.message, rid, www_authenticate=exc.www_authenticate
|
||||
)
|
||||
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def validation_handler(request: Request, exc: RequestValidationError):
|
||||
rid = getattr(request.state, "request_id", None) or request_id_from(request)
|
||||
return error_response(400, "invalid_request", "Invalid request", rid)
|
||||
|
||||
|
||||
@app.exception_handler(StarletteHTTPException)
|
||||
async def http_exception_handler(request: Request, exc: StarletteHTTPException):
|
||||
rid = getattr(request.state, "request_id", None) or request_id_from(request)
|
||||
if exc.status_code == 404:
|
||||
return error_response(404, "not_found", "Not found", rid)
|
||||
if exc.status_code == 405:
|
||||
return error_response(400, "invalid_request", "Method not allowed", rid)
|
||||
return error_response(exc.status_code, "invalid_request", "Request failed", rid)
|
||||
|
||||
|
||||
@app.exception_handler(StoreUnavailable)
|
||||
async def store_unavailable_handler(request: Request, exc: StoreUnavailable):
|
||||
rid = getattr(request.state, "request_id", None) or request_id_from(request)
|
||||
return error_response(503, "unavailable", "Database unavailable", rid)
|
||||
|
||||
|
||||
@app.exception_handler(SessionStoreError)
|
||||
async def session_store_unavailable_handler(request: Request, exc: SessionStoreError):
|
||||
rid = getattr(request.state, "request_id", None) or request_id_from(request)
|
||||
return error_response(503, "unavailable", "Session store unavailable", rid)
|
||||
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
async def unhandled_handler(request: Request, exc: Exception):
|
||||
rid = getattr(request.state, "request_id", None) or request_id_from(request)
|
||||
log.exception("unhandled error request_id=%s", rid)
|
||||
return error_response(500, "internal_error", "Internal error", rid)
|
||||
|
||||
|
||||
async def read_json_object(request: Request) -> Any:
|
||||
content_type = request.headers.get("content-type", "")
|
||||
media = content_type.split(";")[0].strip().lower()
|
||||
if media and media != "application/json":
|
||||
raise ApiError(415, "unsupported_media_type", "Unsupported media type")
|
||||
try:
|
||||
body = await asyncio.wait_for(request.body(), timeout=BODY_READ_TIMEOUT_S)
|
||||
except asyncio.TimeoutError:
|
||||
raise ApiError(400, "invalid_request", "Request body timeout")
|
||||
if not body:
|
||||
raise ApiError(400, "invalid_request", "Empty body")
|
||||
try:
|
||||
text = body.decode("utf-8")
|
||||
# Reject trailing garbage after one JSON value (read to EOF).
|
||||
data, end = json.JSONDecoder().raw_decode(text)
|
||||
if text[end:].strip():
|
||||
raise ApiError(400, "invalid_request", "Malformed JSON")
|
||||
except ApiError:
|
||||
raise
|
||||
except (UnicodeDecodeError, json.JSONDecodeError, ValueError):
|
||||
raise ApiError(400, "invalid_request", "Malformed JSON")
|
||||
if not isinstance(data, dict):
|
||||
raise ApiError(400, "invalid_request", "JSON object required")
|
||||
return data
|
||||
|
||||
def require_bearer(authorization: Optional[str]) -> Tuple[str, str]:
|
||||
"""Return (user_id, token). X-User-Id is ignored."""
|
||||
bearer_challenge = 'Bearer realm="messenger"'
|
||||
if not authorization or not authorization.strip():
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Missing bearer token", www_authenticate=bearer_challenge
|
||||
)
|
||||
parts = authorization.strip().split(None, 1)
|
||||
if len(parts) != 2 or parts[0].lower() != "bearer" or not parts[1].strip():
|
||||
# Basic on business endpoints must not work.
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid authorization", www_authenticate=bearer_challenge
|
||||
)
|
||||
token = parts[1].strip()
|
||||
try:
|
||||
session = sessions.get_session(token)
|
||||
except SessionStoreError:
|
||||
raise ApiError(503, "unavailable", "Session store unavailable")
|
||||
if session is None:
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid or expired session", www_authenticate=bearer_challenge
|
||||
)
|
||||
user_id = session["user_id"]
|
||||
user = store.get_user(user_id)
|
||||
if user is None:
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid or expired session", www_authenticate=bearer_challenge
|
||||
)
|
||||
return user_id, token
|
||||
|
||||
|
||||
def require_chat_member(chat_id_raw: str, user_id: str):
|
||||
chat_id = parse_uuid(chat_id_raw)
|
||||
if chat_id is None:
|
||||
raise ApiError(400, "invalid_request", "Invalid chat_id")
|
||||
chat = store.get_chat(chat_id)
|
||||
if chat is None or user_id not in chat.member_ids:
|
||||
raise ApiError(404, "not_found", "Chat not found")
|
||||
return chat
|
||||
|
||||
|
||||
def parse_limit(limit: Optional[int]) -> int:
|
||||
if limit is None:
|
||||
return 50
|
||||
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 100:
|
||||
raise ApiError(400, "invalid_request", "Invalid limit")
|
||||
return limit
|
||||
|
||||
|
||||
def validate_display_text(value: Any, field: str, max_len: int) -> str:
|
||||
if not isinstance(value, str) or is_blank(value) or has_nul(value):
|
||||
raise ApiError(400, "invalid_request", f"Invalid {field}")
|
||||
if has_lone_surrogate(value):
|
||||
raise ApiError(400, "invalid_request", f"Invalid {field}")
|
||||
length = codepoint_len(value)
|
||||
if length < 1 or length > max_len:
|
||||
raise ApiError(400, "invalid_request", f"Invalid {field}")
|
||||
return value
|
||||
|
||||
|
||||
def validate_password(value: Any) -> str:
|
||||
# Required strings must not be whitespace-only (contracts/README).
|
||||
if not isinstance(value, str) or is_blank(value) or has_nul(value):
|
||||
raise ApiError(400, "invalid_request", "Invalid password")
|
||||
if has_lone_surrogate(value):
|
||||
raise ApiError(400, "invalid_request", "Invalid password")
|
||||
length = codepoint_len(value)
|
||||
if length < PASSWORD_MIN or length > PASSWORD_MAX:
|
||||
raise ApiError(400, "invalid_request", "Invalid password")
|
||||
return value
|
||||
|
||||
|
||||
def parse_basic_credentials(authorization: Optional[str]) -> Tuple[str, str]:
|
||||
challenge = 'Basic realm="messenger", charset="UTF-8"'
|
||||
if not authorization or not authorization.strip():
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
|
||||
)
|
||||
parts = authorization.strip().split(None, 1)
|
||||
if len(parts) != 2 or parts[0].lower() != "basic":
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
|
||||
)
|
||||
try:
|
||||
decoded = base64.b64decode(parts[1].strip(), validate=True).decode("utf-8")
|
||||
except Exception:
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
|
||||
)
|
||||
if ":" not in decoded:
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
|
||||
)
|
||||
username, password = decoded.split(":", 1)
|
||||
# Username ASCII-only per contract; password may contain ':'.
|
||||
if not username or any(ord(ch) > 127 for ch in username):
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
|
||||
)
|
||||
return username, password
|
||||
|
||||
|
||||
@app.get("/health/live")
|
||||
async def liveness(request: Request):
|
||||
rid = request.state.request_id
|
||||
return JSONResponse(content={"status": "ok"}, headers=common_headers(rid))
|
||||
|
||||
|
||||
@app.get("/health/ready")
|
||||
async def readiness(request: Request):
|
||||
rid = request.state.request_id
|
||||
if not ready:
|
||||
return error_response(503, "unavailable", "Not ready", rid)
|
||||
if not db.ping() or not sessions.ping():
|
||||
return error_response(503, "unavailable", "Dependency unavailable", rid)
|
||||
return JSONResponse(content={"status": "ok"}, headers=common_headers(rid))
|
||||
|
||||
|
||||
@app.post("/api/v1/users")
|
||||
async def register_user(request: Request):
|
||||
rid = request.state.request_id
|
||||
data = await read_json_object(request)
|
||||
allowed = {"username", "display_name", "password"}
|
||||
if set(data.keys()) - allowed:
|
||||
raise ApiError(400, "invalid_request", "Unexpected fields")
|
||||
if "username" not in data or "display_name" not in data or "password" not in data:
|
||||
raise ApiError(400, "invalid_request", "Missing required fields")
|
||||
username = data["username"]
|
||||
display_name = data["display_name"]
|
||||
if (
|
||||
not isinstance(username, str)
|
||||
or is_blank(username)
|
||||
or has_lone_surrogate(username)
|
||||
or not USERNAME_RE.match(username)
|
||||
):
|
||||
raise ApiError(400, "invalid_request", "Invalid username")
|
||||
display_name = validate_display_text(display_name, "display_name", 100)
|
||||
password = validate_password(data["password"])
|
||||
try:
|
||||
user = store.create_user(username, display_name, password)
|
||||
except KeyError:
|
||||
raise ApiError(409, "conflict", "Username already taken")
|
||||
return JSONResponse(
|
||||
status_code=201,
|
||||
content=user.to_dict(),
|
||||
headers=common_headers(rid),
|
||||
)
|
||||
|
||||
|
||||
@app.post("/api/v1/auth/sessions")
|
||||
async def create_session(request: Request):
|
||||
rid = request.state.request_id
|
||||
challenge = 'Basic realm="messenger", charset="UTF-8"'
|
||||
# Contract: Basic only, no JSON body (even with valid credentials).
|
||||
try:
|
||||
raw_body = await asyncio.wait_for(request.body(), timeout=BODY_READ_TIMEOUT_S)
|
||||
except asyncio.TimeoutError:
|
||||
raise ApiError(400, "invalid_request", "Request body timeout")
|
||||
if raw_body and raw_body.strip():
|
||||
raise ApiError(400, "invalid_request", "Login must not include a body")
|
||||
username, password = parse_basic_credentials(
|
||||
request.headers.get("authorization")
|
||||
)
|
||||
# Uniform 401 for wrong user / wrong password (no user enumeration).
|
||||
user = store.get_user_by_username(username)
|
||||
if user is None or not store.verify_password(user.password_hash, password):
|
||||
raise ApiError(
|
||||
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
|
||||
)
|
||||
try:
|
||||
session = sessions.create_session(user.id)
|
||||
except SessionStoreError:
|
||||
raise ApiError(503, "unavailable", "Session store unavailable")
|
||||
body = {
|
||||
"token": session["token"],
|
||||
"token_type": "Bearer",
|
||||
"expires_at": session["expires_at"],
|
||||
"user": user.to_dict(),
|
||||
}
|
||||
return JSONResponse(
|
||||
status_code=201, content=body, headers=common_headers(rid)
|
||||
)
|
||||
|
||||
|
||||
@app.delete("/api/v1/auth/sessions/current")
|
||||
async def logout(
|
||||
request: Request,
|
||||
authorization: Optional[str] = Header(default=None, alias="Authorization"),
|
||||
):
|
||||
rid = request.state.request_id
|
||||
_user_id, token = require_bearer(authorization)
|
||||
try:
|
||||
sessions.revoke_session(token)
|
||||
except SessionStoreError:
|
||||
raise ApiError(503, "unavailable", "Session store unavailable")
|
||||
return Response(status_code=204, headers=common_headers(rid))
|
||||
|
||||
|
||||
@app.get("/api/v1/users/me")
|
||||
async def current_user(
|
||||
request: Request,
|
||||
authorization: Optional[str] = Header(default=None, alias="Authorization"),
|
||||
):
|
||||
rid = request.state.request_id
|
||||
user_id, _token = require_bearer(authorization)
|
||||
user = store.get_user(user_id)
|
||||
assert user is not None
|
||||
return JSONResponse(content=user.to_dict(), headers=common_headers(rid))
|
||||
|
||||
|
||||
@app.post("/api/v1/chats")
|
||||
async def create_chat(
|
||||
request: Request,
|
||||
authorization: Optional[str] = Header(default=None, alias="Authorization"),
|
||||
):
|
||||
rid = request.state.request_id
|
||||
user_id, _token = require_bearer(authorization)
|
||||
data = await read_json_object(request)
|
||||
allowed = {"title", "member_ids"}
|
||||
if set(data.keys()) - allowed:
|
||||
raise ApiError(400, "invalid_request", "Unexpected fields")
|
||||
if "title" not in data or "member_ids" not in data:
|
||||
raise ApiError(400, "invalid_request", "Missing required fields")
|
||||
title = validate_display_text(data["title"], "title", 100)
|
||||
member_ids = data["member_ids"]
|
||||
if not isinstance(member_ids, list) or not member_ids or len(member_ids) > 100:
|
||||
raise ApiError(400, "invalid_request", "Invalid member_ids")
|
||||
parsed: List[str] = []
|
||||
seen = set()
|
||||
for mid in member_ids:
|
||||
if not isinstance(mid, str):
|
||||
raise ApiError(400, "invalid_request", "Invalid member_ids")
|
||||
uid = parse_uuid(mid)
|
||||
if uid is None:
|
||||
raise ApiError(400, "invalid_request", "Invalid member_ids")
|
||||
if uid in seen:
|
||||
raise ApiError(400, "invalid_request", "Duplicate member_ids")
|
||||
seen.add(uid)
|
||||
parsed.append(uid)
|
||||
if user_id not in seen:
|
||||
raise ApiError(400, "invalid_request", "Caller must be a member")
|
||||
try:
|
||||
chat = store.create_chat(title, parsed)
|
||||
except KeyError:
|
||||
raise ApiError(400, "invalid_request", "Unknown member")
|
||||
return JSONResponse(
|
||||
status_code=201,
|
||||
content=chat.to_dict(),
|
||||
headers=common_headers(rid),
|
||||
)
|
||||
|
||||
|
||||
@app.get("/api/v1/chats")
|
||||
async def list_chats(
|
||||
request: Request,
|
||||
authorization: Optional[str] = Header(default=None, alias="Authorization"),
|
||||
limit: Optional[int] = Query(default=None),
|
||||
cursor: Optional[str] = Query(default=None),
|
||||
):
|
||||
rid = request.state.request_id
|
||||
user_id, _token = require_bearer(authorization)
|
||||
lim = parse_limit(limit)
|
||||
resource = f"chats:{user_id}"
|
||||
cursor_key = None
|
||||
if cursor is not None:
|
||||
if not cursor:
|
||||
raise ApiError(400, "invalid_request", "Invalid cursor")
|
||||
try:
|
||||
cursor_ts, cursor_id = decode_cursor(cursor, resource)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Invalid cursor")
|
||||
if not store.chat_exists_for_user(user_id, cursor_ts, cursor_id):
|
||||
raise ApiError(400, "invalid_request", "Invalid cursor")
|
||||
cursor_key = (cursor_ts, cursor_id)
|
||||
chats, more = store.list_chats_for_user(user_id, lim, cursor_key)
|
||||
next_cursor = None
|
||||
if more and chats:
|
||||
last = chats[-1]
|
||||
next_cursor = encode_cursor(resource, last.created_at, last.id)
|
||||
return JSONResponse(
|
||||
content={"items": [c.to_dict() for c in chats], "next_cursor": next_cursor},
|
||||
headers=common_headers(rid),
|
||||
)
|
||||
|
||||
|
||||
@app.get("/api/v1/chats/{chat_id}")
|
||||
async def get_chat(
|
||||
chat_id: str,
|
||||
request: Request,
|
||||
authorization: Optional[str] = Header(default=None, alias="Authorization"),
|
||||
):
|
||||
rid = request.state.request_id
|
||||
user_id, _token = require_bearer(authorization)
|
||||
chat = require_chat_member(chat_id, user_id)
|
||||
return JSONResponse(content=chat.to_dict(), headers=common_headers(rid))
|
||||
|
||||
|
||||
def parse_idempotency_key(raw: Optional[str]) -> str:
|
||||
if raw is None or not raw.strip() or not IDEMPOTENCY_KEY_RE.match(raw.strip()):
|
||||
raise ApiError(400, "invalid_request", "Invalid or missing Idempotency-Key")
|
||||
return raw.strip()
|
||||
|
||||
|
||||
@app.post("/api/v1/chats/{chat_id}/messages")
|
||||
async def create_message(
|
||||
chat_id: str,
|
||||
request: Request,
|
||||
authorization: Optional[str] = Header(default=None, alias="Authorization"),
|
||||
idempotency_key: Optional[str] = Header(default=None, alias="Idempotency-Key"),
|
||||
):
|
||||
rid = request.state.request_id
|
||||
user_id, _token = require_bearer(authorization)
|
||||
chat = require_chat_member(chat_id, user_id)
|
||||
key = parse_idempotency_key(idempotency_key)
|
||||
data = await read_json_object(request)
|
||||
# Contract: absent attachment_ids ≡ []; lab3 has no uploads, so only [] is accepted.
|
||||
allowed = {"text", "attachment_ids"}
|
||||
if set(data.keys()) - allowed:
|
||||
raise ApiError(400, "invalid_request", "Unexpected fields")
|
||||
if "text" not in data:
|
||||
raise ApiError(400, "invalid_request", "Missing text")
|
||||
if "attachment_ids" in data:
|
||||
aids = data["attachment_ids"]
|
||||
if not isinstance(aids, list) or aids:
|
||||
raise ApiError(400, "invalid_request", "Invalid attachment_ids")
|
||||
text = data["text"]
|
||||
if (
|
||||
not isinstance(text, str)
|
||||
or is_blank(text)
|
||||
or has_nul(text)
|
||||
or has_lone_surrogate(text)
|
||||
or codepoint_len(text) > 4000
|
||||
):
|
||||
raise ApiError(400, "invalid_request", "Invalid text")
|
||||
path = f"/api/v1/chats/{chat.id}/messages"
|
||||
try:
|
||||
message, _replayed = store.create_message_idempotent(
|
||||
chat.id, user_id, text, "POST", path, key
|
||||
)
|
||||
except KeyError:
|
||||
raise ApiError(409, "conflict", "Idempotency-Key reused with different body")
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Invalid text")
|
||||
return JSONResponse(
|
||||
status_code=201,
|
||||
content=message.to_dict(),
|
||||
headers=common_headers(rid),
|
||||
)
|
||||
|
||||
|
||||
@app.get("/api/v1/chats/{chat_id}/messages")
|
||||
async def list_messages(
|
||||
chat_id: str,
|
||||
request: Request,
|
||||
authorization: Optional[str] = Header(default=None, alias="Authorization"),
|
||||
limit: Optional[int] = Query(default=None),
|
||||
cursor: Optional[str] = Query(default=None),
|
||||
):
|
||||
rid = request.state.request_id
|
||||
user_id, _token = require_bearer(authorization)
|
||||
chat = require_chat_member(chat_id, user_id)
|
||||
lim = parse_limit(limit)
|
||||
resource = f"messages:{chat.id}"
|
||||
cursor_key = None
|
||||
if cursor is not None:
|
||||
if not cursor:
|
||||
raise ApiError(400, "invalid_request", "Invalid cursor")
|
||||
try:
|
||||
cursor_ts, cursor_id = decode_cursor(cursor, resource)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Invalid cursor")
|
||||
if not store.message_exists(chat.id, cursor_ts, cursor_id):
|
||||
raise ApiError(400, "invalid_request", "Invalid cursor")
|
||||
cursor_key = (cursor_ts, cursor_id)
|
||||
messages, has_more = store.list_messages(chat.id, lim, cursor_key)
|
||||
next_cursor = None
|
||||
if has_more and messages:
|
||||
last = messages[-1]
|
||||
next_cursor = encode_cursor(resource, last.created_at, last.id)
|
||||
return JSONResponse(
|
||||
content={
|
||||
"items": [m.to_dict() for m in messages],
|
||||
"next_cursor": next_cursor,
|
||||
},
|
||||
headers=common_headers(rid),
|
||||
)
|
||||
|
||||
|
||||
def create_app() -> FastAPI:
|
||||
return app
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Runtime configuration from environment / secret files."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import socket
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _read_secret(env_key: str, file_key: str, default: str | None = None) -> str:
|
||||
file_path = os.environ.get(file_key, "").strip()
|
||||
if file_path:
|
||||
path = Path(file_path)
|
||||
if path.is_file():
|
||||
return path.read_text(encoding="utf-8").strip()
|
||||
value = os.environ.get(env_key, "").strip()
|
||||
if value:
|
||||
return value
|
||||
if default is not None:
|
||||
return default
|
||||
raise RuntimeError(f"Missing secret: set {env_key} or {file_key}")
|
||||
|
||||
|
||||
def database_url() -> str:
|
||||
return _read_secret("DATABASE_URL", "DATABASE_URL_FILE")
|
||||
|
||||
|
||||
def cache_url() -> str:
|
||||
return _read_secret("CACHE_URL", "CACHE_URL_FILE")
|
||||
|
||||
|
||||
def session_ttl_seconds() -> int:
|
||||
raw = os.environ.get("SESSION_TTL_SECONDS", "3600").strip()
|
||||
try:
|
||||
value = int(raw)
|
||||
except ValueError as exc:
|
||||
raise RuntimeError("SESSION_TTL_SECONDS must be an integer") from exc
|
||||
if value < 1:
|
||||
raise RuntimeError("SESSION_TTL_SECONDS must be >= 1")
|
||||
return value
|
||||
|
||||
|
||||
def idempotency_ttl_seconds() -> int:
|
||||
raw = os.environ.get("IDEMPOTENCY_TTL_SECONDS", "86400").strip()
|
||||
try:
|
||||
value = int(raw)
|
||||
except ValueError as exc:
|
||||
raise RuntimeError("IDEMPOTENCY_TTL_SECONDS must be an integer") from exc
|
||||
if value < 1:
|
||||
raise RuntimeError("IDEMPOTENCY_TTL_SECONDS must be >= 1")
|
||||
return value
|
||||
|
||||
|
||||
def instance_id() -> str:
|
||||
"""Stable for process lifetime; used as X-Instance-Id (maps to container hostname)."""
|
||||
override = os.environ.get("INSTANCE_ID", "").strip()
|
||||
if override:
|
||||
return override[:128]
|
||||
return socket.gethostname()[:128]
|
||||
|
||||
|
||||
def app_host() -> str:
|
||||
return os.environ.get("APP_HOST", "0.0.0.0")
|
||||
|
||||
|
||||
def app_port() -> int:
|
||||
return int(os.environ.get("APP_PORT", "8080"))
|
||||
|
||||
|
||||
def migrations_dir() -> Path:
|
||||
override = os.environ.get("MIGRATIONS_DIR", "").strip()
|
||||
if override:
|
||||
return Path(override)
|
||||
here = Path(__file__).resolve()
|
||||
candidates = [
|
||||
Path("/app/migrations"),
|
||||
here.parents[2] / "migrations",
|
||||
Path.cwd() / "migrations",
|
||||
]
|
||||
for path in candidates:
|
||||
if path.is_dir():
|
||||
return path
|
||||
return candidates[0]
|
||||
@@ -0,0 +1,107 @@
|
||||
"""PostgreSQL connection pool and versioned migrations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
from messenger import config
|
||||
|
||||
log = logging.getLogger("messenger.db")
|
||||
|
||||
_pool: Optional[ConnectionPool] = None
|
||||
|
||||
|
||||
def get_pool() -> ConnectionPool:
|
||||
if _pool is None:
|
||||
raise RuntimeError("Database pool is not initialized")
|
||||
return _pool
|
||||
|
||||
|
||||
def init_pool() -> ConnectionPool:
|
||||
global _pool
|
||||
if _pool is not None:
|
||||
return _pool
|
||||
url = config.database_url()
|
||||
_pool = ConnectionPool(
|
||||
conninfo=url,
|
||||
min_size=1,
|
||||
max_size=10,
|
||||
kwargs={
|
||||
"row_factory": dict_row,
|
||||
"connect_timeout": 5,
|
||||
"options": "-c statement_timeout=5000",
|
||||
},
|
||||
open=True,
|
||||
timeout=10,
|
||||
)
|
||||
return _pool
|
||||
|
||||
|
||||
def close_pool() -> None:
|
||||
global _pool
|
||||
if _pool is not None:
|
||||
_pool.close()
|
||||
_pool = None
|
||||
|
||||
|
||||
def _split_sql(script: str) -> List[str]:
|
||||
"""Split SQL file into statements (no procedure bodies in our migrations)."""
|
||||
# Drop line comments.
|
||||
cleaned = re.sub(r"--[^\n]*", "", script)
|
||||
parts = []
|
||||
for chunk in cleaned.split(";"):
|
||||
stmt = chunk.strip()
|
||||
if stmt:
|
||||
parts.append(stmt)
|
||||
return parts
|
||||
|
||||
|
||||
def apply_migrations(migrations_path: Optional[Path] = None) -> None:
|
||||
path = migrations_path or config.migrations_dir()
|
||||
sql_files = sorted(path.glob("*.sql"))
|
||||
if not sql_files:
|
||||
raise RuntimeError(f"No migrations found in {path}")
|
||||
pool = get_pool()
|
||||
with pool.connection() as conn:
|
||||
with conn.transaction():
|
||||
conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version TEXT PRIMARY KEY,
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
)
|
||||
"""
|
||||
)
|
||||
for sql_file in sql_files:
|
||||
version = sql_file.name
|
||||
row = conn.execute(
|
||||
"SELECT 1 AS ok FROM schema_migrations WHERE version = %s",
|
||||
(version,),
|
||||
).fetchone()
|
||||
if row:
|
||||
log.info("migration already applied: %s", version)
|
||||
continue
|
||||
script = sql_file.read_text(encoding="utf-8")
|
||||
for stmt in _split_sql(script):
|
||||
conn.execute(stmt)
|
||||
conn.execute(
|
||||
"INSERT INTO schema_migrations (version) VALUES (%s)",
|
||||
(version,),
|
||||
)
|
||||
log.info("applied migration: %s", version)
|
||||
|
||||
|
||||
def ping() -> bool:
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
conn.execute("SELECT 1")
|
||||
return True
|
||||
except Exception as exc:
|
||||
log.warning("postgres ping failed: %s", type(exc).__name__)
|
||||
return False
|
||||
@@ -0,0 +1,28 @@
|
||||
"""One-shot migrations entrypoint (run once before API replicas)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
|
||||
from messenger import db
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s %(levelname)s %(message)s",
|
||||
)
|
||||
log = logging.getLogger("messenger.migrate")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
db.init_pool()
|
||||
try:
|
||||
db.apply_migrations()
|
||||
log.info("migrations complete")
|
||||
return 0
|
||||
finally:
|
||||
db.close_pool()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,136 @@
|
||||
"""Opaque Bearer sessions in Redis with TTL (durable via Redis AOF)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Optional
|
||||
|
||||
import redis
|
||||
|
||||
from messenger import config
|
||||
|
||||
log = logging.getLogger("messenger.sessions")
|
||||
|
||||
_client: Optional[redis.Redis] = None
|
||||
|
||||
TOKEN_BYTES = 32 # 256-bit entropy
|
||||
KEY_PREFIX = "sess:"
|
||||
|
||||
|
||||
def utc_now() -> datetime:
|
||||
return datetime.now(timezone.utc).replace(microsecond=0)
|
||||
|
||||
|
||||
def format_ts(value: datetime) -> str:
|
||||
if value.tzinfo is None:
|
||||
value = value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
|
||||
def parse_ts(value: str) -> datetime:
|
||||
return datetime.strptime(value, "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def init_client() -> redis.Redis:
|
||||
global _client
|
||||
if _client is not None:
|
||||
return _client
|
||||
url = config.cache_url()
|
||||
_client = redis.Redis.from_url(
|
||||
url,
|
||||
decode_responses=True,
|
||||
socket_connect_timeout=3,
|
||||
socket_timeout=3,
|
||||
health_check_interval=30,
|
||||
)
|
||||
return _client
|
||||
|
||||
|
||||
def close_client() -> None:
|
||||
global _client
|
||||
if _client is not None:
|
||||
try:
|
||||
_client.close()
|
||||
except Exception:
|
||||
pass
|
||||
_client = None
|
||||
|
||||
|
||||
def get_client() -> redis.Redis:
|
||||
if _client is None:
|
||||
raise RuntimeError("Redis client is not initialized")
|
||||
return _client
|
||||
|
||||
|
||||
def ping() -> bool:
|
||||
try:
|
||||
return bool(get_client().ping())
|
||||
except Exception as exc:
|
||||
log.warning("redis ping failed: %s", type(exc).__name__)
|
||||
return False
|
||||
|
||||
|
||||
class SessionStoreError(Exception):
|
||||
"""Raised when Redis is unavailable for session operations."""
|
||||
|
||||
|
||||
def create_session(user_id: str) -> dict:
|
||||
ttl = config.session_ttl_seconds()
|
||||
token = secrets.token_urlsafe(TOKEN_BYTES)
|
||||
now = utc_now()
|
||||
expires_at = now + timedelta(seconds=ttl)
|
||||
payload = {
|
||||
"user_id": user_id,
|
||||
"expires_at": format_ts(expires_at),
|
||||
"created_at": format_ts(now),
|
||||
}
|
||||
key = KEY_PREFIX + token
|
||||
try:
|
||||
client = get_client()
|
||||
# SET with EX: Redis is sole session store; AOF keeps it across restart.
|
||||
client.set(key, json.dumps(payload), ex=ttl)
|
||||
except redis.RedisError as exc:
|
||||
raise SessionStoreError("session store unavailable") from exc
|
||||
return {
|
||||
"token": token,
|
||||
"token_type": "Bearer",
|
||||
"expires_at": payload["expires_at"],
|
||||
"user_id": user_id,
|
||||
}
|
||||
|
||||
|
||||
def get_session(token: str) -> Optional[dict]:
|
||||
if not token:
|
||||
return None
|
||||
key = KEY_PREFIX + token
|
||||
try:
|
||||
raw = get_client().get(key)
|
||||
except redis.RedisError as exc:
|
||||
raise SessionStoreError("session store unavailable") from exc
|
||||
if not raw:
|
||||
return None
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
expires_at = parse_ts(data["expires_at"])
|
||||
except (ValueError, KeyError, TypeError, json.JSONDecodeError):
|
||||
return None
|
||||
if expires_at <= utc_now():
|
||||
try:
|
||||
get_client().delete(key)
|
||||
except redis.RedisError:
|
||||
pass
|
||||
return None
|
||||
return data
|
||||
|
||||
|
||||
def revoke_session(token: str) -> bool:
|
||||
if not token:
|
||||
return False
|
||||
key = KEY_PREFIX + token
|
||||
try:
|
||||
return bool(get_client().delete(key))
|
||||
except redis.RedisError as exc:
|
||||
raise SessionStoreError("session store unavailable") from exc
|
||||
@@ -0,0 +1,480 @@
|
||||
"""PostgreSQL repository: users, chats, membership, messages, idempotency."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Any, List, Optional, Tuple
|
||||
|
||||
from argon2 import PasswordHasher
|
||||
from argon2.exceptions import VerifyMismatchError
|
||||
from psycopg import Error as PsycopgError
|
||||
from psycopg import errors as pg_errors
|
||||
from psycopg.types.json import Jsonb
|
||||
|
||||
from messenger import config
|
||||
from messenger.db import get_pool
|
||||
|
||||
_ph = PasswordHasher()
|
||||
|
||||
|
||||
class StoreUnavailable(Exception):
|
||||
"""Raised when PostgreSQL is unreachable or timed out."""
|
||||
|
||||
|
||||
def utc_now() -> datetime:
|
||||
return datetime.now(timezone.utc).replace(microsecond=0)
|
||||
|
||||
|
||||
def format_ts(value: datetime) -> str:
|
||||
if value.tzinfo is None:
|
||||
value = value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
|
||||
@dataclass
|
||||
class User:
|
||||
id: str
|
||||
username: str
|
||||
display_name: str
|
||||
created_at: datetime
|
||||
password_hash: str = ""
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"id": self.id,
|
||||
"username": self.username,
|
||||
"display_name": self.display_name,
|
||||
"created_at": format_ts(self.created_at),
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Chat:
|
||||
id: str
|
||||
title: str
|
||||
member_ids: List[str]
|
||||
created_at: datetime
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"id": self.id,
|
||||
"title": self.title,
|
||||
"member_ids": list(self.member_ids),
|
||||
"created_at": format_ts(self.created_at),
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Message:
|
||||
id: str
|
||||
chat_id: str
|
||||
sender_id: str
|
||||
text: str
|
||||
created_at: datetime
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"id": self.id,
|
||||
"chat_id": self.chat_id,
|
||||
"sender_id": self.sender_id,
|
||||
"text": self.text,
|
||||
"created_at": format_ts(self.created_at),
|
||||
}
|
||||
|
||||
|
||||
def _row_user(row: dict) -> User:
|
||||
return User(
|
||||
id=str(row["id"]),
|
||||
username=row["username"],
|
||||
display_name=row["display_name"],
|
||||
created_at=row["created_at"],
|
||||
password_hash=row.get("password_hash", ""),
|
||||
)
|
||||
|
||||
|
||||
def _row_message(row: dict) -> Message:
|
||||
return Message(
|
||||
id=str(row["id"]),
|
||||
chat_id=str(row["chat_id"]),
|
||||
sender_id=str(row["sender_id"]),
|
||||
text=row["text"],
|
||||
created_at=row["created_at"],
|
||||
)
|
||||
|
||||
|
||||
class Store:
|
||||
def hash_password(self, password: str) -> str:
|
||||
return _ph.hash(password)
|
||||
|
||||
def verify_password(self, password_hash: str, password: str) -> bool:
|
||||
try:
|
||||
return _ph.verify(password_hash, password)
|
||||
except VerifyMismatchError:
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def create_user(self, username: str, display_name: str, password: str) -> User:
|
||||
user_id = uuid.uuid4()
|
||||
created = utc_now()
|
||||
password_hash = self.hash_password(password)
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
with conn.transaction():
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO users (id, username, display_name, password_hash, created_at)
|
||||
VALUES (%s, %s, %s, %s, %s)
|
||||
""",
|
||||
(user_id, username, display_name, password_hash, created),
|
||||
)
|
||||
except (pg_errors.UniqueViolation, pg_errors.IntegrityError) as exc:
|
||||
raise KeyError("username_taken") from exc
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
return User(
|
||||
id=str(user_id),
|
||||
username=username,
|
||||
display_name=display_name,
|
||||
created_at=created,
|
||||
)
|
||||
|
||||
def get_user(self, user_id: str) -> Optional[User]:
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
row = conn.execute(
|
||||
"""
|
||||
SELECT id, username, display_name, password_hash, created_at
|
||||
FROM users WHERE id = %s
|
||||
""",
|
||||
(user_id,),
|
||||
).fetchone()
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
return _row_user(row) if row else None
|
||||
|
||||
def get_user_by_username(self, username: str) -> Optional[User]:
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
row = conn.execute(
|
||||
"""
|
||||
SELECT id, username, display_name, password_hash, created_at
|
||||
FROM users WHERE username = %s
|
||||
""",
|
||||
(username,),
|
||||
).fetchone()
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
return _row_user(row) if row else None
|
||||
|
||||
def create_chat(self, title: str, member_ids: List[str]) -> Chat:
|
||||
chat_id = uuid.uuid4()
|
||||
created = utc_now()
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
with conn.transaction():
|
||||
# Atomic membership check + insert; rollback if any member missing.
|
||||
rows = conn.execute(
|
||||
"SELECT id FROM users WHERE id = ANY(%s::uuid[])",
|
||||
(member_ids,),
|
||||
).fetchall()
|
||||
found = {str(r["id"]) for r in rows}
|
||||
if found != set(member_ids):
|
||||
raise KeyError("unknown_member")
|
||||
conn.execute(
|
||||
"INSERT INTO chats (id, title, created_at) VALUES (%s, %s, %s)",
|
||||
(chat_id, title, created),
|
||||
)
|
||||
for mid in member_ids:
|
||||
conn.execute(
|
||||
"INSERT INTO chat_members (chat_id, user_id) VALUES (%s, %s)",
|
||||
(chat_id, mid),
|
||||
)
|
||||
except KeyError:
|
||||
raise
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
return Chat(
|
||||
id=str(chat_id),
|
||||
title=title,
|
||||
member_ids=list(member_ids),
|
||||
created_at=created,
|
||||
)
|
||||
|
||||
def get_chat(self, chat_id: str) -> Optional[Chat]:
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
row = conn.execute(
|
||||
"SELECT id, title, created_at FROM chats WHERE id = %s",
|
||||
(chat_id,),
|
||||
).fetchone()
|
||||
if not row:
|
||||
return None
|
||||
members = conn.execute(
|
||||
"""
|
||||
SELECT user_id FROM chat_members
|
||||
WHERE chat_id = %s
|
||||
ORDER BY user_id
|
||||
""",
|
||||
(chat_id,),
|
||||
).fetchall()
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
return Chat(
|
||||
id=str(row["id"]),
|
||||
title=row["title"],
|
||||
member_ids=[str(m["user_id"]) for m in members],
|
||||
created_at=row["created_at"],
|
||||
)
|
||||
|
||||
def list_chats_for_user(
|
||||
self,
|
||||
user_id: str,
|
||||
limit: int,
|
||||
cursor: Optional[Tuple[datetime, uuid.UUID]],
|
||||
) -> Tuple[List[Chat], Optional[str]]:
|
||||
params: list = [user_id]
|
||||
where = "cm.user_id = %s"
|
||||
if cursor is not None:
|
||||
where += " AND (c.created_at, c.id) > (%s, %s)"
|
||||
params.extend([cursor[0], cursor[1]])
|
||||
params.append(limit + 1)
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
rows = conn.execute(
|
||||
f"""
|
||||
SELECT c.id, c.title, c.created_at
|
||||
FROM chats c
|
||||
JOIN chat_members cm ON cm.chat_id = c.id
|
||||
WHERE {where}
|
||||
ORDER BY c.created_at ASC, c.id ASC
|
||||
LIMIT %s
|
||||
""",
|
||||
params,
|
||||
).fetchall()
|
||||
chats: List[Chat] = []
|
||||
for row in rows[:limit]:
|
||||
members = conn.execute(
|
||||
"""
|
||||
SELECT user_id FROM chat_members
|
||||
WHERE chat_id = %s
|
||||
ORDER BY user_id
|
||||
""",
|
||||
(row["id"],),
|
||||
).fetchall()
|
||||
chats.append(
|
||||
Chat(
|
||||
id=str(row["id"]),
|
||||
title=row["title"],
|
||||
member_ids=[str(m["user_id"]) for m in members],
|
||||
created_at=row["created_at"],
|
||||
)
|
||||
)
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
has_more = len(rows) > limit
|
||||
return chats, "more" if has_more else None
|
||||
|
||||
@staticmethod
|
||||
def request_hash_for_message(text: str) -> str:
|
||||
# Canonical form: sorted keys; absent attachment_ids ≡ [].
|
||||
canonical = json.dumps(
|
||||
{"attachment_ids": [], "text": text},
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
||||
|
||||
def create_message(self, chat_id: str, sender_id: str, text: str) -> Message:
|
||||
msg_id = uuid.uuid4()
|
||||
created = utc_now()
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
with conn.transaction():
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
VALUES (%s, %s, %s, %s, %s)
|
||||
""",
|
||||
(msg_id, chat_id, sender_id, text, created),
|
||||
)
|
||||
except pg_errors.DataError as exc:
|
||||
raise ValueError("invalid_text") from exc
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
return Message(
|
||||
id=str(msg_id),
|
||||
chat_id=chat_id,
|
||||
sender_id=sender_id,
|
||||
text=text,
|
||||
created_at=created,
|
||||
)
|
||||
|
||||
def create_message_idempotent(
|
||||
self,
|
||||
chat_id: str,
|
||||
sender_id: str,
|
||||
text: str,
|
||||
method: str,
|
||||
path: str,
|
||||
idem_key: str,
|
||||
) -> Tuple[Message, bool]:
|
||||
"""
|
||||
Create message with shared idempotency across replicas.
|
||||
Returns (message, replayed).
|
||||
Raises KeyError('conflict') when same key was used with another body.
|
||||
"""
|
||||
req_hash = self.request_hash_for_message(text)
|
||||
ttl = timedelta(seconds=config.idempotency_ttl_seconds())
|
||||
# Transaction-scoped advisory lock serializes same-key concurrent POSTs.
|
||||
lock_material = f"{sender_id}|{method}|{path}|{idem_key}"
|
||||
lock_key = int(
|
||||
hashlib.sha256(lock_material.encode("utf-8")).hexdigest()[:15], 16
|
||||
)
|
||||
|
||||
def _load_existing(conn) -> Optional[dict]:
|
||||
return conn.execute(
|
||||
"""
|
||||
SELECT request_hash, response_status, response_body, created_at
|
||||
FROM idempotency_records
|
||||
WHERE user_id = %s AND method = %s AND path = %s AND idem_key = %s
|
||||
""",
|
||||
(sender_id, method, path, idem_key),
|
||||
).fetchone()
|
||||
|
||||
def _message_from_body(body: Any) -> Message:
|
||||
return Message(
|
||||
id=str(body["id"]),
|
||||
chat_id=str(body["chat_id"]),
|
||||
sender_id=str(body["sender_id"]),
|
||||
text=body["text"],
|
||||
created_at=datetime.strptime(
|
||||
body["created_at"], "%Y-%m-%dT%H:%M:%SZ"
|
||||
).replace(tzinfo=timezone.utc),
|
||||
)
|
||||
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
with conn.transaction():
|
||||
conn.execute("SELECT pg_advisory_xact_lock(%s)", (lock_key,))
|
||||
existing = _load_existing(conn)
|
||||
if existing is not None:
|
||||
if existing["created_at"] >= utc_now() - ttl:
|
||||
if existing["request_hash"] != req_hash:
|
||||
raise KeyError("conflict")
|
||||
return _message_from_body(existing["response_body"]), True
|
||||
conn.execute(
|
||||
"""
|
||||
DELETE FROM idempotency_records
|
||||
WHERE user_id = %s AND method = %s AND path = %s AND idem_key = %s
|
||||
""",
|
||||
(sender_id, method, path, idem_key),
|
||||
)
|
||||
|
||||
msg_id = uuid.uuid4()
|
||||
created = utc_now()
|
||||
message = Message(
|
||||
id=str(msg_id),
|
||||
chat_id=chat_id,
|
||||
sender_id=sender_id,
|
||||
text=text,
|
||||
created_at=created,
|
||||
)
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
VALUES (%s, %s, %s, %s, %s)
|
||||
""",
|
||||
(msg_id, chat_id, sender_id, text, created),
|
||||
)
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO idempotency_records (
|
||||
user_id, method, path, idem_key,
|
||||
request_hash, response_status, response_body, created_at
|
||||
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)
|
||||
""",
|
||||
(
|
||||
sender_id,
|
||||
method,
|
||||
path,
|
||||
idem_key,
|
||||
req_hash,
|
||||
201,
|
||||
Jsonb(message.to_dict()),
|
||||
created,
|
||||
),
|
||||
)
|
||||
return message, False
|
||||
except KeyError:
|
||||
raise
|
||||
except pg_errors.DataError as exc:
|
||||
raise ValueError("invalid_text") from exc
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
|
||||
def list_messages(
|
||||
self,
|
||||
chat_id: str,
|
||||
limit: int,
|
||||
cursor: Optional[Tuple[datetime, uuid.UUID]],
|
||||
) -> Tuple[List[Message], bool]:
|
||||
params: list = [chat_id]
|
||||
where = "chat_id = %s"
|
||||
if cursor is not None:
|
||||
where += " AND (created_at, id) > (%s, %s)"
|
||||
params.extend([cursor[0], cursor[1]])
|
||||
params.append(limit + 1)
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
rows = conn.execute(
|
||||
f"""
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE {where}
|
||||
ORDER BY created_at ASC, id ASC
|
||||
LIMIT %s
|
||||
""",
|
||||
params,
|
||||
).fetchall()
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
messages = [_row_message(r) for r in rows[:limit]]
|
||||
return messages, len(rows) > limit
|
||||
|
||||
def message_exists(self, chat_id: str, created_at: datetime, item_id: uuid.UUID) -> bool:
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
row = conn.execute(
|
||||
"""
|
||||
SELECT 1 AS ok FROM messages
|
||||
WHERE chat_id = %s AND created_at = %s AND id = %s
|
||||
""",
|
||||
(chat_id, created_at, item_id),
|
||||
).fetchone()
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
return bool(row)
|
||||
|
||||
def chat_exists_for_user(
|
||||
self, user_id: str, created_at: datetime, item_id: uuid.UUID
|
||||
) -> bool:
|
||||
try:
|
||||
with get_pool().connection() as conn:
|
||||
row = conn.execute(
|
||||
"""
|
||||
SELECT 1 AS ok
|
||||
FROM chats c
|
||||
JOIN chat_members cm ON cm.chat_id = c.id
|
||||
WHERE cm.user_id = %s AND c.created_at = %s AND c.id = %s
|
||||
""",
|
||||
(user_id, created_at, item_id),
|
||||
).fetchone()
|
||||
except PsycopgError as exc:
|
||||
raise StoreUnavailable("database unavailable") from exc
|
||||
return bool(row)
|
||||
@@ -0,0 +1,12 @@
|
||||
# Собственные проверки (lab3)
|
||||
|
||||
Требуется уже запущенный стенд (`scripts/up.cmd` / `scripts/up.sh`, ≥2 API за LB).
|
||||
|
||||
```sh
|
||||
# из корня репозитория:
|
||||
cd student-tests
|
||||
BASE_URL=http://127.0.0.1:8080 python run_all.py
|
||||
```
|
||||
|
||||
`test_idempotency` проверяет общий Idempotency-Key и наблюдение ≥2 `X-Instance-Id`.
|
||||
Для expiry поднимите API с `SESSION_TTL_SECONDS=2` и повторите `test_sessions`.
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Minimal HTTP helpers for student tests (stdlib only)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
|
||||
BASE_URL = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
|
||||
|
||||
|
||||
def call(method, path, expected=None, body=None, token=None, headers=None, raw=None):
|
||||
h = {"Accept": "application/json"}
|
||||
if token:
|
||||
h["Authorization"] = "Bearer " + token
|
||||
if body is not None:
|
||||
raw = json.dumps(body, ensure_ascii=False).encode("utf-8")
|
||||
h["Content-Type"] = "application/json"
|
||||
# Lab3+: POST messages require Idempotency-Key unless caller supplies headers.
|
||||
if (
|
||||
method == "POST"
|
||||
and path.rstrip("/").endswith("/messages")
|
||||
and (headers is None or "Idempotency-Key" not in headers)
|
||||
):
|
||||
h["Idempotency-Key"] = uuid.uuid4().hex
|
||||
h.update(headers or {})
|
||||
req = urllib.request.Request(BASE_URL + path, data=raw, headers=h, method=method)
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=30)
|
||||
except urllib.error.HTTPError as err:
|
||||
resp = err
|
||||
data = resp.read()
|
||||
status = resp.code
|
||||
if expected is not None:
|
||||
assert status == expected, (
|
||||
f"{method} {path}: expected {expected}, got {status}: {data[:200]!r}"
|
||||
)
|
||||
payload = None
|
||||
ctype = resp.headers.get_content_type()
|
||||
if ctype == "application/json" and data:
|
||||
payload = json.loads(data.decode("utf-8"))
|
||||
assert resp.headers.get("X-Request-Id"), "missing X-Request-Id"
|
||||
assert resp.headers.get("X-Instance-Id"), "missing X-Instance-Id"
|
||||
return {"status": status, "headers": resp.headers, "json": payload, "bytes": data}
|
||||
|
||||
|
||||
def register(label: str):
|
||||
username = f"st_{label}_{uuid.uuid4().hex[:10]}"
|
||||
password = "T3st-" + uuid.uuid4().hex
|
||||
body = {
|
||||
"username": username,
|
||||
"display_name": f"Student {label}",
|
||||
"password": password,
|
||||
}
|
||||
user = call("POST", "/api/v1/users", 201, body=body)["json"]
|
||||
credential = base64.b64encode(f"{username}:{password}".encode()).decode()
|
||||
session = call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
201,
|
||||
headers={"Authorization": "Basic " + credential},
|
||||
)["json"]
|
||||
return {
|
||||
"id": user["id"],
|
||||
"user": user,
|
||||
"token": session["token"],
|
||||
"registration": body,
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run all student tests against a live API (BASE_URL)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import traceback
|
||||
|
||||
|
||||
MODULES = [
|
||||
"test_api",
|
||||
"test_concurrency",
|
||||
"test_sessions",
|
||||
"test_idempotency",
|
||||
"test_deps",
|
||||
]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
failed = 0
|
||||
passed = 0
|
||||
for name in MODULES:
|
||||
mod = importlib.import_module(name)
|
||||
tests = [getattr(mod, n) for n in dir(mod) if n.startswith("test_")]
|
||||
for fn in tests:
|
||||
label = f"{name}.{fn.__name__}"
|
||||
try:
|
||||
fn()
|
||||
print("OK", label)
|
||||
passed += 1
|
||||
except Exception as exc:
|
||||
failed += 1
|
||||
print("FAIL", label, ":", exc)
|
||||
traceback.print_exc()
|
||||
print(f"Summary: {passed} passed, {failed} failed")
|
||||
return 1 if failed else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.path.insert(0, ".")
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,159 @@
|
||||
"""Grade 3/4 API checks: auth, conflict, X-User-Id ignored, logout."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import time
|
||||
import uuid
|
||||
|
||||
from helpers import call, register
|
||||
|
||||
|
||||
def test_username_conflict():
|
||||
a = register("conflict")
|
||||
body = dict(a["registration"])
|
||||
call("POST", "/api/v1/users", 409, body=body)
|
||||
|
||||
|
||||
def test_wrong_password():
|
||||
a = register("wrongpw")
|
||||
bad = base64.b64encode(
|
||||
(a["registration"]["username"] + ":wrong-password-xx").encode()
|
||||
).decode()
|
||||
resp = call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
401,
|
||||
headers={"Authorization": "Basic " + bad},
|
||||
)
|
||||
assert resp["headers"].get("WWW-Authenticate", "").lower().startswith("basic")
|
||||
|
||||
|
||||
def test_x_user_id_ignored():
|
||||
a = register("xu")
|
||||
b = register("xv")
|
||||
me = call(
|
||||
"GET",
|
||||
"/api/v1/users/me",
|
||||
200,
|
||||
token=a["token"],
|
||||
headers={"X-User-Id": b["id"]},
|
||||
)["json"]
|
||||
assert me["id"] == a["id"]
|
||||
|
||||
|
||||
def test_basic_on_business_rejected():
|
||||
a = register("basicbiz")
|
||||
cred = base64.b64encode(
|
||||
(a["registration"]["username"] + ":" + a["registration"]["password"]).encode()
|
||||
).decode()
|
||||
call(
|
||||
"GET",
|
||||
"/api/v1/users/me",
|
||||
401,
|
||||
headers={"Authorization": "Basic " + cred},
|
||||
)
|
||||
|
||||
|
||||
def test_logout_revokes():
|
||||
a = register("logout")
|
||||
call("DELETE", "/api/v1/auth/sessions/current", 204, token=a["token"])
|
||||
call("GET", "/api/v1/users/me", 401, token=a["token"])
|
||||
|
||||
|
||||
def test_chat_requires_caller_member():
|
||||
a = register("owner")
|
||||
b = register("peer")
|
||||
call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
400,
|
||||
token=a["token"],
|
||||
body={"title": "no me", "member_ids": [b["id"]]},
|
||||
)
|
||||
|
||||
|
||||
def test_unknown_member_rolls_back():
|
||||
a = register("atom")
|
||||
fake = str(uuid.uuid4())
|
||||
call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
400,
|
||||
token=a["token"],
|
||||
body={"title": "bad", "member_ids": [a["id"], fake]},
|
||||
)
|
||||
|
||||
|
||||
def test_whitespace_only_password_rejected():
|
||||
# Maintainer remark: required strings must not be whitespace-only.
|
||||
body = {
|
||||
"username": "ws_" + uuid.uuid4().hex[:10],
|
||||
"display_name": "Spaces",
|
||||
"password": " " * 12,
|
||||
}
|
||||
call("POST", "/api/v1/users", 400, body=body)
|
||||
|
||||
|
||||
def test_nul_in_text_rejected():
|
||||
a = register("nul")
|
||||
peer = register("nulb")
|
||||
chat = call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"title": "nul-chat", "member_ids": [a["id"], peer["id"]]},
|
||||
)["json"]
|
||||
call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
400,
|
||||
token=a["token"],
|
||||
body={"text": "hello\u0000world"},
|
||||
)
|
||||
|
||||
|
||||
def test_login_rejects_json_body():
|
||||
a = register("bodylogin")
|
||||
cred = base64.b64encode(
|
||||
(a["registration"]["username"] + ":" + a["registration"]["password"]).encode()
|
||||
).decode()
|
||||
call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
400,
|
||||
body={"unexpected": True},
|
||||
headers={"Authorization": "Basic " + cred},
|
||||
)
|
||||
|
||||
|
||||
def test_invalid_basic_encoding_is_401():
|
||||
# Maintainer remark: malformed Basic must be 401, not 500.
|
||||
call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
401,
|
||||
headers={"Authorization": "Basic " + base64.b64encode("é".encode()).decode()},
|
||||
)
|
||||
|
||||
|
||||
def test_unsupported_media_type():
|
||||
a = register("ctype")
|
||||
peer = register("ctypeb")
|
||||
chat = call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"title": "ctype", "member_ids": [a["id"], peer["id"]]},
|
||||
)["json"]
|
||||
raw = b'{"text":"x"}'
|
||||
call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
415,
|
||||
token=a["token"],
|
||||
raw=raw,
|
||||
headers={"Content-Type": "text/plain"},
|
||||
)
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Concurrent registration race and parallel messages."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import concurrent.futures
|
||||
import uuid
|
||||
|
||||
from helpers import BASE_URL, call, register
|
||||
|
||||
|
||||
def test_parallel_messages_unique():
|
||||
a = register("par")
|
||||
b = register("parb")
|
||||
chat = call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"title": "parallel", "member_ids": [a["id"], b["id"]]},
|
||||
)["json"]
|
||||
|
||||
def send(i: int):
|
||||
return call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"text": f"msg-{i}"},
|
||||
)["json"]["id"]
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as pool:
|
||||
ids = list(pool.map(send, range(100)))
|
||||
assert len(ids) == len(set(ids)) == 100
|
||||
|
||||
|
||||
def test_username_race_one_winner():
|
||||
username = "race_" + uuid.uuid4().hex[:12]
|
||||
password = "T3st-" + uuid.uuid4().hex
|
||||
body = {
|
||||
"username": username,
|
||||
"display_name": "Race",
|
||||
"password": password,
|
||||
}
|
||||
|
||||
def attempt(_):
|
||||
import json
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
raw = json.dumps(body).encode("utf-8")
|
||||
req = urllib.request.Request(
|
||||
BASE_URL + "/api/v1/users",
|
||||
data=raw,
|
||||
headers={"Content-Type": "application/json", "Accept": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=30)
|
||||
except urllib.error.HTTPError as err:
|
||||
resp = err
|
||||
return resp.code
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
|
||||
codes = list(pool.map(attempt, range(8)))
|
||||
assert codes.count(201) == 1, codes
|
||||
assert codes.count(409) == 7, codes
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Dependency failure checks (Redis/Postgres). Optional: RUN_DEPS_TEST=1."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
from helpers import BASE_URL, call, register
|
||||
|
||||
|
||||
def _compose(*args: str) -> None:
|
||||
cmd = ["docker", "compose", "--env-file", ".env", *args]
|
||||
subprocess.check_call(cmd, cwd=os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
|
||||
def _wait_ready(timeout_s: int = 60) -> None:
|
||||
deadline = time.time() + timeout_s
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
if call("GET", "/health/ready")["status"] == 200:
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
time.sleep(1)
|
||||
raise AssertionError("stack did not become ready in time")
|
||||
|
||||
|
||||
def test_redis_stop_returns_503_or_unready():
|
||||
if os.environ.get("RUN_DEPS_TEST") != "1":
|
||||
print("SKIP test_redis_stop_returns_503_or_unready (set RUN_DEPS_TEST=1)")
|
||||
return
|
||||
actor = register("deps")
|
||||
_compose("stop", "redis")
|
||||
time.sleep(2)
|
||||
try:
|
||||
ready = call("GET", "/health/ready")
|
||||
assert ready["status"] == 503, ready["status"]
|
||||
me = call("GET", "/api/v1/users/me", token=actor["token"])
|
||||
assert me["status"] in (401, 503)
|
||||
finally:
|
||||
_compose("start", "redis")
|
||||
_wait_ready()
|
||||
|
||||
|
||||
def test_postgres_stop_returns_503():
|
||||
# Maintainer remark (level 4): cover PostgreSQL unavailability like Redis.
|
||||
if os.environ.get("RUN_DEPS_TEST") != "1":
|
||||
print("SKIP test_postgres_stop_returns_503 (set RUN_DEPS_TEST=1)")
|
||||
return
|
||||
actor = register("deps_pg")
|
||||
_compose("stop", "postgres")
|
||||
time.sleep(2)
|
||||
try:
|
||||
ready = call("GET", "/health/ready")
|
||||
assert ready["status"] == 503, ready["status"]
|
||||
# Authenticated business path must fail fast as 503 (not hang / not 500).
|
||||
me = call("GET", "/api/v1/users/me", token=actor["token"])
|
||||
assert me["status"] == 503, me["status"]
|
||||
hist = call(
|
||||
"GET",
|
||||
f"/api/v1/chats/{actor['id']}/messages",
|
||||
token=actor["token"],
|
||||
)
|
||||
assert hist["status"] in (400, 404, 503), hist["status"]
|
||||
finally:
|
||||
_compose("start", "postgres")
|
||||
_wait_ready(90)
|
||||
@@ -0,0 +1,139 @@
|
||||
"""Lab3: shared Idempotency-Key and multi-replica visibility."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import concurrent.futures
|
||||
import uuid
|
||||
|
||||
from helpers import call, register
|
||||
|
||||
|
||||
def _chat_pair():
|
||||
a = register("idem_a")
|
||||
b = register("idem_b")
|
||||
chat = call(
|
||||
"POST",
|
||||
"/api/v1/chats",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"title": "idem", "member_ids": [a["id"], b["id"]]},
|
||||
)["json"]
|
||||
return a, b, chat
|
||||
|
||||
|
||||
def test_missing_idempotency_key_400():
|
||||
a, _b, chat = _chat_pair()
|
||||
call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
400,
|
||||
token=a["token"],
|
||||
body={"text": "no-key"},
|
||||
headers={"Idempotency-Key": ""},
|
||||
)
|
||||
|
||||
|
||||
def test_replay_same_key_same_body():
|
||||
a, _b, chat = _chat_pair()
|
||||
key = uuid.uuid4().hex
|
||||
first = call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"text": "same"},
|
||||
headers={"Idempotency-Key": key},
|
||||
)["json"]
|
||||
second = call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"text": "same"},
|
||||
headers={"Idempotency-Key": key},
|
||||
)["json"]
|
||||
assert first == second
|
||||
|
||||
|
||||
def test_empty_attachment_ids_equals_absent():
|
||||
# Peer remark: attachment_ids: [] ≡ absent field; must not 400; same idempotency hash.
|
||||
a, _b, chat = _chat_pair()
|
||||
key = uuid.uuid4().hex
|
||||
first = call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"text": "with-empty-attachments", "attachment_ids": []},
|
||||
headers={"Idempotency-Key": key},
|
||||
)["json"]
|
||||
second = call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"text": "with-empty-attachments"},
|
||||
headers={"Idempotency-Key": key},
|
||||
)["json"]
|
||||
assert first == second
|
||||
|
||||
|
||||
def test_same_key_different_body_409():
|
||||
a, _b, chat = _chat_pair()
|
||||
key = uuid.uuid4().hex
|
||||
call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"text": "one"},
|
||||
headers={"Idempotency-Key": key},
|
||||
)
|
||||
call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
409,
|
||||
token=a["token"],
|
||||
body={"text": "two"},
|
||||
headers={"Idempotency-Key": key},
|
||||
)
|
||||
|
||||
|
||||
def test_parallel_same_key_one_message():
|
||||
a, b, chat = _chat_pair()
|
||||
key = uuid.uuid4().hex
|
||||
text = "parallel-idem-" + key
|
||||
|
||||
def send(_):
|
||||
return call(
|
||||
"POST",
|
||||
f"/api/v1/chats/{chat['id']}/messages",
|
||||
201,
|
||||
token=a["token"],
|
||||
body={"text": text},
|
||||
headers={"Idempotency-Key": key},
|
||||
)["json"]
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
|
||||
results = list(pool.map(send, range(8)))
|
||||
ids = {m["id"] for m in results}
|
||||
assert len(ids) == 1, ids
|
||||
page = call(
|
||||
"GET",
|
||||
f"/api/v1/chats/{chat['id']}/messages?limit=100",
|
||||
200,
|
||||
token=b["token"],
|
||||
)["json"]
|
||||
assert sum(1 for m in page["items"] if m["text"] == text) == 1
|
||||
|
||||
|
||||
def test_multiple_instance_ids_observed():
|
||||
a = register("inst")
|
||||
seen = set()
|
||||
for _ in range(40):
|
||||
seen.add(
|
||||
call("GET", "/api/v1/users/me", 200, token=a["token"])["headers"].get(
|
||||
"X-Instance-Id"
|
||||
)
|
||||
)
|
||||
assert len(seen) >= 2, f"expected ≥2 replicas via LB, got {seen}"
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Session expiry (short TTL) — run against stack with SESSION_TTL_SECONDS=2."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import os
|
||||
import time
|
||||
|
||||
from helpers import call, register
|
||||
|
||||
|
||||
def test_session_expiry_if_short_ttl():
|
||||
ttl = int(os.environ.get("SESSION_TTL_SECONDS", "3600"))
|
||||
if ttl > 5:
|
||||
print("SKIP expiry: SESSION_TTL_SECONDS=%s (need <=5 for this test)" % ttl)
|
||||
return
|
||||
a = register("expire")
|
||||
time.sleep(ttl + 1.5)
|
||||
call("GET", "/api/v1/users/me", 401, token=a["token"])
|
||||
|
||||
|
||||
def test_login_after_register():
|
||||
a = register("relogin")
|
||||
cred = base64.b64encode(
|
||||
(a["registration"]["username"] + ":" + a["registration"]["password"]).encode()
|
||||
).decode()
|
||||
session = call(
|
||||
"POST",
|
||||
"/api/v1/auth/sessions",
|
||||
201,
|
||||
headers={"Authorization": "Basic " + cred},
|
||||
)["json"]
|
||||
assert session["token_type"] == "Bearer"
|
||||
assert session["user"]["id"] == a["id"]
|
||||
Reference in New Issue
Block a user