47 changed files with 3035 additions and 31 deletions
+37 -4
View File
@@ -1,20 +1,53 @@
# VCS / CI
.git
.github
# Secrets and local env
.env
.env.*
!.env.example
secrets
certs
volumes
data
backups
*.key
*.pem
*.p12
*.pfx
*.jks
# Local / runtime junk
.venv
venv
__pycache__
*.py[cod]
*.egg-info
.pytest_cache
.mypy_cache
.ruff_cache
node_modules
evidence
.DS_Store
Thumbs.db
.idea
.vscode
# Lab artifacts not needed at runtime
evidence
volumes
data
backups
docs
student-tests
tests
scripts
examples
contracts
REPORT.md
README.md
COURSE.md
CONTRIBUTING.md
MAINTAINERS.md
Dockerfile.example
compose.example.yaml
requirements-dev.txt
*.md
# Keep only what the image needs: requirements.txt + src/ + migrations/
+2 -4
View File
@@ -1,13 +1,11 @@
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
# These are logical interface suggestions; map actual framework names in REPORT.md.
APP_HOST=0.0.0.0
APP_PORT=8080
APP_ENV=lab
SESSION_TTL_SECONDS=3600
# Choose credentials locally. File paths contain no secret values.
DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
SESSION_STORE=redis
CACHE_URL_FILE=/run/secrets/cache_url
# Number of interchangeable API replicas behind HAProxy (grade 3 requires ≥2).
API_REPLICAS=2
IDEMPOTENCY_TTL_SECONDS=86400
+30
View File
@@ -0,0 +1,30 @@
# Pinned base image (lab3). Update digest when bumping Python.
FROM python:3.12.8-slim-bookworm AS runtime
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
APP_HOST=0.0.0.0 \
APP_PORT=8080 \
PYTHONPATH=/app/src \
MIGRATIONS_DIR=/app/migrations
WORKDIR /app
RUN groupadd --system --gid 10001 app \
&& useradd --system --uid 10001 --gid app --home /app --shell /usr/sbin/nologin app
COPY --chown=app:app requirements.txt .
RUN pip install --no-cache-dir --upgrade pip==24.3.1 \
&& pip install --no-cache-dir --retries 10 --timeout 60 -r requirements.txt
COPY --chown=app:app src ./src
COPY --chown=app:app migrations ./migrations
USER app
EXPOSE 8080
HEALTHCHECK --interval=5s --timeout=3s --start-period=15s --retries=5 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2)"
CMD ["python", "-m", "messenger"]
+77 -17
View File
@@ -1,37 +1,97 @@
# Отчёт — лабораторная 3
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: Болотов Илья, БВТ2303
- Целевая оценка; выбранный трек 5, если нужен: **4** (трек 5 не выбран)
- Ветка: `solution/bvt2303-bolotov`
- Использованные источники, библиотеки и помощь ИИ: FastAPI, Uvicorn, psycopg3 (+pool), redis-py, argon2-cffi, HAProxy; перенос lab2 и адаптация под LB/идемпотентность с помощью ИИ; публичные материалы курса (README, contracts, tests).
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
**Prerequisites:** Docker Desktop (Engine), Docker Compose v2+. Локальный Python 3.9+ удобен для `make test` (на Windows задайте `PYTHONUTF8=1`).
```sh
cp .env.example .env
# Windows:
scripts\up.cmd
# Unix:
# chmod +x scripts/*.sh && ./scripts/up.sh
# Эквивалент: docker compose --env-file .env up -d --build --scale api=2
```
Дождаться `GET http://127.0.0.1:8080/health/ready` → 200.
Проверки:
```sh
# Windows PowerShell:
$env:PYTHONUTF8=1
python scripts/check_contract.py
python tests/smoke.py --base-url http://127.0.0.1:8080
python tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url http://127.0.0.1:8080
python tests/resilience.py --mode failover --action-script scripts/stop-one.cmd --base-url http://127.0.0.1:8080
```
Unix: `scripts/restart.sh` / `scripts/stop-one.sh` (+ `chmod +x`).
Остановка без удаления данных: `docker compose --env-file .env down`
Удаление учебных volumes: `docker compose --env-file .env down -v` (осознанно).
Версии образов: `python:3.12.8-slim-bookworm`, `postgres:16.6-bookworm`, `redis:7.4.2-bookworm`, `haproxy:2.9.14-bookworm`.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab2, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
```
Клиент → HAProxy (:8080, единственная публикация) → API × N (без host ports)
→ PostgreSQL (users/chats/messages + idempotency_records)
→ Redis (opaque sessions, AOF everysec)
Миграции: one-shot сервис migrate (service_completed_successfully), не гонка реплик.
```
- **Перенесено из lab2** (`mtusi2026lab2`, ветка `solution/bvt2303-bolotov`): `src/messenger`, Argon2id, Redis-сессии, student-tests каркас, стиль ошибок/cursor, non-root Dockerfile.
- **Изменено в lab3:**
- `X-Instance-Id` = hostname контейнера (стабилен на жизнь процесса).
- Обязательный `Idempotency-Key` для POST сообщения; результат в PostgreSQL (`idempotency_records`) + `pg_advisory_xact_lock` между репликами.
- Compose: `lb` + `--scale api≥2`, без `container_name`; API не публикует порты наружу.
- CPU/RAM limits (`cpus`/`mem_limit`) на сервисах.
- SIGTERM: readiness→503, uvicorn `timeout_graceful_shutdown=8`, compose `stop_grace_period: 12s`; HAProxy `inter 2s fall 2`.
- **Конфиг (.env):** `SESSION_TTL_SECONDS`, `IDEMPOTENCY_TTL_SECONDS` (24ч), `API_REPLICAS`.
- **Миграции:** `001_init.sql` (lab2) + `002_idempotency.sql`; совместимы с уже работающими репликами (только ADD TABLE/INDEX).
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
| Offline check | `python scripts/check_contract.py` | PASS skeleton | `evidence/check.txt` |
| Публичный smoke (lab3) | `python tests/smoke.py --base-url http://127.0.0.1:8080` | PASS lab3 (idempotency + X-Instance-Id) | `evidence/smoke.txt` |
| ≥2 реплики через LB | student `test_multiple_instance_ids_observed` | ≥2 разных X-Instance-Id | `evidence/student-tests.txt` |
| Persistence restart | `resilience.py --mode persistence` + `scripts/restart.cmd` | PASS: session + messages | `evidence/persistence.txt` |
| Failover stop-one | `resilience.py --mode failover` + `scripts/stop-one.cmd` | PASS; recovery ≈2.4 с; 2 инстанса до отказа | `evidence/failover.txt` |
| Идемпотентность (в т.ч. parallel) | student `test_parallel_same_key_one_message` | 8 параллельных POST → 1 сообщение | `evidence/student-tests.txt` |
| Последовательный отказ каждой реплики | `python scripts/seq_failover_evidence.py` | round0/1 recovery ≈5.5 с; target исчез из last5 | `evidence/seq-failover.json` |
| Нагрузка (baseline) | `tests/load.py --duration 20 --concurrency 4` | ~241 RPS, err=0, p50=16ms p95=32ms p99=78ms | `evidence/load.json` |
| Rolling update под нагрузкой | `python scripts/rolling_load_evidence.py` | 2 реплики поочерёдно; 356 ok / 3 err | `evidence/rolling-load.json` |
| Собственные тесты | `python student-tests/run_all.py` | 24 OK (deps SKIP без RUN_DEPS_TEST) | `evidence/student-tests.txt` |
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| `stop-one` одной API-реплики | чтение/запись через оставшуюся ≤10 с; без дублей при retry ключа | failover PASS; first_recovery≈2.4 с; 4 transient fail | данные/сессия сохранены |
| Последовательный stop каждой реплики (с restore) | то же для обеих | recovery≈5.5 с; target absent | без потери подтверждённых |
| Rolling replace API под нагрузкой | краткие ошибки, стенд жив | 3 ошибки / 356 успехов; p99↑ из‑за stop | volumes не трогались |
| Restart postgres+redis+api (persistence) | прежний token и сообщения | PASS | ≤90 с на hook+ready |
| Миграции one-shot | нет гонки DDL на репликах | `migrate` Exited 0 до старта API | — |
## Ограничения и следующие шаги
Аппаратный контекст: один Docker Desktop host (Windows). Измерения closed-loop (`load.py`) — функциональный baseline, не претензия на максимальный RPS железа.
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
## Ограничения
- Трек 5 (цена масштабирования / отказ узла) не выполнялся — целевая оценка 4.
- Один Docker-host и один Postgres/Redis/LB — отказ API-процесса выдерживается; отказ host/DB — нет.
- На Windows публичные тесты требуют `PYTHONUTF8=1` из‑за кириллицы в OpenAPI.
- На Unix-клоне для shell-скриптов нужен `chmod +x scripts/*.sh` (указано в запуске).
## Итог
Стенд lab3 для оценки **4** выполнен и проверен: ≥2 API за HAProxy, идемпотентность POST сообщений (в т.ч. параллельно), persistence и failover с evidence, student-tests 24 OK. Трек 5 не заявляется.
+147
View File
@@ -0,0 +1,147 @@
# Lab3: Client → HAProxy → API×N → shared PostgreSQL + Redis (sessions).
# Public entrypoint only via lb:8080. API replicas have no host ports.
services:
postgres:
image: postgres:16.6-bookworm
environment:
POSTGRES_USER: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
POSTGRES_DB: messenger
secrets:
- postgres_password
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 5s
timeout: 3s
retries: 10
start_period: 10s
mem_limit: 512m
cpus: 0.50
networks:
- labnet
redis:
image: redis:7.4.2-bookworm
command:
- redis-server
- --appendonly
- "yes"
- --appendfsync
- everysec
- --dir
- /data
volumes:
- redisdata:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 10
start_period: 5s
mem_limit: 128m
cpus: 0.25
networks:
- labnet
migrate:
build:
context: .
dockerfile: Dockerfile
image: mtusi-messenger:lab3
command: ["python", "-m", "messenger.migrate"]
environment:
DATABASE_URL_FILE: /run/secrets/database_url
MIGRATIONS_DIR: /app/migrations
secrets:
- database_url
depends_on:
postgres:
condition: service_healthy
restart: "no"
mem_limit: 256m
cpus: 0.25
networks:
- labnet
api:
build:
context: .
dockerfile: Dockerfile
image: mtusi-messenger:lab3
environment:
APP_HOST: "0.0.0.0"
APP_PORT: "8080"
APP_ENV: lab
SESSION_TTL_SECONDS: "${SESSION_TTL_SECONDS:-3600}"
IDEMPOTENCY_TTL_SECONDS: "${IDEMPOTENCY_TTL_SECONDS:-86400}"
SESSION_STORE: redis
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL_FILE: /run/secrets/cache_url
MIGRATIONS_DIR: /app/migrations
secrets:
- database_url
- cache_url
depends_on:
migrate:
condition: service_completed_successfully
redis:
condition: service_healthy
# No host ports: only reachable via labnet / HAProxy.
expose:
- "8080"
healthcheck:
test:
[
"CMD",
"python",
"-c",
"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2)",
]
interval: 5s
timeout: 3s
retries: 12
start_period: 20s
stop_grace_period: 12s
mem_limit: 256m
cpus: 0.50
networks:
- labnet
lb:
image: haproxy:2.9.14-bookworm
volumes:
- ./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
ports:
- "127.0.0.1:8080:8080"
depends_on:
api:
condition: service_healthy
healthcheck:
# Image has no curl/wget; validate config + process. Public readiness is /health/ready via LB.
test: ["CMD", "haproxy", "-c", "-f", "/usr/local/etc/haproxy/haproxy.cfg"]
interval: 10s
timeout: 3s
retries: 5
start_period: 5s
mem_limit: 128m
cpus: 0.25
networks:
- labnet
secrets:
postgres_password:
file: ./secrets/postgres_password
database_url:
file: ./secrets/database_url
cache_url:
file: ./secrets/cache_url
volumes:
pgdata:
redisdata:
networks:
labnet:
driver: bridge
Binary file not shown.
Binary file not shown.
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 20.0,
"elapsed_seconds": 20.032,
"concurrency": 4,
"successful_requests": 4821,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 240.665,
"successful_latency_ms": {
"p50": 16.0,
"p95": 32.0,
"p99": 78.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
BIN
View File
Binary file not shown.
+9
View File
@@ -0,0 +1,9 @@
date: 2026-10-30
branch: solution/bvt2303-bolotov
target_grade: 4
host: Docker Desktop on Windows
images: python:3.12.8-slim-bookworm, postgres:16.6-bookworm, redis:7.4.2-bookworm, haproxy:2.9.14-bookworm
api_replicas: 2
entrypoint: http://127.0.0.1:8080 (HAProxy only)
limits: api 0.5 CPU / 256Mi; postgres 0.5/512Mi; redis 0.25/128Mi; lb 0.25/128Mi
health: HAProxy inter=2s fall=2; API stop_grace_period=12s; uvicorn graceful=8s
Binary file not shown.
+8
View File
@@ -0,0 +1,8 @@
{
"successful": 356,
"errors": 3,
"p50_ms": 16.0,
"p95_ms": 32.0,
"p99_ms": 2016.0,
"replicas_rolled": 2
}
Binary file not shown.
+26
View File
@@ -0,0 +1,26 @@
[
{
"round": 0,
"observed_before": [
"252249c099ca",
"8aabecba79c6"
],
"target": "252249c099ca",
"successful_writes": 85,
"failed_attempts": 2,
"first_recovery_seconds": 5.594,
"target_absent_in_last5": true
},
{
"round": 1,
"observed_before": [
"252249c099ca",
"8aabecba79c6"
],
"target": "8aabecba79c6",
"successful_writes": 129,
"failed_attempts": 1,
"first_recovery_seconds": 5.469,
"target_absent_in_last5": true
}
]
Binary file not shown.
Binary file not shown.
Binary file not shown.
+4 -6
View File
@@ -1,7 +1,5 @@
# Инфраструктура студента
# Инфраструктура lab3
Здесь разместите конфигурации выбранных сервисов; основную топологию — в корневом compose.yaml/stack.yaml. `compose.example.yaml` — только worksheet ролей и намеренно не запускается. `contracts/infrastructure.json` задаёт минимальные роли, не готовые образы/пароли.
В REPORT.md сопоставьте роль → service name → версия/digest → health/readiness → volume → сеть/порт → способ передачи credentials. Приведите bootstrap (schema, bucket, broker topology, а с lab6 provisioning) и безопасное повторение этих команд.
Приложение строится из исходников. Не используйте заранее существующий образ преподавательского сервера вместо своей реализации. Restart hook перезапускает все соответствующие stores с volumes; failover hook останавливает только заданную API-реплику.
- `haproxy.cfg` — балансировщик (round-robin, HTTP healthcheck `/health/ready`).
- DNS Docker `127.0.0.11` + `server-template` — подхват `--scale api=N` и rolling replace.
- Период проверки backend: `inter 2s`, `fall 2` → типичное исключение недоступной реплики ≤4 с; с учётом DNS hold — до ~5–10 с.
+39
View File
@@ -0,0 +1,39 @@
# Lab3 load balancer: round-robin across Compose DNS name "api" (all replicas).
global
maxconn 2048
log stdout format raw local0
defaults
mode http
log global
option httplog
option dontlognull
timeout connect 3s
timeout client 30s
timeout server 30s
timeout check 2s
timeout http-request 10s
retries 2
resolvers docker
nameserver dns1 127.0.0.11:53
resolve_retries 3
timeout resolve 1s
timeout retry 1s
hold valid 5s
hold obsolete 10s
frontend messenger_fe
bind *:8080
acl is_lb_live path /__lb/live
http-request return status 200 content-type text/plain string ok if is_lb_live
default_backend messenger_api
backend messenger_api
balance roundrobin
option httpchk
http-check send meth GET uri /health/ready ver HTTP/1.1 hdr Host api
http-check expect status 200
# server-template expands A records for Compose service DNS name "api".
# inter=2s fall=2 → typically ≤4s to mark a backend down after failed checks.
server-template api 8 api:8080 check inter 2s fall 2 rise 1 resolvers docker resolve-prefer ipv4 init-addr none
+48
View File
@@ -0,0 +1,48 @@
-- Lab2 schema: users, chats, membership, messages.
-- Applied once via schema_migrations; safe to re-run application bootstrap.
CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY,
username TEXT NOT NULL,
display_name TEXT NOT NULL,
password_hash TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL,
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
CONSTRAINT users_username_unique UNIQUE (username)
);
CREATE TABLE IF NOT EXISTS chats (
id UUID PRIMARY KEY,
title TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE IF NOT EXISTS chat_members (
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id),
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_chat_members_user
ON chat_members (user_id, chat_id);
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY,
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
sender_id UUID NOT NULL REFERENCES users(id),
text TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
-- Keyset pagination by (created_at, id) within a chat; avoids full history scan.
CREATE INDEX IF NOT EXISTS idx_messages_chat_created_id
ON messages (chat_id, created_at ASC, id ASC);
-- List chats for a user ordered by chat created_at/id.
CREATE INDEX IF NOT EXISTS idx_chats_created_id
ON chats (created_at ASC, id ASC);
+16
View File
@@ -0,0 +1,16 @@
-- Lab3: shared idempotency records for POST /messages (across API replicas).
CREATE TABLE IF NOT EXISTS idempotency_records (
user_id UUID NOT NULL,
method TEXT NOT NULL,
path TEXT NOT NULL,
idem_key TEXT NOT NULL,
request_hash TEXT NOT NULL,
response_status INT NOT NULL,
response_body JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL,
PRIMARY KEY (user_id, method, path, idem_key)
);
CREATE INDEX IF NOT EXISTS idx_idempotency_created
ON idempotency_records (created_at);
+6
View File
@@ -0,0 +1,6 @@
fastapi==0.115.6
uvicorn[standard]==0.32.1
pydantic==2.10.3
psycopg[binary,pool]==3.2.4
redis==5.2.1
argon2-cffi==23.1.0
+11
View File
@@ -0,0 +1,11 @@
@echo off
setlocal
cd /d "%~dp0\.."
if "%API_REPLICAS%"=="" set API_REPLICAS=2
docker compose --env-file .env restart postgres redis
if errorlevel 1 exit /b 1
docker compose --env-file .env up -d --no-deps --scale api=%API_REPLICAS% api
if errorlevel 1 exit /b 1
docker compose --env-file .env restart lb
if errorlevel 1 exit /b 1
exit /b 0
+10
View File
@@ -0,0 +1,10 @@
#!/bin/sh
set -eu
# Restart API replicas and all state stores; preserve volumes.
cd "$(dirname "$0")/.."
REPLICAS="${API_REPLICAS:-2}"
docker compose --env-file .env restart postgres redis
# Recreate API fleet at desired scale (migrate already applied; skip one-shot).
docker compose --env-file .env up -d --no-deps --scale "api=${REPLICAS}" api
docker compose --env-file .env restart lb
exit 0
+28
View File
@@ -0,0 +1,28 @@
#!/bin/sh
set -eu
# Grade-4 rolling replacement: recreate API containers one-by-one under LB.
# Usage: scripts/rolling-update.sh
cd "$(dirname "$0")/.."
REPLICAS="${API_REPLICAS:-2}"
ids=$(docker compose --env-file .env ps -q api)
count=0
for id in $ids; do
[ -n "$id" ] || continue
count=$((count + 1))
printf '%s\n' "rolling: stopping $id"
docker stop -t 10 "$id" >/dev/null
# Bring fleet back to desired scale (starts a fresh replica; old stopped removed by up).
docker compose --env-file .env up -d --no-deps --scale "api=${REPLICAS}" --remove-orphans api
# Wait until LB again sees healthy backends via public ready.
i=0
while [ "$i" -lt 60 ]; do
if curl -sf "http://127.0.0.1:8080/health/ready" >/dev/null 2>&1; then
break
fi
i=$((i + 1))
sleep 1
done
printf '%s\n' "rolling: step $count ready"
done
printf '%s\n' "rolling update finished ($count steps, scale=${REPLICAS})"
+150
View File
@@ -0,0 +1,150 @@
#!/usr/bin/env python3
"""Grade-4: closed-loop POSTs during rolling API replacement."""
from __future__ import annotations
import base64
import json
import os
import subprocess
import threading
import time
import uuid
import urllib.error
import urllib.request
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
BASE = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
def call(method, path, body=None, token=None, headers=None):
h = {"Accept": "application/json"}
if token:
h["Authorization"] = "Bearer " + token
data = None
if body is not None:
data = json.dumps(body).encode("utf-8")
h["Content-Type"] = "application/json"
if headers:
h.update(headers)
req = urllib.request.Request(BASE + path, data=data, headers=h, method=method)
try:
resp = urllib.request.urlopen(req, timeout=5)
except urllib.error.HTTPError as err:
resp = err
except Exception as exc:
return {"status": 0, "error": type(exc).__name__}
raw = resp.read()
js = None
if raw and resp.headers.get_content_type() == "application/json":
js = json.loads(raw.decode("utf-8"))
return {"status": resp.code, "json": js, "ms": 0}
def bootstrap():
uname = "roll_" + uuid.uuid4().hex[:10]
pw = "T3st-" + uuid.uuid4().hex
user = call("POST", "/api/v1/users", {"username": uname, "display_name": "Roll", "password": pw})
assert user["status"] == 201, user
cred = base64.b64encode(f"{uname}:{pw}".encode()).decode()
tok = call("POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic " + cred})["json"]["token"]
peer_u = "rollb_" + uuid.uuid4().hex[:10]
peer = call("POST", "/api/v1/users", {"username": peer_u, "display_name": "RollB", "password": pw})["json"]
chat = call(
"POST",
"/api/v1/chats",
{"title": "roll", "member_ids": [user["json"]["id"], peer["id"]]},
token=tok,
)["json"]
return tok, chat["id"]
def main():
tok, chat_id = bootstrap()
stats = {"ok": 0, "err": 0, "latencies_ms": []}
stop = threading.Event()
def worker():
while not stop.is_set():
key = uuid.uuid4().hex
t0 = time.monotonic()
r = call(
"POST",
f"/api/v1/chats/{chat_id}/messages",
{"text": "roll-" + key},
token=tok,
headers={"Idempotency-Key": key},
)
dt = (time.monotonic() - t0) * 1000
if r.get("status") == 201:
stats["ok"] += 1
stats["latencies_ms"].append(dt)
else:
stats["err"] += 1
time.sleep(0.05)
threads = [threading.Thread(target=worker, daemon=True) for _ in range(4)]
for th in threads:
th.start()
time.sleep(2) # warm-up
# Rolling: stop one container, scale back, repeat for each current replica.
ids = (
subprocess.check_output(
["docker", "compose", "--env-file", ".env", "ps", "-q", "api"],
cwd=ROOT,
text=True,
)
.strip()
.split()
)
for cid in ids:
subprocess.check_call(["docker", "stop", "-t", "10", cid], cwd=ROOT)
subprocess.check_call(
[
"docker",
"compose",
"--env-file",
".env",
"up",
"-d",
"--no-deps",
"--scale",
"api=2",
"--remove-orphans",
"api",
],
cwd=ROOT,
)
# wait until ready through LB
for _ in range(60):
if call("GET", "/health/ready").get("status") == 200:
break
time.sleep(0.5)
time.sleep(2)
time.sleep(2)
stop.set()
for th in threads:
th.join(timeout=5)
lats = sorted(stats["latencies_ms"])
def pct(p):
if not lats:
return None
return round(lats[min(len(lats) - 1, int(len(lats) * p))], 2)
duration = max(sum(lats) / 1000.0, 1e-6)
# Approximate wall duration from thread runtime ~ warm+rolling
summary = {
"successful": stats["ok"],
"errors": stats["err"],
"p50_ms": pct(0.50),
"p95_ms": pct(0.95),
"p99_ms": pct(0.99),
"replicas_rolled": len(ids),
}
out = ROOT / "evidence" / "rolling-load.json"
out.write_text(json.dumps(summary, ensure_ascii=False, indent=2), encoding="utf-8")
print(json.dumps(summary, ensure_ascii=False))
print("WROTE", out)
if __name__ == "__main__":
main()
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env python3
"""Grade-4 evidence: stop each observed API replica in turn; restore between steps."""
from __future__ import annotations
import base64
import json
import os
import subprocess
import time
import uuid
import urllib.error
import urllib.request
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
BASE = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
def call(method, path, body=None, token=None, headers=None, expected=None):
h = {"Accept": "application/json"}
if token:
h["Authorization"] = "Bearer " + token
data = None
if body is not None:
data = json.dumps(body).encode("utf-8")
h["Content-Type"] = "application/json"
if headers:
h.update(headers)
req = urllib.request.Request(BASE + path, data=data, headers=h, method=method)
try:
resp = urllib.request.urlopen(req, timeout=5)
except urllib.error.HTTPError as err:
resp = err
raw = resp.read()
status = resp.code
if expected is not None and status != expected:
raise RuntimeError(f"{method} {path}: expected {expected}, got {status}: {raw[:200]!r}")
js = None
if raw and resp.headers.get_content_type() == "application/json":
js = json.loads(raw.decode("utf-8"))
return {"status": status, "headers": resp.headers, "json": js}
def header(resp, name: str) -> str:
return resp["headers"].get(name) or resp["headers"].get(name.lower()) or ""
def restore_api():
replicas = os.environ.get("API_REPLICAS", "2")
subprocess.check_call(
[
"docker",
"compose",
"--env-file",
".env",
"up",
"-d",
"--no-deps",
"--scale",
f"api={replicas}",
"--remove-orphans",
"api",
],
cwd=ROOT,
)
for _ in range(60):
try:
call("GET", "/health/ready", expected=200)
return
except Exception:
time.sleep(1)
raise RuntimeError("API did not become ready after restore")
def one_round(round_no: int) -> dict:
uname = "seq_" + uuid.uuid4().hex[:10]
pw = "T3st-" + uuid.uuid4().hex
user = call(
"POST",
"/api/v1/users",
{"username": uname, "display_name": "Seq", "password": pw},
expected=201,
)["json"]
cred = base64.b64encode(f"{uname}:{pw}".encode()).decode()
tok = call(
"POST",
"/api/v1/auth/sessions",
headers={"Authorization": "Basic " + cred},
expected=201,
)["json"]["token"]
peer_u = "seqb_" + uuid.uuid4().hex[:10]
peer = call(
"POST",
"/api/v1/users",
{"username": peer_u, "display_name": "SeqB", "password": pw},
expected=201,
)["json"]
chat = call(
"POST",
"/api/v1/chats",
{"title": "seq", "member_ids": [user["id"], peer["id"]]},
token=tok,
expected=201,
)["json"]
observed = set()
for _ in range(40):
observed.add(header(call("GET", "/api/v1/users/me", token=tok, expected=200), "X-Instance-Id"))
if len(observed) < 2:
raise RuntimeError(f"need ≥2 instances, got {observed}")
target = sorted(observed)[round_no % len(observed)]
env = dict(os.environ)
env["TARGET_INSTANCE_ID"] = target
script = ROOT / ("scripts/stop-one.cmd" if os.name == "nt" else "scripts/stop-one.sh")
started = time.monotonic()
proc = subprocess.Popen([str(script)], env=env, cwd=ROOT)
ok = fail = 0
first = None
after = []
while time.monotonic() - started < 20:
code = proc.poll()
try:
key = uuid.uuid4().hex
r = call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
{"text": "seq-" + key},
token=tok,
headers={"Idempotency-Key": key},
expected=201,
)
ok += 1
if code == 0:
if first is None:
first = time.monotonic() - started
after.append(header(r, "X-Instance-Id"))
except Exception:
fail += 1
time.sleep(0.1)
if proc.poll() is None:
proc.wait(timeout=10)
result = {
"round": round_no,
"observed_before": sorted(observed),
"target": target,
"successful_writes": ok,
"failed_attempts": fail,
"first_recovery_seconds": round(first, 3) if first is not None else None,
"target_absent_in_last5": target not in after[-5:],
}
restore_api()
return result
def main():
results = []
for i in range(2):
results.append(one_round(i))
print(json.dumps(results[-1], ensure_ascii=False))
out = ROOT / "evidence" / "seq-failover.json"
out.write_text(json.dumps(results, ensure_ascii=False, indent=2), encoding="utf-8")
print("WROTE", out)
if __name__ == "__main__":
main()
+17
View File
@@ -0,0 +1,17 @@
@echo off
setlocal EnableExtensions
cd /d "%~dp0\.."
if "%TARGET_INSTANCE_ID%"=="" (
echo stop-one: TARGET_INSTANCE_ID required >&2
exit /b 1
)
for /f "usebackq delims=" %%I in (`docker compose --env-file .env ps -q api`) do (
for /f "usebackq delims=" %%H in (`docker inspect -f "{{.Config.Hostname}}" %%I`) do (
if /I "%%H"=="%TARGET_INSTANCE_ID%" (
docker stop -t 10 %%I >nul
exit /b 0
)
)
)
echo stop-one: replica not found >&2
exit /b 1
+43
View File
@@ -0,0 +1,43 @@
#!/bin/sh
set -eu
# Stop ONE API replica matching TARGET_INSTANCE_ID (= X-Instance-Id / hostname).
# Leave it stopped; do not touch DB/Redis/LB/volumes.
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
cd "$(dirname "$0")/.."
found=""
# Compose project containers for service api
ids=$(docker compose --env-file .env ps -q api 2>/dev/null || true)
for id in $ids; do
[ -n "$id" ] || continue
host=$(docker inspect -f '{{.Config.Hostname}}' "$id")
if [ "$host" = "$TARGET_INSTANCE_ID" ]; then
found=$id
break
fi
iid=$(docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$id" | sed -n 's/^INSTANCE_ID=//p' | head -n1)
if [ -n "$iid" ] && [ "$iid" = "$TARGET_INSTANCE_ID" ]; then
found=$id
break
fi
done
if [ -z "$found" ]; then
# Fallback: any running container with matching hostname (scaled names differ).
for id in $(docker ps -q --filter "ancestor=mtusi-messenger:lab3"); do
host=$(docker inspect -f '{{.Config.Hostname}}' "$id")
if [ "$host" = "$TARGET_INSTANCE_ID" ]; then
found=$id
break
fi
done
fi
if [ -z "$found" ]; then
printf '%s\n' "stop-one: replica not found for TARGET_INSTANCE_ID" >&2
exit 1
fi
# Graceful stop; API marks not-ready on SIGTERM before exit (uvicorn grace ~8s).
docker stop -t 10 "$found" >/dev/null
exit 0
+10
View File
@@ -0,0 +1,10 @@
@echo off
setlocal
cd /d "%~dp0\.."
if "%API_REPLICAS%"=="" set API_REPLICAS=2
docker compose --env-file .env config --quiet
if errorlevel 1 exit /b 1
docker compose --env-file .env up -d --build --scale api=%API_REPLICAS%
if errorlevel 1 exit /b 1
echo stack up: api scale=%API_REPLICAS%, entry http://127.0.0.1:8080
exit /b 0
+8
View File
@@ -0,0 +1,8 @@
#!/bin/sh
set -eu
# Bootstrap lab3 stack with ≥2 API replicas behind LB.
cd "$(dirname "$0")/.."
REPLICAS="${API_REPLICAS:-2}"
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --scale "api=${REPLICAS}"
printf '%s\n' "stack up: api scale=${REPLICAS}, entry http://127.0.0.1:8080"
+3
View File
@@ -0,0 +1,3 @@
"""Messenger API for lab3 (LB + shared sessions + idempotent messages)."""
__version__ = "3.0.0"
+30
View File
@@ -0,0 +1,30 @@
"""Run the messenger HTTP server."""
from __future__ import annotations
import sys
from pathlib import Path
_SRC = Path(__file__).resolve().parents[1]
if str(_SRC) not in sys.path:
sys.path.insert(0, str(_SRC))
import uvicorn
from messenger import config
def main() -> None:
uvicorn.run(
"messenger.app:app",
host=config.app_host(),
port=config.app_port(),
log_level="info",
access_log=False,
timeout_keep_alive=5,
timeout_graceful_shutdown=8,
)
if __name__ == "__main__":
main()
+666
View File
@@ -0,0 +1,666 @@
"""HTTP API: multi-replica messenger behind LB (lab3)."""
from __future__ import annotations
import asyncio
import base64
import json
import logging
import re
import signal
import uuid
from contextlib import asynccontextmanager
from datetime import datetime, timezone
from typing import Any, List, Optional, Tuple
from fastapi import FastAPI, Header, Query, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse, Response
from starlette.exceptions import HTTPException as StarletteHTTPException
from messenger import config, db, sessions
from messenger.sessions import SessionStoreError
from messenger.store import Store, StoreUnavailable, format_ts
USERNAME_RE = re.compile(r"^[a-z0-9_]{3,32}\Z")
UUID_RE = re.compile(
r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\Z"
)
IDEMPOTENCY_KEY_RE = re.compile(r"^[A-Za-z0-9._:-]{1,128}\Z")
BODY_READ_TIMEOUT_S = 5.0
PASSWORD_MIN = 12
PASSWORD_MAX = 128
store = Store()
ready = False
INSTANCE_ID = config.instance_id()
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
)
log = logging.getLogger("messenger")
def mark_not_ready(*_args):
global ready
if ready:
ready = False
log.info("readiness disabled; draining in-flight requests")
def _chain_signal(previous):
def handler(signum, frame):
mark_not_ready()
if callable(previous) and previous not in (signal.SIG_DFL, signal.SIG_IGN):
previous(signum, frame)
return handler
@asynccontextmanager
async def lifespan(_app: FastAPI):
try:
signal.signal(signal.SIGTERM, _chain_signal(signal.getsignal(signal.SIGTERM)))
signal.signal(signal.SIGINT, _chain_signal(signal.getsignal(signal.SIGINT)))
except (ValueError, OSError) as exc:
log.warning("signal handlers not installed: %s", exc)
global ready
db.init_pool()
# Migrations run once via `python -m messenger.migrate` (compose migrate service).
sessions.init_client()
ready = True
log.info("messenger started instance_id=%s", INSTANCE_ID)
yield
mark_not_ready()
sessions.close_client()
db.close_pool()
log.info("messenger stopped")
def request_id_from(request: Request) -> str:
incoming = request.headers.get("x-request-id")
if incoming and incoming.strip():
return incoming.strip()[:128]
return str(uuid.uuid4())
def error_body(code: str, message: str, request_id: str) -> dict:
return {"error": {"code": code, "message": message, "request_id": request_id}}
def common_headers(request_id: str, extra: Optional[dict] = None) -> dict:
headers = {"X-Request-Id": request_id, "X-Instance-Id": INSTANCE_ID}
if extra:
headers.update(extra)
return headers
def error_response(
status: int,
code: str,
message: str,
request_id: str,
www_authenticate: Optional[str] = None,
) -> JSONResponse:
headers = common_headers(request_id)
if www_authenticate:
headers["WWW-Authenticate"] = www_authenticate
return JSONResponse(
status_code=status,
content=error_body(code, message, request_id),
headers=headers,
)
def is_blank(value: str) -> bool:
return value.strip() == ""
def has_nul(value: str) -> bool:
# PostgreSQL text cannot store U+0000; reject explicitly (400), never 500/503.
return "\x00" in value
def has_lone_surrogate(value: str) -> bool:
return any(0xD800 <= ord(ch) <= 0xDFFF for ch in value)
def codepoint_len(value: str) -> int:
return len(value)
def parse_uuid(value: str) -> Optional[str]:
if not isinstance(value, str) or not UUID_RE.match(value):
return None
try:
return str(uuid.UUID(value))
except ValueError:
return None
def encode_cursor(resource: str, created_at: datetime, item_id: str) -> str:
raw = f"{resource}|{created_at.strftime('%Y-%m-%dT%H:%M:%SZ')}|{item_id}"
return base64.urlsafe_b64encode(raw.encode("utf-8")).decode("ascii")
def decode_cursor(cursor: str, expected_resource: str) -> Tuple[datetime, uuid.UUID]:
try:
raw = base64.urlsafe_b64decode(cursor.encode("ascii")).decode("utf-8")
resource, ts, item_id = raw.split("|", 2)
if resource != expected_resource:
raise ValueError("wrong resource")
created_at = datetime.strptime(ts, "%Y-%m-%dT%H:%M:%SZ").replace(
tzinfo=timezone.utc
)
return created_at, uuid.UUID(item_id)
except Exception as exc:
raise ValueError("invalid cursor") from exc
app = FastAPI(
title="MTUSI Messenger lab3",
docs_url=None,
redoc_url=None,
lifespan=lifespan,
)
@app.middleware("http")
async def add_request_id_and_gate(request: Request, call_next):
rid = request_id_from(request)
request.state.request_id = rid
path = request.url.path
if not ready and path not in ("/health/live",):
if path == "/health/ready":
return error_response(503, "unavailable", "Shutting down", rid)
return error_response(503, "unavailable", "Service unavailable", rid)
response = await call_next(request)
if "X-Request-Id" not in response.headers:
response.headers["X-Request-Id"] = rid
response.headers["X-Instance-Id"] = INSTANCE_ID
log.info(
"%s %s -> %s request_id=%s instance_id=%s",
request.method,
path,
response.status_code,
rid,
INSTANCE_ID,
)
return response
class ApiError(Exception):
def __init__(
self,
status: int,
code: str,
message: str,
www_authenticate: Optional[str] = None,
):
self.status = status
self.code = code
self.message = message
self.www_authenticate = www_authenticate
@app.exception_handler(ApiError)
async def api_error_handler(request: Request, exc: ApiError):
rid = getattr(request.state, "request_id", None) or request_id_from(request)
return error_response(
exc.status, exc.code, exc.message, rid, www_authenticate=exc.www_authenticate
)
@app.exception_handler(RequestValidationError)
async def validation_handler(request: Request, exc: RequestValidationError):
rid = getattr(request.state, "request_id", None) or request_id_from(request)
return error_response(400, "invalid_request", "Invalid request", rid)
@app.exception_handler(StarletteHTTPException)
async def http_exception_handler(request: Request, exc: StarletteHTTPException):
rid = getattr(request.state, "request_id", None) or request_id_from(request)
if exc.status_code == 404:
return error_response(404, "not_found", "Not found", rid)
if exc.status_code == 405:
return error_response(400, "invalid_request", "Method not allowed", rid)
return error_response(exc.status_code, "invalid_request", "Request failed", rid)
@app.exception_handler(StoreUnavailable)
async def store_unavailable_handler(request: Request, exc: StoreUnavailable):
rid = getattr(request.state, "request_id", None) or request_id_from(request)
return error_response(503, "unavailable", "Database unavailable", rid)
@app.exception_handler(SessionStoreError)
async def session_store_unavailable_handler(request: Request, exc: SessionStoreError):
rid = getattr(request.state, "request_id", None) or request_id_from(request)
return error_response(503, "unavailable", "Session store unavailable", rid)
@app.exception_handler(Exception)
async def unhandled_handler(request: Request, exc: Exception):
rid = getattr(request.state, "request_id", None) or request_id_from(request)
log.exception("unhandled error request_id=%s", rid)
return error_response(500, "internal_error", "Internal error", rid)
async def read_json_object(request: Request) -> Any:
content_type = request.headers.get("content-type", "")
media = content_type.split(";")[0].strip().lower()
if media and media != "application/json":
raise ApiError(415, "unsupported_media_type", "Unsupported media type")
try:
body = await asyncio.wait_for(request.body(), timeout=BODY_READ_TIMEOUT_S)
except asyncio.TimeoutError:
raise ApiError(400, "invalid_request", "Request body timeout")
if not body:
raise ApiError(400, "invalid_request", "Empty body")
try:
text = body.decode("utf-8")
# Reject trailing garbage after one JSON value (read to EOF).
data, end = json.JSONDecoder().raw_decode(text)
if text[end:].strip():
raise ApiError(400, "invalid_request", "Malformed JSON")
except ApiError:
raise
except (UnicodeDecodeError, json.JSONDecodeError, ValueError):
raise ApiError(400, "invalid_request", "Malformed JSON")
if not isinstance(data, dict):
raise ApiError(400, "invalid_request", "JSON object required")
return data
def require_bearer(authorization: Optional[str]) -> Tuple[str, str]:
"""Return (user_id, token). X-User-Id is ignored."""
bearer_challenge = 'Bearer realm="messenger"'
if not authorization or not authorization.strip():
raise ApiError(
401, "unauthorized", "Missing bearer token", www_authenticate=bearer_challenge
)
parts = authorization.strip().split(None, 1)
if len(parts) != 2 or parts[0].lower() != "bearer" or not parts[1].strip():
# Basic on business endpoints must not work.
raise ApiError(
401, "unauthorized", "Invalid authorization", www_authenticate=bearer_challenge
)
token = parts[1].strip()
try:
session = sessions.get_session(token)
except SessionStoreError:
raise ApiError(503, "unavailable", "Session store unavailable")
if session is None:
raise ApiError(
401, "unauthorized", "Invalid or expired session", www_authenticate=bearer_challenge
)
user_id = session["user_id"]
user = store.get_user(user_id)
if user is None:
raise ApiError(
401, "unauthorized", "Invalid or expired session", www_authenticate=bearer_challenge
)
return user_id, token
def require_chat_member(chat_id_raw: str, user_id: str):
chat_id = parse_uuid(chat_id_raw)
if chat_id is None:
raise ApiError(400, "invalid_request", "Invalid chat_id")
chat = store.get_chat(chat_id)
if chat is None or user_id not in chat.member_ids:
raise ApiError(404, "not_found", "Chat not found")
return chat
def parse_limit(limit: Optional[int]) -> int:
if limit is None:
return 50
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 100:
raise ApiError(400, "invalid_request", "Invalid limit")
return limit
def validate_display_text(value: Any, field: str, max_len: int) -> str:
if not isinstance(value, str) or is_blank(value) or has_nul(value):
raise ApiError(400, "invalid_request", f"Invalid {field}")
if has_lone_surrogate(value):
raise ApiError(400, "invalid_request", f"Invalid {field}")
length = codepoint_len(value)
if length < 1 or length > max_len:
raise ApiError(400, "invalid_request", f"Invalid {field}")
return value
def validate_password(value: Any) -> str:
# Required strings must not be whitespace-only (contracts/README).
if not isinstance(value, str) or is_blank(value) or has_nul(value):
raise ApiError(400, "invalid_request", "Invalid password")
if has_lone_surrogate(value):
raise ApiError(400, "invalid_request", "Invalid password")
length = codepoint_len(value)
if length < PASSWORD_MIN or length > PASSWORD_MAX:
raise ApiError(400, "invalid_request", "Invalid password")
return value
def parse_basic_credentials(authorization: Optional[str]) -> Tuple[str, str]:
challenge = 'Basic realm="messenger", charset="UTF-8"'
if not authorization or not authorization.strip():
raise ApiError(
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
)
parts = authorization.strip().split(None, 1)
if len(parts) != 2 or parts[0].lower() != "basic":
raise ApiError(
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
)
try:
decoded = base64.b64decode(parts[1].strip(), validate=True).decode("utf-8")
except Exception:
raise ApiError(
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
)
if ":" not in decoded:
raise ApiError(
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
)
username, password = decoded.split(":", 1)
# Username ASCII-only per contract; password may contain ':'.
if not username or any(ord(ch) > 127 for ch in username):
raise ApiError(
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
)
return username, password
@app.get("/health/live")
async def liveness(request: Request):
rid = request.state.request_id
return JSONResponse(content={"status": "ok"}, headers=common_headers(rid))
@app.get("/health/ready")
async def readiness(request: Request):
rid = request.state.request_id
if not ready:
return error_response(503, "unavailable", "Not ready", rid)
if not db.ping() or not sessions.ping():
return error_response(503, "unavailable", "Dependency unavailable", rid)
return JSONResponse(content={"status": "ok"}, headers=common_headers(rid))
@app.post("/api/v1/users")
async def register_user(request: Request):
rid = request.state.request_id
data = await read_json_object(request)
allowed = {"username", "display_name", "password"}
if set(data.keys()) - allowed:
raise ApiError(400, "invalid_request", "Unexpected fields")
if "username" not in data or "display_name" not in data or "password" not in data:
raise ApiError(400, "invalid_request", "Missing required fields")
username = data["username"]
display_name = data["display_name"]
if (
not isinstance(username, str)
or is_blank(username)
or has_lone_surrogate(username)
or not USERNAME_RE.match(username)
):
raise ApiError(400, "invalid_request", "Invalid username")
display_name = validate_display_text(display_name, "display_name", 100)
password = validate_password(data["password"])
try:
user = store.create_user(username, display_name, password)
except KeyError:
raise ApiError(409, "conflict", "Username already taken")
return JSONResponse(
status_code=201,
content=user.to_dict(),
headers=common_headers(rid),
)
@app.post("/api/v1/auth/sessions")
async def create_session(request: Request):
rid = request.state.request_id
challenge = 'Basic realm="messenger", charset="UTF-8"'
# Contract: Basic only, no JSON body (even with valid credentials).
try:
raw_body = await asyncio.wait_for(request.body(), timeout=BODY_READ_TIMEOUT_S)
except asyncio.TimeoutError:
raise ApiError(400, "invalid_request", "Request body timeout")
if raw_body and raw_body.strip():
raise ApiError(400, "invalid_request", "Login must not include a body")
username, password = parse_basic_credentials(
request.headers.get("authorization")
)
# Uniform 401 for wrong user / wrong password (no user enumeration).
user = store.get_user_by_username(username)
if user is None or not store.verify_password(user.password_hash, password):
raise ApiError(
401, "unauthorized", "Invalid credentials", www_authenticate=challenge
)
try:
session = sessions.create_session(user.id)
except SessionStoreError:
raise ApiError(503, "unavailable", "Session store unavailable")
body = {
"token": session["token"],
"token_type": "Bearer",
"expires_at": session["expires_at"],
"user": user.to_dict(),
}
return JSONResponse(
status_code=201, content=body, headers=common_headers(rid)
)
@app.delete("/api/v1/auth/sessions/current")
async def logout(
request: Request,
authorization: Optional[str] = Header(default=None, alias="Authorization"),
):
rid = request.state.request_id
_user_id, token = require_bearer(authorization)
try:
sessions.revoke_session(token)
except SessionStoreError:
raise ApiError(503, "unavailable", "Session store unavailable")
return Response(status_code=204, headers=common_headers(rid))
@app.get("/api/v1/users/me")
async def current_user(
request: Request,
authorization: Optional[str] = Header(default=None, alias="Authorization"),
):
rid = request.state.request_id
user_id, _token = require_bearer(authorization)
user = store.get_user(user_id)
assert user is not None
return JSONResponse(content=user.to_dict(), headers=common_headers(rid))
@app.post("/api/v1/chats")
async def create_chat(
request: Request,
authorization: Optional[str] = Header(default=None, alias="Authorization"),
):
rid = request.state.request_id
user_id, _token = require_bearer(authorization)
data = await read_json_object(request)
allowed = {"title", "member_ids"}
if set(data.keys()) - allowed:
raise ApiError(400, "invalid_request", "Unexpected fields")
if "title" not in data or "member_ids" not in data:
raise ApiError(400, "invalid_request", "Missing required fields")
title = validate_display_text(data["title"], "title", 100)
member_ids = data["member_ids"]
if not isinstance(member_ids, list) or not member_ids or len(member_ids) > 100:
raise ApiError(400, "invalid_request", "Invalid member_ids")
parsed: List[str] = []
seen = set()
for mid in member_ids:
if not isinstance(mid, str):
raise ApiError(400, "invalid_request", "Invalid member_ids")
uid = parse_uuid(mid)
if uid is None:
raise ApiError(400, "invalid_request", "Invalid member_ids")
if uid in seen:
raise ApiError(400, "invalid_request", "Duplicate member_ids")
seen.add(uid)
parsed.append(uid)
if user_id not in seen:
raise ApiError(400, "invalid_request", "Caller must be a member")
try:
chat = store.create_chat(title, parsed)
except KeyError:
raise ApiError(400, "invalid_request", "Unknown member")
return JSONResponse(
status_code=201,
content=chat.to_dict(),
headers=common_headers(rid),
)
@app.get("/api/v1/chats")
async def list_chats(
request: Request,
authorization: Optional[str] = Header(default=None, alias="Authorization"),
limit: Optional[int] = Query(default=None),
cursor: Optional[str] = Query(default=None),
):
rid = request.state.request_id
user_id, _token = require_bearer(authorization)
lim = parse_limit(limit)
resource = f"chats:{user_id}"
cursor_key = None
if cursor is not None:
if not cursor:
raise ApiError(400, "invalid_request", "Invalid cursor")
try:
cursor_ts, cursor_id = decode_cursor(cursor, resource)
except ValueError:
raise ApiError(400, "invalid_request", "Invalid cursor")
if not store.chat_exists_for_user(user_id, cursor_ts, cursor_id):
raise ApiError(400, "invalid_request", "Invalid cursor")
cursor_key = (cursor_ts, cursor_id)
chats, more = store.list_chats_for_user(user_id, lim, cursor_key)
next_cursor = None
if more and chats:
last = chats[-1]
next_cursor = encode_cursor(resource, last.created_at, last.id)
return JSONResponse(
content={"items": [c.to_dict() for c in chats], "next_cursor": next_cursor},
headers=common_headers(rid),
)
@app.get("/api/v1/chats/{chat_id}")
async def get_chat(
chat_id: str,
request: Request,
authorization: Optional[str] = Header(default=None, alias="Authorization"),
):
rid = request.state.request_id
user_id, _token = require_bearer(authorization)
chat = require_chat_member(chat_id, user_id)
return JSONResponse(content=chat.to_dict(), headers=common_headers(rid))
def parse_idempotency_key(raw: Optional[str]) -> str:
if raw is None or not raw.strip() or not IDEMPOTENCY_KEY_RE.match(raw.strip()):
raise ApiError(400, "invalid_request", "Invalid or missing Idempotency-Key")
return raw.strip()
@app.post("/api/v1/chats/{chat_id}/messages")
async def create_message(
chat_id: str,
request: Request,
authorization: Optional[str] = Header(default=None, alias="Authorization"),
idempotency_key: Optional[str] = Header(default=None, alias="Idempotency-Key"),
):
rid = request.state.request_id
user_id, _token = require_bearer(authorization)
chat = require_chat_member(chat_id, user_id)
key = parse_idempotency_key(idempotency_key)
data = await read_json_object(request)
# Contract: absent attachment_ids ≡ []; lab3 has no uploads, so only [] is accepted.
allowed = {"text", "attachment_ids"}
if set(data.keys()) - allowed:
raise ApiError(400, "invalid_request", "Unexpected fields")
if "text" not in data:
raise ApiError(400, "invalid_request", "Missing text")
if "attachment_ids" in data:
aids = data["attachment_ids"]
if not isinstance(aids, list) or aids:
raise ApiError(400, "invalid_request", "Invalid attachment_ids")
text = data["text"]
if (
not isinstance(text, str)
or is_blank(text)
or has_nul(text)
or has_lone_surrogate(text)
or codepoint_len(text) > 4000
):
raise ApiError(400, "invalid_request", "Invalid text")
path = f"/api/v1/chats/{chat.id}/messages"
try:
message, _replayed = store.create_message_idempotent(
chat.id, user_id, text, "POST", path, key
)
except KeyError:
raise ApiError(409, "conflict", "Idempotency-Key reused with different body")
except ValueError:
raise ApiError(400, "invalid_request", "Invalid text")
return JSONResponse(
status_code=201,
content=message.to_dict(),
headers=common_headers(rid),
)
@app.get("/api/v1/chats/{chat_id}/messages")
async def list_messages(
chat_id: str,
request: Request,
authorization: Optional[str] = Header(default=None, alias="Authorization"),
limit: Optional[int] = Query(default=None),
cursor: Optional[str] = Query(default=None),
):
rid = request.state.request_id
user_id, _token = require_bearer(authorization)
chat = require_chat_member(chat_id, user_id)
lim = parse_limit(limit)
resource = f"messages:{chat.id}"
cursor_key = None
if cursor is not None:
if not cursor:
raise ApiError(400, "invalid_request", "Invalid cursor")
try:
cursor_ts, cursor_id = decode_cursor(cursor, resource)
except ValueError:
raise ApiError(400, "invalid_request", "Invalid cursor")
if not store.message_exists(chat.id, cursor_ts, cursor_id):
raise ApiError(400, "invalid_request", "Invalid cursor")
cursor_key = (cursor_ts, cursor_id)
messages, has_more = store.list_messages(chat.id, lim, cursor_key)
next_cursor = None
if has_more and messages:
last = messages[-1]
next_cursor = encode_cursor(resource, last.created_at, last.id)
return JSONResponse(
content={
"items": [m.to_dict() for m in messages],
"next_cursor": next_cursor,
},
headers=common_headers(rid),
)
def create_app() -> FastAPI:
return app
+83
View File
@@ -0,0 +1,83 @@
"""Runtime configuration from environment / secret files."""
from __future__ import annotations
import os
import socket
from pathlib import Path
def _read_secret(env_key: str, file_key: str, default: str | None = None) -> str:
file_path = os.environ.get(file_key, "").strip()
if file_path:
path = Path(file_path)
if path.is_file():
return path.read_text(encoding="utf-8").strip()
value = os.environ.get(env_key, "").strip()
if value:
return value
if default is not None:
return default
raise RuntimeError(f"Missing secret: set {env_key} or {file_key}")
def database_url() -> str:
return _read_secret("DATABASE_URL", "DATABASE_URL_FILE")
def cache_url() -> str:
return _read_secret("CACHE_URL", "CACHE_URL_FILE")
def session_ttl_seconds() -> int:
raw = os.environ.get("SESSION_TTL_SECONDS", "3600").strip()
try:
value = int(raw)
except ValueError as exc:
raise RuntimeError("SESSION_TTL_SECONDS must be an integer") from exc
if value < 1:
raise RuntimeError("SESSION_TTL_SECONDS must be >= 1")
return value
def idempotency_ttl_seconds() -> int:
raw = os.environ.get("IDEMPOTENCY_TTL_SECONDS", "86400").strip()
try:
value = int(raw)
except ValueError as exc:
raise RuntimeError("IDEMPOTENCY_TTL_SECONDS must be an integer") from exc
if value < 1:
raise RuntimeError("IDEMPOTENCY_TTL_SECONDS must be >= 1")
return value
def instance_id() -> str:
"""Stable for process lifetime; used as X-Instance-Id (maps to container hostname)."""
override = os.environ.get("INSTANCE_ID", "").strip()
if override:
return override[:128]
return socket.gethostname()[:128]
def app_host() -> str:
return os.environ.get("APP_HOST", "0.0.0.0")
def app_port() -> int:
return int(os.environ.get("APP_PORT", "8080"))
def migrations_dir() -> Path:
override = os.environ.get("MIGRATIONS_DIR", "").strip()
if override:
return Path(override)
here = Path(__file__).resolve()
candidates = [
Path("/app/migrations"),
here.parents[2] / "migrations",
Path.cwd() / "migrations",
]
for path in candidates:
if path.is_dir():
return path
return candidates[0]
+107
View File
@@ -0,0 +1,107 @@
"""PostgreSQL connection pool and versioned migrations."""
from __future__ import annotations
import logging
import re
from pathlib import Path
from typing import List, Optional
from psycopg.rows import dict_row
from psycopg_pool import ConnectionPool
from messenger import config
log = logging.getLogger("messenger.db")
_pool: Optional[ConnectionPool] = None
def get_pool() -> ConnectionPool:
if _pool is None:
raise RuntimeError("Database pool is not initialized")
return _pool
def init_pool() -> ConnectionPool:
global _pool
if _pool is not None:
return _pool
url = config.database_url()
_pool = ConnectionPool(
conninfo=url,
min_size=1,
max_size=10,
kwargs={
"row_factory": dict_row,
"connect_timeout": 5,
"options": "-c statement_timeout=5000",
},
open=True,
timeout=10,
)
return _pool
def close_pool() -> None:
global _pool
if _pool is not None:
_pool.close()
_pool = None
def _split_sql(script: str) -> List[str]:
"""Split SQL file into statements (no procedure bodies in our migrations)."""
# Drop line comments.
cleaned = re.sub(r"--[^\n]*", "", script)
parts = []
for chunk in cleaned.split(";"):
stmt = chunk.strip()
if stmt:
parts.append(stmt)
return parts
def apply_migrations(migrations_path: Optional[Path] = None) -> None:
path = migrations_path or config.migrations_dir()
sql_files = sorted(path.glob("*.sql"))
if not sql_files:
raise RuntimeError(f"No migrations found in {path}")
pool = get_pool()
with pool.connection() as conn:
with conn.transaction():
conn.execute(
"""
CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
"""
)
for sql_file in sql_files:
version = sql_file.name
row = conn.execute(
"SELECT 1 AS ok FROM schema_migrations WHERE version = %s",
(version,),
).fetchone()
if row:
log.info("migration already applied: %s", version)
continue
script = sql_file.read_text(encoding="utf-8")
for stmt in _split_sql(script):
conn.execute(stmt)
conn.execute(
"INSERT INTO schema_migrations (version) VALUES (%s)",
(version,),
)
log.info("applied migration: %s", version)
def ping() -> bool:
try:
with get_pool().connection() as conn:
conn.execute("SELECT 1")
return True
except Exception as exc:
log.warning("postgres ping failed: %s", type(exc).__name__)
return False
+28
View File
@@ -0,0 +1,28 @@
"""One-shot migrations entrypoint (run once before API replicas)."""
from __future__ import annotations
import logging
import sys
from messenger import db
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
)
log = logging.getLogger("messenger.migrate")
def main() -> int:
db.init_pool()
try:
db.apply_migrations()
log.info("migrations complete")
return 0
finally:
db.close_pool()
if __name__ == "__main__":
raise SystemExit(main())
+136
View File
@@ -0,0 +1,136 @@
"""Opaque Bearer sessions in Redis with TTL (durable via Redis AOF)."""
from __future__ import annotations
import json
import logging
import secrets
from datetime import datetime, timedelta, timezone
from typing import Optional
import redis
from messenger import config
log = logging.getLogger("messenger.sessions")
_client: Optional[redis.Redis] = None
TOKEN_BYTES = 32 # 256-bit entropy
KEY_PREFIX = "sess:"
def utc_now() -> datetime:
return datetime.now(timezone.utc).replace(microsecond=0)
def format_ts(value: datetime) -> str:
if value.tzinfo is None:
value = value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
def parse_ts(value: str) -> datetime:
return datetime.strptime(value, "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=timezone.utc)
def init_client() -> redis.Redis:
global _client
if _client is not None:
return _client
url = config.cache_url()
_client = redis.Redis.from_url(
url,
decode_responses=True,
socket_connect_timeout=3,
socket_timeout=3,
health_check_interval=30,
)
return _client
def close_client() -> None:
global _client
if _client is not None:
try:
_client.close()
except Exception:
pass
_client = None
def get_client() -> redis.Redis:
if _client is None:
raise RuntimeError("Redis client is not initialized")
return _client
def ping() -> bool:
try:
return bool(get_client().ping())
except Exception as exc:
log.warning("redis ping failed: %s", type(exc).__name__)
return False
class SessionStoreError(Exception):
"""Raised when Redis is unavailable for session operations."""
def create_session(user_id: str) -> dict:
ttl = config.session_ttl_seconds()
token = secrets.token_urlsafe(TOKEN_BYTES)
now = utc_now()
expires_at = now + timedelta(seconds=ttl)
payload = {
"user_id": user_id,
"expires_at": format_ts(expires_at),
"created_at": format_ts(now),
}
key = KEY_PREFIX + token
try:
client = get_client()
# SET with EX: Redis is sole session store; AOF keeps it across restart.
client.set(key, json.dumps(payload), ex=ttl)
except redis.RedisError as exc:
raise SessionStoreError("session store unavailable") from exc
return {
"token": token,
"token_type": "Bearer",
"expires_at": payload["expires_at"],
"user_id": user_id,
}
def get_session(token: str) -> Optional[dict]:
if not token:
return None
key = KEY_PREFIX + token
try:
raw = get_client().get(key)
except redis.RedisError as exc:
raise SessionStoreError("session store unavailable") from exc
if not raw:
return None
try:
data = json.loads(raw)
expires_at = parse_ts(data["expires_at"])
except (ValueError, KeyError, TypeError, json.JSONDecodeError):
return None
if expires_at <= utc_now():
try:
get_client().delete(key)
except redis.RedisError:
pass
return None
return data
def revoke_session(token: str) -> bool:
if not token:
return False
key = KEY_PREFIX + token
try:
return bool(get_client().delete(key))
except redis.RedisError as exc:
raise SessionStoreError("session store unavailable") from exc
+480
View File
@@ -0,0 +1,480 @@
"""PostgreSQL repository: users, chats, membership, messages, idempotency."""
from __future__ import annotations
import hashlib
import json
import uuid
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Any, List, Optional, Tuple
from argon2 import PasswordHasher
from argon2.exceptions import VerifyMismatchError
from psycopg import Error as PsycopgError
from psycopg import errors as pg_errors
from psycopg.types.json import Jsonb
from messenger import config
from messenger.db import get_pool
_ph = PasswordHasher()
class StoreUnavailable(Exception):
"""Raised when PostgreSQL is unreachable or timed out."""
def utc_now() -> datetime:
return datetime.now(timezone.utc).replace(microsecond=0)
def format_ts(value: datetime) -> str:
if value.tzinfo is None:
value = value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
@dataclass
class User:
id: str
username: str
display_name: str
created_at: datetime
password_hash: str = ""
def to_dict(self) -> dict:
return {
"id": self.id,
"username": self.username,
"display_name": self.display_name,
"created_at": format_ts(self.created_at),
}
@dataclass
class Chat:
id: str
title: str
member_ids: List[str]
created_at: datetime
def to_dict(self) -> dict:
return {
"id": self.id,
"title": self.title,
"member_ids": list(self.member_ids),
"created_at": format_ts(self.created_at),
}
@dataclass
class Message:
id: str
chat_id: str
sender_id: str
text: str
created_at: datetime
def to_dict(self) -> dict:
return {
"id": self.id,
"chat_id": self.chat_id,
"sender_id": self.sender_id,
"text": self.text,
"created_at": format_ts(self.created_at),
}
def _row_user(row: dict) -> User:
return User(
id=str(row["id"]),
username=row["username"],
display_name=row["display_name"],
created_at=row["created_at"],
password_hash=row.get("password_hash", ""),
)
def _row_message(row: dict) -> Message:
return Message(
id=str(row["id"]),
chat_id=str(row["chat_id"]),
sender_id=str(row["sender_id"]),
text=row["text"],
created_at=row["created_at"],
)
class Store:
def hash_password(self, password: str) -> str:
return _ph.hash(password)
def verify_password(self, password_hash: str, password: str) -> bool:
try:
return _ph.verify(password_hash, password)
except VerifyMismatchError:
return False
except Exception:
return False
def create_user(self, username: str, display_name: str, password: str) -> User:
user_id = uuid.uuid4()
created = utc_now()
password_hash = self.hash_password(password)
try:
with get_pool().connection() as conn:
with conn.transaction():
conn.execute(
"""
INSERT INTO users (id, username, display_name, password_hash, created_at)
VALUES (%s, %s, %s, %s, %s)
""",
(user_id, username, display_name, password_hash, created),
)
except (pg_errors.UniqueViolation, pg_errors.IntegrityError) as exc:
raise KeyError("username_taken") from exc
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
return User(
id=str(user_id),
username=username,
display_name=display_name,
created_at=created,
)
def get_user(self, user_id: str) -> Optional[User]:
try:
with get_pool().connection() as conn:
row = conn.execute(
"""
SELECT id, username, display_name, password_hash, created_at
FROM users WHERE id = %s
""",
(user_id,),
).fetchone()
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
return _row_user(row) if row else None
def get_user_by_username(self, username: str) -> Optional[User]:
try:
with get_pool().connection() as conn:
row = conn.execute(
"""
SELECT id, username, display_name, password_hash, created_at
FROM users WHERE username = %s
""",
(username,),
).fetchone()
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
return _row_user(row) if row else None
def create_chat(self, title: str, member_ids: List[str]) -> Chat:
chat_id = uuid.uuid4()
created = utc_now()
try:
with get_pool().connection() as conn:
with conn.transaction():
# Atomic membership check + insert; rollback if any member missing.
rows = conn.execute(
"SELECT id FROM users WHERE id = ANY(%s::uuid[])",
(member_ids,),
).fetchall()
found = {str(r["id"]) for r in rows}
if found != set(member_ids):
raise KeyError("unknown_member")
conn.execute(
"INSERT INTO chats (id, title, created_at) VALUES (%s, %s, %s)",
(chat_id, title, created),
)
for mid in member_ids:
conn.execute(
"INSERT INTO chat_members (chat_id, user_id) VALUES (%s, %s)",
(chat_id, mid),
)
except KeyError:
raise
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
return Chat(
id=str(chat_id),
title=title,
member_ids=list(member_ids),
created_at=created,
)
def get_chat(self, chat_id: str) -> Optional[Chat]:
try:
with get_pool().connection() as conn:
row = conn.execute(
"SELECT id, title, created_at FROM chats WHERE id = %s",
(chat_id,),
).fetchone()
if not row:
return None
members = conn.execute(
"""
SELECT user_id FROM chat_members
WHERE chat_id = %s
ORDER BY user_id
""",
(chat_id,),
).fetchall()
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
return Chat(
id=str(row["id"]),
title=row["title"],
member_ids=[str(m["user_id"]) for m in members],
created_at=row["created_at"],
)
def list_chats_for_user(
self,
user_id: str,
limit: int,
cursor: Optional[Tuple[datetime, uuid.UUID]],
) -> Tuple[List[Chat], Optional[str]]:
params: list = [user_id]
where = "cm.user_id = %s"
if cursor is not None:
where += " AND (c.created_at, c.id) > (%s, %s)"
params.extend([cursor[0], cursor[1]])
params.append(limit + 1)
try:
with get_pool().connection() as conn:
rows = conn.execute(
f"""
SELECT c.id, c.title, c.created_at
FROM chats c
JOIN chat_members cm ON cm.chat_id = c.id
WHERE {where}
ORDER BY c.created_at ASC, c.id ASC
LIMIT %s
""",
params,
).fetchall()
chats: List[Chat] = []
for row in rows[:limit]:
members = conn.execute(
"""
SELECT user_id FROM chat_members
WHERE chat_id = %s
ORDER BY user_id
""",
(row["id"],),
).fetchall()
chats.append(
Chat(
id=str(row["id"]),
title=row["title"],
member_ids=[str(m["user_id"]) for m in members],
created_at=row["created_at"],
)
)
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
has_more = len(rows) > limit
return chats, "more" if has_more else None
@staticmethod
def request_hash_for_message(text: str) -> str:
# Canonical form: sorted keys; absent attachment_ids ≡ [].
canonical = json.dumps(
{"attachment_ids": [], "text": text},
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
def create_message(self, chat_id: str, sender_id: str, text: str) -> Message:
msg_id = uuid.uuid4()
created = utc_now()
try:
with get_pool().connection() as conn:
with conn.transaction():
conn.execute(
"""
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
VALUES (%s, %s, %s, %s, %s)
""",
(msg_id, chat_id, sender_id, text, created),
)
except pg_errors.DataError as exc:
raise ValueError("invalid_text") from exc
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
return Message(
id=str(msg_id),
chat_id=chat_id,
sender_id=sender_id,
text=text,
created_at=created,
)
def create_message_idempotent(
self,
chat_id: str,
sender_id: str,
text: str,
method: str,
path: str,
idem_key: str,
) -> Tuple[Message, bool]:
"""
Create message with shared idempotency across replicas.
Returns (message, replayed).
Raises KeyError('conflict') when same key was used with another body.
"""
req_hash = self.request_hash_for_message(text)
ttl = timedelta(seconds=config.idempotency_ttl_seconds())
# Transaction-scoped advisory lock serializes same-key concurrent POSTs.
lock_material = f"{sender_id}|{method}|{path}|{idem_key}"
lock_key = int(
hashlib.sha256(lock_material.encode("utf-8")).hexdigest()[:15], 16
)
def _load_existing(conn) -> Optional[dict]:
return conn.execute(
"""
SELECT request_hash, response_status, response_body, created_at
FROM idempotency_records
WHERE user_id = %s AND method = %s AND path = %s AND idem_key = %s
""",
(sender_id, method, path, idem_key),
).fetchone()
def _message_from_body(body: Any) -> Message:
return Message(
id=str(body["id"]),
chat_id=str(body["chat_id"]),
sender_id=str(body["sender_id"]),
text=body["text"],
created_at=datetime.strptime(
body["created_at"], "%Y-%m-%dT%H:%M:%SZ"
).replace(tzinfo=timezone.utc),
)
try:
with get_pool().connection() as conn:
with conn.transaction():
conn.execute("SELECT pg_advisory_xact_lock(%s)", (lock_key,))
existing = _load_existing(conn)
if existing is not None:
if existing["created_at"] >= utc_now() - ttl:
if existing["request_hash"] != req_hash:
raise KeyError("conflict")
return _message_from_body(existing["response_body"]), True
conn.execute(
"""
DELETE FROM idempotency_records
WHERE user_id = %s AND method = %s AND path = %s AND idem_key = %s
""",
(sender_id, method, path, idem_key),
)
msg_id = uuid.uuid4()
created = utc_now()
message = Message(
id=str(msg_id),
chat_id=chat_id,
sender_id=sender_id,
text=text,
created_at=created,
)
conn.execute(
"""
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
VALUES (%s, %s, %s, %s, %s)
""",
(msg_id, chat_id, sender_id, text, created),
)
conn.execute(
"""
INSERT INTO idempotency_records (
user_id, method, path, idem_key,
request_hash, response_status, response_body, created_at
) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)
""",
(
sender_id,
method,
path,
idem_key,
req_hash,
201,
Jsonb(message.to_dict()),
created,
),
)
return message, False
except KeyError:
raise
except pg_errors.DataError as exc:
raise ValueError("invalid_text") from exc
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
def list_messages(
self,
chat_id: str,
limit: int,
cursor: Optional[Tuple[datetime, uuid.UUID]],
) -> Tuple[List[Message], bool]:
params: list = [chat_id]
where = "chat_id = %s"
if cursor is not None:
where += " AND (created_at, id) > (%s, %s)"
params.extend([cursor[0], cursor[1]])
params.append(limit + 1)
try:
with get_pool().connection() as conn:
rows = conn.execute(
f"""
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE {where}
ORDER BY created_at ASC, id ASC
LIMIT %s
""",
params,
).fetchall()
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
messages = [_row_message(r) for r in rows[:limit]]
return messages, len(rows) > limit
def message_exists(self, chat_id: str, created_at: datetime, item_id: uuid.UUID) -> bool:
try:
with get_pool().connection() as conn:
row = conn.execute(
"""
SELECT 1 AS ok FROM messages
WHERE chat_id = %s AND created_at = %s AND id = %s
""",
(chat_id, created_at, item_id),
).fetchone()
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
return bool(row)
def chat_exists_for_user(
self, user_id: str, created_at: datetime, item_id: uuid.UUID
) -> bool:
try:
with get_pool().connection() as conn:
row = conn.execute(
"""
SELECT 1 AS ok
FROM chats c
JOIN chat_members cm ON cm.chat_id = c.id
WHERE cm.user_id = %s AND c.created_at = %s AND c.id = %s
""",
(user_id, created_at, item_id),
).fetchone()
except PsycopgError as exc:
raise StoreUnavailable("database unavailable") from exc
return bool(row)
+12
View File
@@ -0,0 +1,12 @@
# Собственные проверки (lab3)
Требуется уже запущенный стенд (`scripts/up.cmd` / `scripts/up.sh`, ≥2 API за LB).
```sh
# из корня репозитория:
cd student-tests
BASE_URL=http://127.0.0.1:8080 python run_all.py
```
`test_idempotency` проверяет общий Idempotency-Key и наблюдение ≥2 `X-Instance-Id`.
Для expiry поднимите API с `SESSION_TTL_SECONDS=2` и повторите `test_sessions`.
+72
View File
@@ -0,0 +1,72 @@
"""Minimal HTTP helpers for student tests (stdlib only)."""
from __future__ import annotations
import base64
import json
import os
import urllib.error
import urllib.request
import uuid
BASE_URL = os.environ.get("BASE_URL", "http://127.0.0.1:8080").rstrip("/")
def call(method, path, expected=None, body=None, token=None, headers=None, raw=None):
h = {"Accept": "application/json"}
if token:
h["Authorization"] = "Bearer " + token
if body is not None:
raw = json.dumps(body, ensure_ascii=False).encode("utf-8")
h["Content-Type"] = "application/json"
# Lab3+: POST messages require Idempotency-Key unless caller supplies headers.
if (
method == "POST"
and path.rstrip("/").endswith("/messages")
and (headers is None or "Idempotency-Key" not in headers)
):
h["Idempotency-Key"] = uuid.uuid4().hex
h.update(headers or {})
req = urllib.request.Request(BASE_URL + path, data=raw, headers=h, method=method)
try:
resp = urllib.request.urlopen(req, timeout=30)
except urllib.error.HTTPError as err:
resp = err
data = resp.read()
status = resp.code
if expected is not None:
assert status == expected, (
f"{method} {path}: expected {expected}, got {status}: {data[:200]!r}"
)
payload = None
ctype = resp.headers.get_content_type()
if ctype == "application/json" and data:
payload = json.loads(data.decode("utf-8"))
assert resp.headers.get("X-Request-Id"), "missing X-Request-Id"
assert resp.headers.get("X-Instance-Id"), "missing X-Instance-Id"
return {"status": status, "headers": resp.headers, "json": payload, "bytes": data}
def register(label: str):
username = f"st_{label}_{uuid.uuid4().hex[:10]}"
password = "T3st-" + uuid.uuid4().hex
body = {
"username": username,
"display_name": f"Student {label}",
"password": password,
}
user = call("POST", "/api/v1/users", 201, body=body)["json"]
credential = base64.b64encode(f"{username}:{password}".encode()).decode()
session = call(
"POST",
"/api/v1/auth/sessions",
201,
headers={"Authorization": "Basic " + credential},
)["json"]
return {
"id": user["id"],
"user": user,
"token": session["token"],
"registration": body,
}
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env python3
"""Run all student tests against a live API (BASE_URL)."""
from __future__ import annotations
import importlib
import sys
import traceback
MODULES = [
"test_api",
"test_concurrency",
"test_sessions",
"test_idempotency",
"test_deps",
]
def main() -> int:
failed = 0
passed = 0
for name in MODULES:
mod = importlib.import_module(name)
tests = [getattr(mod, n) for n in dir(mod) if n.startswith("test_")]
for fn in tests:
label = f"{name}.{fn.__name__}"
try:
fn()
print("OK", label)
passed += 1
except Exception as exc:
failed += 1
print("FAIL", label, ":", exc)
traceback.print_exc()
print(f"Summary: {passed} passed, {failed} failed")
return 1 if failed else 0
if __name__ == "__main__":
sys.path.insert(0, ".")
raise SystemExit(main())
+159
View File
@@ -0,0 +1,159 @@
"""Grade 3/4 API checks: auth, conflict, X-User-Id ignored, logout."""
from __future__ import annotations
import base64
import time
import uuid
from helpers import call, register
def test_username_conflict():
a = register("conflict")
body = dict(a["registration"])
call("POST", "/api/v1/users", 409, body=body)
def test_wrong_password():
a = register("wrongpw")
bad = base64.b64encode(
(a["registration"]["username"] + ":wrong-password-xx").encode()
).decode()
resp = call(
"POST",
"/api/v1/auth/sessions",
401,
headers={"Authorization": "Basic " + bad},
)
assert resp["headers"].get("WWW-Authenticate", "").lower().startswith("basic")
def test_x_user_id_ignored():
a = register("xu")
b = register("xv")
me = call(
"GET",
"/api/v1/users/me",
200,
token=a["token"],
headers={"X-User-Id": b["id"]},
)["json"]
assert me["id"] == a["id"]
def test_basic_on_business_rejected():
a = register("basicbiz")
cred = base64.b64encode(
(a["registration"]["username"] + ":" + a["registration"]["password"]).encode()
).decode()
call(
"GET",
"/api/v1/users/me",
401,
headers={"Authorization": "Basic " + cred},
)
def test_logout_revokes():
a = register("logout")
call("DELETE", "/api/v1/auth/sessions/current", 204, token=a["token"])
call("GET", "/api/v1/users/me", 401, token=a["token"])
def test_chat_requires_caller_member():
a = register("owner")
b = register("peer")
call(
"POST",
"/api/v1/chats",
400,
token=a["token"],
body={"title": "no me", "member_ids": [b["id"]]},
)
def test_unknown_member_rolls_back():
a = register("atom")
fake = str(uuid.uuid4())
call(
"POST",
"/api/v1/chats",
400,
token=a["token"],
body={"title": "bad", "member_ids": [a["id"], fake]},
)
def test_whitespace_only_password_rejected():
# Maintainer remark: required strings must not be whitespace-only.
body = {
"username": "ws_" + uuid.uuid4().hex[:10],
"display_name": "Spaces",
"password": " " * 12,
}
call("POST", "/api/v1/users", 400, body=body)
def test_nul_in_text_rejected():
a = register("nul")
peer = register("nulb")
chat = call(
"POST",
"/api/v1/chats",
201,
token=a["token"],
body={"title": "nul-chat", "member_ids": [a["id"], peer["id"]]},
)["json"]
call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
400,
token=a["token"],
body={"text": "hello\u0000world"},
)
def test_login_rejects_json_body():
a = register("bodylogin")
cred = base64.b64encode(
(a["registration"]["username"] + ":" + a["registration"]["password"]).encode()
).decode()
call(
"POST",
"/api/v1/auth/sessions",
400,
body={"unexpected": True},
headers={"Authorization": "Basic " + cred},
)
def test_invalid_basic_encoding_is_401():
# Maintainer remark: malformed Basic must be 401, not 500.
call(
"POST",
"/api/v1/auth/sessions",
401,
headers={"Authorization": "Basic " + base64.b64encode("é".encode()).decode()},
)
def test_unsupported_media_type():
a = register("ctype")
peer = register("ctypeb")
chat = call(
"POST",
"/api/v1/chats",
201,
token=a["token"],
body={"title": "ctype", "member_ids": [a["id"], peer["id"]]},
)["json"]
raw = b'{"text":"x"}'
call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
415,
token=a["token"],
raw=raw,
headers={"Content-Type": "text/plain"},
)
+66
View File
@@ -0,0 +1,66 @@
"""Concurrent registration race and parallel messages."""
from __future__ import annotations
import concurrent.futures
import uuid
from helpers import BASE_URL, call, register
def test_parallel_messages_unique():
a = register("par")
b = register("parb")
chat = call(
"POST",
"/api/v1/chats",
201,
token=a["token"],
body={"title": "parallel", "member_ids": [a["id"], b["id"]]},
)["json"]
def send(i: int):
return call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
201,
token=a["token"],
body={"text": f"msg-{i}"},
)["json"]["id"]
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as pool:
ids = list(pool.map(send, range(100)))
assert len(ids) == len(set(ids)) == 100
def test_username_race_one_winner():
username = "race_" + uuid.uuid4().hex[:12]
password = "T3st-" + uuid.uuid4().hex
body = {
"username": username,
"display_name": "Race",
"password": password,
}
def attempt(_):
import json
import urllib.error
import urllib.request
raw = json.dumps(body).encode("utf-8")
req = urllib.request.Request(
BASE_URL + "/api/v1/users",
data=raw,
headers={"Content-Type": "application/json", "Accept": "application/json"},
method="POST",
)
try:
resp = urllib.request.urlopen(req, timeout=30)
except urllib.error.HTTPError as err:
resp = err
return resp.code
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
codes = list(pool.map(attempt, range(8)))
assert codes.count(201) == 1, codes
assert codes.count(409) == 7, codes
+68
View File
@@ -0,0 +1,68 @@
"""Dependency failure checks (Redis/Postgres). Optional: RUN_DEPS_TEST=1."""
from __future__ import annotations
import os
import subprocess
import time
from helpers import BASE_URL, call, register
def _compose(*args: str) -> None:
cmd = ["docker", "compose", "--env-file", ".env", *args]
subprocess.check_call(cmd, cwd=os.path.join(os.path.dirname(__file__), ".."))
def _wait_ready(timeout_s: int = 60) -> None:
deadline = time.time() + timeout_s
while time.time() < deadline:
try:
if call("GET", "/health/ready")["status"] == 200:
return
except Exception:
pass
time.sleep(1)
raise AssertionError("stack did not become ready in time")
def test_redis_stop_returns_503_or_unready():
if os.environ.get("RUN_DEPS_TEST") != "1":
print("SKIP test_redis_stop_returns_503_or_unready (set RUN_DEPS_TEST=1)")
return
actor = register("deps")
_compose("stop", "redis")
time.sleep(2)
try:
ready = call("GET", "/health/ready")
assert ready["status"] == 503, ready["status"]
me = call("GET", "/api/v1/users/me", token=actor["token"])
assert me["status"] in (401, 503)
finally:
_compose("start", "redis")
_wait_ready()
def test_postgres_stop_returns_503():
# Maintainer remark (level 4): cover PostgreSQL unavailability like Redis.
if os.environ.get("RUN_DEPS_TEST") != "1":
print("SKIP test_postgres_stop_returns_503 (set RUN_DEPS_TEST=1)")
return
actor = register("deps_pg")
_compose("stop", "postgres")
time.sleep(2)
try:
ready = call("GET", "/health/ready")
assert ready["status"] == 503, ready["status"]
# Authenticated business path must fail fast as 503 (not hang / not 500).
me = call("GET", "/api/v1/users/me", token=actor["token"])
assert me["status"] == 503, me["status"]
hist = call(
"GET",
f"/api/v1/chats/{actor['id']}/messages",
token=actor["token"],
)
assert hist["status"] in (400, 404, 503), hist["status"]
finally:
_compose("start", "postgres")
_wait_ready(90)
+139
View File
@@ -0,0 +1,139 @@
"""Lab3: shared Idempotency-Key and multi-replica visibility."""
from __future__ import annotations
import concurrent.futures
import uuid
from helpers import call, register
def _chat_pair():
a = register("idem_a")
b = register("idem_b")
chat = call(
"POST",
"/api/v1/chats",
201,
token=a["token"],
body={"title": "idem", "member_ids": [a["id"], b["id"]]},
)["json"]
return a, b, chat
def test_missing_idempotency_key_400():
a, _b, chat = _chat_pair()
call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
400,
token=a["token"],
body={"text": "no-key"},
headers={"Idempotency-Key": ""},
)
def test_replay_same_key_same_body():
a, _b, chat = _chat_pair()
key = uuid.uuid4().hex
first = call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
201,
token=a["token"],
body={"text": "same"},
headers={"Idempotency-Key": key},
)["json"]
second = call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
201,
token=a["token"],
body={"text": "same"},
headers={"Idempotency-Key": key},
)["json"]
assert first == second
def test_empty_attachment_ids_equals_absent():
# Peer remark: attachment_ids: [] ≡ absent field; must not 400; same idempotency hash.
a, _b, chat = _chat_pair()
key = uuid.uuid4().hex
first = call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
201,
token=a["token"],
body={"text": "with-empty-attachments", "attachment_ids": []},
headers={"Idempotency-Key": key},
)["json"]
second = call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
201,
token=a["token"],
body={"text": "with-empty-attachments"},
headers={"Idempotency-Key": key},
)["json"]
assert first == second
def test_same_key_different_body_409():
a, _b, chat = _chat_pair()
key = uuid.uuid4().hex
call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
201,
token=a["token"],
body={"text": "one"},
headers={"Idempotency-Key": key},
)
call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
409,
token=a["token"],
body={"text": "two"},
headers={"Idempotency-Key": key},
)
def test_parallel_same_key_one_message():
a, b, chat = _chat_pair()
key = uuid.uuid4().hex
text = "parallel-idem-" + key
def send(_):
return call(
"POST",
f"/api/v1/chats/{chat['id']}/messages",
201,
token=a["token"],
body={"text": text},
headers={"Idempotency-Key": key},
)["json"]
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
results = list(pool.map(send, range(8)))
ids = {m["id"] for m in results}
assert len(ids) == 1, ids
page = call(
"GET",
f"/api/v1/chats/{chat['id']}/messages?limit=100",
200,
token=b["token"],
)["json"]
assert sum(1 for m in page["items"] if m["text"] == text) == 1
def test_multiple_instance_ids_observed():
a = register("inst")
seen = set()
for _ in range(40):
seen.add(
call("GET", "/api/v1/users/me", 200, token=a["token"])["headers"].get(
"X-Instance-Id"
)
)
assert len(seen) >= 2, f"expected ≥2 replicas via LB, got {seen}"
+34
View File
@@ -0,0 +1,34 @@
"""Session expiry (short TTL) — run against stack with SESSION_TTL_SECONDS=2."""
from __future__ import annotations
import base64
import os
import time
from helpers import call, register
def test_session_expiry_if_short_ttl():
ttl = int(os.environ.get("SESSION_TTL_SECONDS", "3600"))
if ttl > 5:
print("SKIP expiry: SESSION_TTL_SECONDS=%s (need <=5 for this test)" % ttl)
return
a = register("expire")
time.sleep(ttl + 1.5)
call("GET", "/api/v1/users/me", 401, token=a["token"])
def test_login_after_register():
a = register("relogin")
cred = base64.b64encode(
(a["registration"]["username"] + ":" + a["registration"]["password"]).encode()
).decode()
session = call(
"POST",
"/api/v1/auth/sessions",
201,
headers={"Authorization": "Basic " + cred},
)["json"]
assert session["token_type"] == "Bearer"
assert session["user"]["id"] == a["id"]