34 changed files with 2261 additions and 35 deletions
+4 -3
View File
@@ -4,10 +4,11 @@ APP_HOST=0.0.0.0
APP_PORT=8080
APP_ENV=lab
SESSION_TTL_SECONDS=3600
IDEMPOTENCY_TTL_SECONDS=86400
DB_TIMEOUT_MS=3000
DB_MAX_CONNS=10
# Choose credentials locally. File paths contain no secret values.
DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
SESSION_STORE=redis
CACHE_URL_FILE=/run/secrets/cache_url
API_REPLICAS=2
IDEMPOTENCY_TTL_SECONDS=86400
+14 -14
View File
@@ -1,18 +1,18 @@
## Лабораторная 3
Автор / группа: TODO
Целевая оценка (3/4/5): TODO
Трек 5, если заявлен: TODO
Автор / группа: Меркулова М.А. / БВТ2303
Целевая оценка (3/4/5): 4
Трек 5, если заявлен: не заявлен
Что реализовано и что перенесено: TODO
Команды запуска из чистого clone: TODO
Результаты проверок и ссылка на REPORT.md: TODO
Ограничения / известные ошибки: TODO
Источники и использование ИИ: TODO
Что реализовано и что перенесено: перенесён совместимый Go API lab2; добавлены Nginx, две stateless API-реплики, общий Idempotency-Key, X-Instance-Id, отдельный migrate service, failover и rolling update.
Команды запуска из чистого clone: `./scripts/init-local.sh`; `docker compose --env-file .env up -d --build --wait`; `make test`.
Результаты проверок и ссылка на REPORT.md: все команды и локальные результаты приведены в `REPORT.md` и `evidence/lab3-results.md`.
Ограничения / известные ошибки: один Docker-host, один Nginx, PostgreSQL и Redis остаются SPOF; multi-host failover не заявлен.
Источники и использование ИИ: контракт и материалы преподавателя; ИИ использован для проектирования, кода и проверок, результаты воспроизведены локально.
- [ ] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
- [ ] Контракт и публичные тесты не ослаблены.
- [ ] make check и make test выполнены; приложен реальный вывод.
- [ ] Сценарии отказов/безопасности своего уровня воспроизведены.
- [ ] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
- [ ] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
- [x] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
- [x] Контракт и публичные тесты не ослаблены.
- [x] make check и make test выполнены; приложен реальный вывод.
- [x] Сценарии отказов/безопасности своего уровня воспроизведены.
- [x] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
- [x] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
+15
View File
@@ -0,0 +1,15 @@
FROM golang:1.26.8-alpine3.23@sha256:a8fa79c5bd40d880b52bd3b6d7669ecdcfd00e85facdd427d279efb5ddd79cb1 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY cmd/ ./cmd/
COPY internal/ ./internal/
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /messenger ./cmd/messenger
FROM scratch
COPY --from=build /messenger /messenger
USER 65532:65532
ENV APP_HOST=0.0.0.0 APP_PORT=8080
EXPOSE 8080
HEALTHCHECK --interval=10s --timeout=3s --start-period=5s --retries=3 CMD ["/messenger", "healthcheck"]
ENTRYPOINT ["/messenger"]
+110 -18
View File
@@ -1,37 +1,129 @@
# Отчёт — лабораторная 3
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: Меркулова М.А., БВТ2303
- Целевая оценка: 4
- Ветка: `solution/BVT2303-Merkulova`; commit/PR указываются после публикации
- Реализация: Go 1.26, PostgreSQL 17.6, Redis 8.2.1, Nginx 1.29.1, Docker Compose
- Библиотеки приложения: `pgx/v5`, `go-redis/v9`, `x/crypto/scrypt`
- Помощь ИИ: использовалась для проектирования, написания и проверки кода; требования сверялись с файлами текущего skeleton, все приведённые результаты получены локальным запуском
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Нужны Docker с Compose, Python 3.9+ и `make`. Локальный Go нужен только для `go test`; Docker-образ собирает бинарный файл самостоятельно.
```sh
./scripts/init-local.sh
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --wait
docker compose --env-file .env ps
make check
make test
```
`init-local.sh` создаёт случайные пароли, URL подключения и `.env` в игнорируемых Git файлах. Единственная публичная точка — `http://127.0.0.1:8080` на Nginx. Порты API-реплик, PostgreSQL и Redis на host не публикуются.
Остальные проверки:
```sh
go test ./...
go vet ./...
ACTION_SCRIPT=./scripts/restart.sh make test-persistence
ACTION_SCRIPT=./scripts/stop-one.sh make test-failover
./scripts/start-api.sh
python3 student-tests/verify_idempotency.py
python3 student-tests/failover_experiment.py
python3 student-tests/verify_unready_backend.py
python3 student-tests/rolling_update_experiment.py
python3 student-tests/verify_grade4.py
python3 student-tests/verify_expiry.py
python3 student-tests/verify_dependencies.py
./scripts/explain_history.sh
```
Безопасная остановка с сохранением данных:
```sh
docker compose --env-file .env down
```
Намеренное удаление всех учебных данных и сессий:
```sh
docker compose --env-file .env down -v
```
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab2, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
```mermaid
flowchart LR
C[Клиент] -->|HTTP :8080| L[Nginx]
L --> A1[API api-1]
L --> A2[API api-2]
A1 --> P[(PostgreSQL)]
A2 --> P
A1 --> R[(Redis + AOF)]
A2 --> R
M[Одноразовый migrate] --> P
```
Обе API-реплики собраны из одного образа, не имеют локального бизнес-состояния и используют общие PostgreSQL и Redis. PostgreSQL хранит пользователей, чаты, сообщения и идемпотентность. Redis хранит общие Bearer-сессии как `SHA-256(token) → user UUID` с TTL и AOF. Поэтому запросы одной сессии обслуживаются обеими репликами без sticky sessions.
Nginx выполняет round-robin между готовыми репликами. В его контейнере `health-aware.sh` опрашивает `/health/ready` каждой API-реплики по frontend-сети с паузой 1 секунду между циклами и при изменении набора готовых реплик обновляет upstream и перезагружает конфигурацию Nginx. Если обе реплики неготовы, он оставляет обе в upstream: API может вернуть обычные JSON-ответы `401` и `503` вместо ошибки Nginx «no live upstreams». Пока обновление upstream ещё не произошло, Nginx повторяет идемпотентный GET на второй реплике при `503`, сетевой ошибке или таймауте. Автоматический повтор POST после отправки запроса не включён; после обнаружения отказа POST идёт только на готовую реплику. Docker healthcheck API запускается каждые 3 секунды и используется для начального запуска и наблюдения. Connect timeout равен 500 ms, Docker DNS перечитывается с периодом 2 секунды. `X-Instance-Id` равен стабильным alias `api-1` или `api-2`.
API обрабатывает SIGTERM через `http.Server.Shutdown` с бюджетом 8 секунд; Compose даёт 10 секунд до принудительной остановки. Таймауты HTTP: header 5 s, read/write 10 s, idle 30 s. SQL получает `DB_TIMEOUT_MS=3000`; каждая реплика имеет пул максимум из 10 соединений.
Миграции выполняет отдельный одноразовый сервис `migrate` до API. При rolling update тот же сервис запускается ещё раз до замены первой реплики; уже применённые версии он пропускает. Таблица `schema_migrations` и транзакция предотвращают повторное применение. Миграция 002 аддитивно добавляет таблицу идемпотентности, поэтому старая схема lab2 остаётся читаемой во время последовательной замены процессов.
Для `POST /messages` обязателен `Idempotency-Key`. Область уникальности — `(user_id, method, path, key)`. В одной PostgreSQL-транзакции создаётся запись ключа, сообщение и ссылка на результат. Конкурентный повтор блокируется общим уникальным ключом и после commit получает исходное сообщение с `201`. Тот же ключ с другим логическим JSON даёт `409`. Ключ действует минимум `IDEMPOTENCY_TTL_SECONDS=86400`; при повторном использовании просроченная запись удаляется. Записи ключей, которые больше не используют, пока остаются в таблице.
Из lab2 сохранены PostgreSQL, Redis-сессии, scrypt-пароли, транзакционное создание чатов, cursor pagination, request ID, JSON-ошибки, health endpoints и graceful shutdown. Новые обязательные изменения lab3: Nginx, две реплики, `X-Instance-Id`, общий `Idempotency-Key`, отдельные миграции и отказ одной API-реплики.
## Ресурсные границы и секреты
Compose реально применил следующие limits:
| Сервис | CPU | RAM |
| --- | ---: | ---: |
| `api1`, `api2` | по 0.50 | по 192 MiB |
| `nginx` | 0.25 | 64 MiB |
| `postgres` | 1.00 | 512 MiB |
| `redis` | 0.25 | 128 MiB |
| `migrate` | 0.25 | 128 MiB |
PostgreSQL и Redis находятся только во внутренней backend-сети. Nginx видит API по frontend-сети. API получает только URL PostgreSQL и Redis; Nginx не получает секреты, PostgreSQL не получает Redis-пароль. Секретные значения создаются локально в `secrets/` и не входят в Git.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| Критерий README | Команда/сценарий | Наблюдаемый результат | Доказательство |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
| Публичный API | `make test` | PASS lab3, включая `X-Instance-Id` и повтор сообщения | `evidence/lab3-results.md` |
| Две реплики через LB | шесть `GET /health/live` | `api-2`, `api-1` по очереди | `evidence/lab3-results.md` |
| Persistence | `ACTION_SCRIPT=... make test-persistence` | прежний токен и сообщение доступны после restart всех процессов и хранилищ | `evidence/lab3-results.md` |
| Отказ любой реплики | официальный failover и последовательный эксперимент | остановлены реальные `api1` и `api2`; подтверждённые сообщения сохранились | `evidence/lab3-results.md` |
| Метрики двух отказов | `failover_experiment.py` | 642.783 и 569.984 successful RPS; 0 ошибок; 0.600/0.610 s до подтверждения survivor | `evidence/failover-grade4.json` |
| Неготовая реплика с работающим HTTP | `verify_unready_backend.py` | после отключения backend-сети `api1` и состояния `unhealthy`: 40 GET и 10 POST через LB обслужила `api2` | `evidence/lab3-results.md` |
| Общая идемпотентность | `verify_idempotency.py` | 12 параллельных запросов через обе реплики, один message ID; другой body — 409 | `evidence/idempotency.json` |
| Rolling update | `rolling_update_experiment.py` | поочерёдно пересозданы обе реплики; 25 388 успехов, 0 ошибок | `evidence/rolling-update.json` |
| Отказ Redis/PostgreSQL | `verify_dependencies.py` | осмысленные 503, после возврата сервисов состояние восстановлено | `evidence/lab3-results.md` |
| Индекс истории | `explain_history.sh` | `Index Scan using messages_chat_history_idx` на 10 000 сообщений | `evidence/explain-history.txt` |
| Возможности lab2 | `verify_grade4.py`, `verify_expiry.py`, `go test ./...` | гонка username, rollback, logout, TTL и password KDF прошли | `evidence/lab3-results.md` |
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Стенд повторно проверен 16 октября 2026 года на Docker Desktop/macOS arm64. До измерений выполнены smoke и несколько запросов прогрева. Все данные синтетические. Failover использует один закрытый цикл HTTP-клиента 8 секунд на реплику. Rolling update использует четыре закрытых параллельных клиента; метрики задержки относятся только к успешным ответам и включают клиентские и LB расходы.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| Воздействие | Успехи / ошибки | Наблюдаемое переключение | Потеря подтверждённых данных |
| --- | ---: | ---: | --- |
| Stop `api-1` | 5143 / 0 за 8.001 s | 0.600 s | нет |
| Stop `api-2` | 4560 / 0 за 8.000 s | 0.610 s | нет |
| Rolling replace `api-1`, затем `api-2` | 25388 / 0 за 16.832 s | обе реплики наблюдались | нет |
Rolling update: 1508.329 successful RPS, p50 2.221 ms, p95 3.998 ms, p99 6.040 ms. Это локальное наблюдение, а не универсальная оценка производительности. Closed-loop профиль имеет coordinated omission и не моделирует фиксированный внешний arrival rate.
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
Стенд выдерживает остановку одного API-процесса. Все компоненты работают на одном Docker-host, поэтому потеря host останавливает систему. Nginx, PostgreSQL и Redis представлены по одному и остаются single points of failure. Остановка PostgreSQL или Redis корректно даёт 503, но бизнес-функции временно недоступны.
Nginx исключает неготовую реплику по HTTP-проверке `/health/ready`; между отказом и очередной проверкой возможна краткая ошибка POST. Docker healthcheck отвечает за запуск и наблюдаемость. Rolling update проверяет последовательную замену процессов одной совместимой версией на одном host. Автоматического rollback образа, репликации БД, отдельного backup/restore и multi-host orchestration в lab3 нет.
Просроченная запись ключа удаляется при повторном использовании того же ключа. Для длительной эксплуатации нужна периодическая очистка остальных записей. Приведённые RPS и latency нельзя переносить на другой компьютер без повторения опыта с теми же ресурсами и набором данных.
+169
View File
@@ -0,0 +1,169 @@
package main
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/redis/go-redis/v9"
"mtusi-messenger/internal/database"
"mtusi-messenger/internal/messenger"
)
func env(name, fallback string) string {
if value := os.Getenv(name); value != "" {
return value
}
return fallback
}
func secret(envName, fileEnv string) (string, error) {
if value := os.Getenv(envName); value != "" {
return value, nil
}
path := os.Getenv(fileEnv)
if path == "" {
return "", fmt.Errorf("set %s or %s", envName, fileEnv)
}
b, err := os.ReadFile(path)
if err != nil {
return "", err
}
value := strings.TrimSpace(string(b))
if value == "" {
return "", fmt.Errorf("%s is empty", path)
}
return value, nil
}
func positiveInt(name string, fallback int) (int, error) {
raw := env(name, strconv.Itoa(fallback))
n, err := strconv.Atoi(raw)
if err != nil || n < 1 {
return 0, fmt.Errorf("invalid %s", name)
}
return n, nil
}
func main() {
port := env("APP_PORT", "8080")
if len(os.Args) > 1 && os.Args[1] == "healthcheck" {
client := &http.Client{Timeout: 2 * time.Second}
resp, err := client.Get("http://127.0.0.1:" + port + "/health/ready")
if err != nil || resp.StatusCode != http.StatusOK {
os.Exit(1)
}
_ = resp.Body.Close()
return
}
host := env("APP_HOST", "0.0.0.0")
if _, err := strconv.Atoi(port); err != nil {
log.Fatal("invalid APP_PORT")
}
ttlSeconds, err := positiveInt("SESSION_TTL_SECONDS", 3600)
if err != nil {
log.Fatal(err)
}
idempotencyTTLSeconds, err := positiveInt("IDEMPOTENCY_TTL_SECONDS", 86400)
if err != nil || idempotencyTTLSeconds < 86400 {
log.Fatal("IDEMPOTENCY_TTL_SECONDS must be at least 86400")
}
dbTimeoutMS, err := positiveInt("DB_TIMEOUT_MS", 3000)
if err != nil {
log.Fatal(err)
}
maxConns, err := positiveInt("DB_MAX_CONNS", 10)
if err != nil {
log.Fatal(err)
}
databaseURL, err := secret("DATABASE_URL", "DATABASE_URL_FILE")
if err != nil {
log.Fatal(err)
}
logger := log.New(os.Stdout, "", log.LstdFlags|log.LUTC)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
config, err := pgxpool.ParseConfig(databaseURL)
if err != nil {
log.Fatal("invalid database configuration")
}
config.MaxConns = int32(maxConns)
config.MinConns = 1
config.MaxConnLifetime = 30 * time.Minute
var pool *pgxpool.Pool
for {
pool, err = pgxpool.NewWithConfig(ctx, config)
if err == nil {
err = pool.Ping(ctx)
}
if err == nil {
break
}
if pool != nil {
pool.Close()
}
if ctx.Err() != nil {
log.Fatal("database unavailable during startup")
}
time.Sleep(time.Second)
}
defer pool.Close()
if len(os.Args) > 1 && os.Args[1] == "migrate" {
if err = database.Migrate(ctx, pool); err != nil {
log.Fatalf("migration failed: %v", err)
}
logger.Print("migrations complete")
return
}
instanceID := env("INSTANCE_ID", "")
if instanceID == "" {
log.Fatal("INSTANCE_ID is required")
}
cacheURL, err := secret("CACHE_URL", "CACHE_URL_FILE")
if err != nil {
log.Fatal(err)
}
redisOptions, err := redis.ParseURL(cacheURL)
if err != nil {
log.Fatal("invalid cache configuration")
}
sessions := redis.NewClient(redisOptions)
defer sessions.Close()
for {
if err = sessions.Ping(ctx).Err(); err == nil {
break
}
if ctx.Err() != nil {
log.Fatal("session store unavailable during startup")
}
time.Sleep(time.Second)
}
dbTimeout := time.Duration(dbTimeoutMS) * time.Millisecond
srv := &http.Server{Addr: host + ":" + port, Handler: messenger.NewHandler(pool, sessions, time.Duration(ttlSeconds)*time.Second, time.Duration(idempotencyTTLSeconds)*time.Second, dbTimeout, instanceID, logger), ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 10 * time.Second, WriteTimeout: 10 * time.Second, IdleTimeout: 30 * time.Second}
stop := make(chan os.Signal, 1)
signal.Notify(stop, syscall.SIGTERM, os.Interrupt)
done := make(chan struct{})
go func() {
defer close(done)
<-stop
logger.Print("shutdown started")
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 8*time.Second)
defer shutdownCancel()
if shutdownErr := srv.Shutdown(shutdownCtx); shutdownErr != nil {
logger.Printf("shutdown error: %v", shutdownErr)
}
logger.Print("shutdown complete")
}()
logger.Printf("listening on %s", srv.Addr)
if err = srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
logger.Fatal(err)
}
<-done
}
+174
View File
@@ -0,0 +1,174 @@
name: mtusi-lab3
x-api-common: &api-common
image: mtusi-lab3-api:local
environment: &api-environment
APP_HOST: ${APP_HOST:-0.0.0.0}
APP_PORT: 8080
APP_ENV: ${APP_ENV:-lab}
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400}
DB_TIMEOUT_MS: ${DB_TIMEOUT_MS:-3000}
DB_MAX_CONNS: ${DB_MAX_CONNS:-10}
DATABASE_URL_FILE: ${DATABASE_URL_FILE:-/run/secrets/database_url}
CACHE_URL_FILE: ${CACHE_URL_FILE:-/run/secrets/cache_url}
secrets:
- database_url
- cache_url
networks:
- frontend
- backend
healthcheck:
test: ["CMD", "/messenger", "healthcheck"]
interval: 3s
timeout: 2s
retries: 20
start_period: 3s
restart: unless-stopped
init: true
stop_grace_period: 10s
deploy:
resources:
limits:
cpus: "0.50"
memory: 192M
services:
postgres:
image: postgres:17.6-alpine3.22
environment:
POSTGRES_DB: messenger
POSTGRES_USER: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
secrets:
- postgres_password
volumes:
- postgres_data:/var/lib/postgresql/data
networks:
- backend
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 3s
timeout: 3s
retries: 20
restart: unless-stopped
deploy:
resources:
limits:
cpus: "1.00"
memory: 512M
redis:
image: redis:8.2.1-alpine3.22
command:
- /bin/sh
- -ec
- exec redis-server --appendonly yes --appendfsync always --requirepass "$$(cat /run/secrets/redis_password)"
secrets:
- redis_password
volumes:
- redis_data:/data
networks:
- backend
healthcheck:
test: ["CMD-SHELL", "redis-cli --no-auth-warning -a \"$$(cat /run/secrets/redis_password)\" ping | grep PONG"]
interval: 3s
timeout: 3s
retries: 20
restart: unless-stopped
deploy:
resources:
limits:
cpus: "0.25"
memory: 128M
migrate:
image: mtusi-lab3-api:local
build:
context: .
dockerfile: Dockerfile
command: ["migrate"]
environment:
<<: *api-environment
secrets:
- database_url
networks:
- backend
depends_on:
postgres:
condition: service_healthy
restart: "no"
deploy:
resources:
limits:
cpus: "0.25"
memory: 128M
api1:
<<: *api-common
environment:
<<: *api-environment
INSTANCE_ID: api-1
depends_on:
migrate:
condition: service_completed_successfully
redis:
condition: service_healthy
api2:
<<: *api-common
environment:
<<: *api-environment
INSTANCE_ID: api-2
depends_on:
migrate:
condition: service_completed_successfully
redis:
condition: service_healthy
nginx:
image: nginx:1.29.1-alpine3.22
command: ["/bin/sh", "/health-aware.sh"]
volumes:
- ./infra/nginx.conf:/etc/nginx/nginx.conf:ro
- ./infra/health-aware.sh:/health-aware.sh:ro
networks:
- frontend
depends_on:
api1:
condition: service_healthy
api2:
condition: service_healthy
ports:
- "127.0.0.1:${APP_PORT:-8080}:8080"
healthcheck:
test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1:8080/health/ready"]
interval: 3s
timeout: 2s
retries: 20
restart: unless-stopped
stop_grace_period: 5s
deploy:
resources:
limits:
cpus: "0.25"
memory: 64M
secrets:
postgres_password:
file: ./secrets/postgres_password
redis_password:
file: ./secrets/redis_password
database_url:
file: ./secrets/database_url
cache_url:
file: ./secrets/cache_url
volumes:
postgres_data:
redis_data:
networks:
frontend:
backend:
internal: true
+24
View File
@@ -0,0 +1,24 @@
INSERT 0 1
INSERT 0 1
INSERT 0 1
DELETE 0
INSERT 0 10000
ANALYZE
fixture_messages
------------------
10000
(1 row)
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.41..20.19 rows=50 width=84) (actual time=0.006..0.018 rows=50 loops=1)
Buffers: shared hit=53
-> Index Scan using messages_chat_history_idx on messages (cost=0.41..4018.79 rows=10160 width=84) (actual time=0.005..0.015 rows=50 loops=1)
Index Cond: (chat_id = '00000000-0000-4000-8000-000000000002'::uuid)
Buffers: shared hit=53
Planning:
Buffers: shared hit=20
Planning Time: 0.069 ms
Execution Time: 0.028 ms
(9 rows)
+21
View File
@@ -0,0 +1,21 @@
{
"workload": "single-client closed-loop POST messages",
"runs": [
{
"stopped_instance": "api-1",
"duration_seconds": 8.001,
"successful_requests": 5143,
"failed_requests": 0,
"successful_rps": 642.783,
"first_survivor_success_seconds": 0.6
},
{
"stopped_instance": "api-2",
"duration_seconds": 8.0,
"successful_requests": 4560,
"failed_requests": 0,
"successful_rps": 569.984,
"first_survivor_success_seconds": 0.61
}
]
}
+9
View File
@@ -0,0 +1,9 @@
{
"parallel_requests": 12,
"instances": [
"api-1",
"api-2"
],
"unique_message_ids": 1,
"conflicting_reuse_status": 409
}
+78
View File
@@ -0,0 +1,78 @@
# Результаты проверок лабораторной 3
Повторная проверка: 2026-10-16. Платформа: Docker Desktop на macOS arm64. Ветка: `solution/BVT2303-Merkulova`.
## Сборка и контракт
```text
$ go test ./...
ok mtusi-messenger/internal/messenger
$ go vet ./...
(ошибок нет)
$ make check
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
$ make test
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
```
Шесть последовательных запросов через единственный публичный порт Nginx вернули `api-2`, `api-1`, `api-2`, `api-1`, `api-2`, `api-1`.
## Persistence и официальный failover
```text
$ ACTION_SCRIPT=./scripts/restart.sh make test-persistence
PASS: persistence preserved confirmed data and existing session.
```
Официальный failover после изменения Nginx: две реплики наблюдались до отказа, 126 записей успешны, ошибок 0, first recovery 0.834 s. После этого `failover_experiment.py` последовательно остановил `api1` и `api2`, каждый раз восстанавливая предыдущую реплику.
## Собственные проверки уровня 4
```text
$ python3 student-tests/verify_idempotency.py
PASS: 12 parallel requests; instances api-1/api-2; one unique message; conflict=409
$ python3 student-tests/failover_experiment.py
PASS: api-1 5143/0, 642.783 RPS, 0.600 s
PASS: api-2 4560/0, 569.984 RPS, 0.610 s
$ python3 student-tests/verify_unready_backend.py
PASS: api1 unhealthy with frontend connection; 40 GETs and 10 POSTs served by api2
$ python3 student-tests/rolling_update_experiment.py
PASS: 25388 successful, 0 failed, 1508.329 RPS
p50=2.221 ms p95=3.998 ms p99=6.040 ms
instances=api-1,api-2
```
Сырые безопасные summaries: `idempotency.json`, `failover-grade4.json`, `rolling-update.json`.
## Сохранённые проверки lab2
```text
$ python3 student-tests/verify_grade4.py
PASS: registration race, transaction rollback, bad password and logout
$ python3 student-tests/verify_expiry.py
PASS: two-second Redis session expired and was rejected
$ python3 student-tests/verify_dependencies.py
PASS: Redis/PostgreSQL failures returned 503; anonymous access stayed 401; state recovered
```
## Ресурсные limits
```text
api1 cpu=500000000 memory=201326592
api2 cpu=500000000 memory=201326592
nginx cpu=250000000 memory=67108864
postgres cpu=1000000000 memory=536870912
redis cpu=250000000 memory=134217728
```
## SQL-план
`./scripts/explain_history.sh` создал 10 000 сообщений. PostgreSQL выбрал `Index Scan using messages_chat_history_idx`; execution time данного локального запуска с прогретым cache — 0.028 ms. Полный вывод находится в `explain-history.txt`.
+16
View File
@@ -0,0 +1,16 @@
{
"workload": "four-client closed-loop POST messages",
"elapsed_seconds": 16.832,
"successful_requests": 25388,
"failed_requests": 0,
"successful_rps": 1508.329,
"latency_ms_successes": {
"p50": 2.221,
"p95": 3.998,
"p99": 6.04
},
"instances": [
"api-1",
"api-2"
]
}
+19
View File
@@ -0,0 +1,19 @@
module mtusi-messenger
go 1.26.0
require (
github.com/jackc/pgx/v5 v5.7.6
github.com/redis/go-redis/v9 v9.14.0
golang.org/x/crypto v0.42.0
)
require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/text v0.29.0 // indirect
)
+38
View File
@@ -0,0 +1,38 @@
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.7.6 h1:rWQc5FwZSPX58r1OQmkuaNicxdmExaEz5A2DO2hUuTk=
github.com/jackc/pgx/v5 v5.7.6/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/redis/go-redis/v9 v9.14.0 h1:u4tNCjXOyzfgeLN+vAZaW1xUooqWDqVEsZN0U01jfAE=
github.com/redis/go-redis/v9 v9.14.0/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
golang.org/x/crypto v0.42.0 h1:chiH31gIWm57EkTXpwnqf8qeuMUi0yekh6mT2AvFlqI=
golang.org/x/crypto v0.42.0/go.mod h1:4+rDnOTJhQCx2q7/j6rAN5XDw8kPjeaXEUR2eL94ix8=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+43
View File
@@ -0,0 +1,43 @@
#!/bin/sh
set -eu
upstream=/run/nginx-upstream.conf
write_servers() {
if [ "$1" = 1 ]; then
printf '%s\n' 'server api1:8080 resolve max_fails=0;'
fi
if [ "$2" = 1 ]; then
printf '%s\n' 'server api2:8080 resolve max_fails=0;'
fi
}
# With no healthy replica, keep both reachable so API can return its normal
# 401/503 JSON responses instead of Nginx reporting "no live upstreams".
write_servers 1 1 > "$upstream"
monitor() {
sleep 2
while :; do
ready1=0
ready2=0
wget -q -T 2 -O /dev/null http://api1:8080/health/ready && ready1=1 || true
wget -q -T 2 -O /dev/null http://api2:8080/health/ready && ready2=1 || true
if [ "$ready1" = 0 ] && [ "$ready2" = 0 ]; then
ready1=1
ready2=1
fi
next=$(mktemp /run/nginx-upstream.XXXXXX)
write_servers "$ready1" "$ready2" > "$next"
if cmp -s "$next" "$upstream"; then
rm -f "$next"
else
mv "$next" "$upstream"
nginx -t >/dev/null 2>&1 && nginx -s reload
fi
sleep 1
done
}
monitor &
exec nginx -g 'daemon off;'
+31
View File
@@ -0,0 +1,31 @@
events {}
http {
access_log /dev/stdout;
error_log /dev/stderr notice;
resolver 127.0.0.11 valid=2s ipv6=off;
upstream messenger_api {
zone messenger_api 64k;
include /run/nginx-upstream.conf;
keepalive 16;
}
server {
listen 8080;
location / {
proxy_pass http://messenger_api;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 500ms;
proxy_send_timeout 12s;
proxy_read_timeout 12s;
proxy_next_upstream error timeout http_502 http_503 http_504;
proxy_next_upstream_tries 2;
}
}
}
+64
View File
@@ -0,0 +1,64 @@
package database
import (
"context"
"embed"
"fmt"
"io/fs"
"sort"
"strconv"
"strings"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
//go:embed migrations/*.sql
var migrations embed.FS
func Migrate(ctx context.Context, pool *pgxpool.Pool) error {
tx, err := pool.BeginTx(ctx, pgx.TxOptions{})
if err != nil {
return err
}
defer tx.Rollback(ctx)
if _, err = tx.Exec(ctx, `SELECT pg_advisory_xact_lock(202602)`); err != nil {
return err
}
if _, err = tx.Exec(ctx, `CREATE TABLE IF NOT EXISTS schema_migrations (version integer PRIMARY KEY, applied_at timestamptz NOT NULL DEFAULT clock_timestamp())`); err != nil {
return err
}
entries, err := fs.ReadDir(migrations, "migrations")
if err != nil {
return err
}
sort.Slice(entries, func(i, j int) bool { return entries[i].Name() < entries[j].Name() })
for _, entry := range entries {
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".sql") {
continue
}
part := strings.SplitN(entry.Name(), "_", 2)[0]
version, parseErr := strconv.Atoi(part)
if parseErr != nil {
return fmt.Errorf("invalid migration name %s", entry.Name())
}
var applied bool
if err = tx.QueryRow(ctx, `SELECT EXISTS(SELECT 1 FROM schema_migrations WHERE version=$1)`, version).Scan(&applied); err != nil {
return err
}
if applied {
continue
}
sqlBytes, readErr := migrations.ReadFile("migrations/" + entry.Name())
if readErr != nil {
return readErr
}
if _, err = tx.Exec(ctx, string(sqlBytes)); err != nil {
return fmt.Errorf("migration %s: %w", entry.Name(), err)
}
if _, err = tx.Exec(ctx, `INSERT INTO schema_migrations(version) VALUES($1)`, version); err != nil {
return err
}
}
return tx.Commit(ctx)
}
@@ -0,0 +1,31 @@
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
display_name text NOT NULL CHECK (char_length(btrim(display_name)) > 0 AND char_length(display_name) <= 100),
password_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL CHECK (char_length(btrim(title)) > 0 AND char_length(title) <= 100),
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users(id) ON DELETE RESTRICT,
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX chat_members_user_chat_idx ON chat_members (user_id, chat_id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
sender_id uuid NOT NULL REFERENCES users(id) ON DELETE RESTRICT,
text text NOT NULL CHECK (char_length(btrim(text)) > 0 AND char_length(text) <= 4000),
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE INDEX messages_chat_history_idx ON messages (chat_id, created_at, id);
@@ -0,0 +1,13 @@
CREATE TABLE message_idempotency (
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
method text NOT NULL,
path text NOT NULL,
idempotency_key text NOT NULL,
request_hash bytea NOT NULL,
message_id uuid REFERENCES messages(id) ON DELETE CASCADE,
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
PRIMARY KEY (user_id, method, path, idempotency_key)
);
CREATE INDEX message_idempotency_created_at_idx
ON message_idempotency (created_at);
+44
View File
@@ -0,0 +1,44 @@
package messenger
import (
"crypto/rand"
"crypto/subtle"
"encoding/base64"
"fmt"
"strconv"
"strings"
"golang.org/x/crypto/scrypt"
)
const scryptN, scryptR, scryptP = 32768, 8, 1
func hashPassword(password string) (string, error) {
salt := make([]byte, 16)
if _, err := rand.Read(salt); err != nil {
return "", err
}
hash, err := scrypt.Key([]byte(password), salt, scryptN, scryptR, scryptP, 32)
if err != nil {
return "", err
}
b64 := base64.RawStdEncoding
return fmt.Sprintf("scrypt$%d$%d$%d$%s$%s", scryptN, scryptR, scryptP, b64.EncodeToString(salt), b64.EncodeToString(hash)), nil
}
func verifyPassword(encoded, password string) bool {
parts := strings.Split(encoded, "$")
if len(parts) != 6 || parts[0] != "scrypt" {
return false
}
n, e1 := strconv.Atoi(parts[1])
r, e2 := strconv.Atoi(parts[2])
p, e3 := strconv.Atoi(parts[3])
salt, e4 := base64.RawStdEncoding.DecodeString(parts[4])
expected, e5 := base64.RawStdEncoding.DecodeString(parts[5])
if e1 != nil || e2 != nil || e3 != nil || e4 != nil || e5 != nil || len(expected) != 32 {
return false
}
actual, err := scrypt.Key([]byte(password), salt, n, r, p, len(expected))
return err == nil && subtle.ConstantTimeCompare(actual, expected) == 1
}
+33
View File
@@ -0,0 +1,33 @@
package messenger
import (
"strings"
"testing"
)
func TestPasswordHash(t *testing.T) {
const password = "long-test-password"
first, err := hashPassword(password)
if err != nil {
t.Fatal(err)
}
second, err := hashPassword(password)
if err != nil {
t.Fatal(err)
}
if first == password || strings.Contains(first, password) {
t.Fatal("hash contains the original password")
}
if first == second {
t.Fatal("independent hashes must use different salts")
}
if !verifyPassword(first, password) {
t.Fatal("correct password was rejected")
}
if verifyPassword(first, "wrong-test-password") {
t.Fatal("wrong password was accepted")
}
if verifyPassword("damaged-hash", password) {
t.Fatal("malformed hash was accepted")
}
}
+673
View File
@@ -0,0 +1,673 @@
package messenger
import (
"bytes"
"context"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"net/http"
"net/url"
"regexp"
"strconv"
"strings"
"time"
"unicode/utf8"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/redis/go-redis/v9"
)
type User struct {
ID string `json:"id"`
Username string `json:"username"`
DisplayName string `json:"display_name"`
CreatedAt string `json:"created_at"`
}
type Chat struct {
ID string `json:"id"`
Title string `json:"title"`
MemberIDs []string `json:"member_ids"`
CreatedAt string `json:"created_at"`
}
type Message struct {
ID string `json:"id"`
ChatID string `json:"chat_id"`
SenderID string `json:"sender_id"`
Text string `json:"text"`
CreatedAt string `json:"created_at"`
}
type Session struct {
Token string `json:"token"`
TokenType string `json:"token_type"`
ExpiresAt string `json:"expires_at"`
User User `json:"user"`
}
type API struct {
db *pgxpool.Pool
sessions *redis.Client
ttl time.Duration
idempotencyTTL time.Duration
dbTimeout time.Duration
instanceID string
logger *log.Logger
}
func NewHandler(db *pgxpool.Pool, sessions *redis.Client, ttl, idempotencyTTL, dbTimeout time.Duration, instanceID string, logger *log.Logger) http.Handler {
return &API{db: db, sessions: sessions, ttl: ttl, idempotencyTTL: idempotencyTTL, dbTimeout: dbTimeout, instanceID: instanceID, logger: logger}
}
var uuidRE = regexp.MustCompile(`^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$`)
var usernameRE = regexp.MustCompile(`^[a-z0-9_]{3,32}$`)
var idempotencyKeyRE = regexp.MustCompile(`^[A-Za-z0-9._:-]{1,128}$`)
type apiError struct {
status int
code, message, challenge string
}
type response struct {
status int
body any
noBody bool
}
func bad(msg string) *apiError { return &apiError{status: 400, code: "invalid_request", message: msg} }
func missing() *apiError {
return &apiError{status: 404, code: "not_found", message: "Объект не найден"}
}
func unavailable() *apiError {
return &apiError{status: 503, code: "unavailable", message: "Сервис временно недоступен"}
}
func unauthorized(scheme string) *apiError {
challenge := `Bearer realm="messenger"`
if scheme == "Basic" {
challenge = `Basic realm="messenger", charset="UTF-8"`
}
return &apiError{status: 401, code: "unauthorized", message: "Неверные учётные данные", challenge: challenge}
}
func (a *API) ServeHTTP(w http.ResponseWriter, r *http.Request) {
id := newID()
w.Header().Set("X-Request-Id", id)
w.Header().Set("X-Instance-Id", a.instanceID)
w.Header().Set("Content-Type", "application/json; charset=utf-8")
start, status := time.Now(), 500
defer func() {
if x := recover(); x != nil {
status = 500
a.logger.Printf("request_id=%s panic=%T", id, x)
writeError(w, id, &apiError{status: 500, code: "internal_error", message: "Внутренняя ошибка"})
}
a.logger.Printf("request_id=%s method=%s path=%s status=%d duration_ms=%d", id, r.Method, r.URL.Path, status, time.Since(start).Milliseconds())
}()
res, apiErr := a.route(r)
if apiErr != nil {
status = apiErr.status
writeError(w, id, apiErr)
return
}
status = res.status
w.WriteHeader(status)
if !res.noBody {
_ = json.NewEncoder(w).Encode(res.body)
}
}
func writeError(w http.ResponseWriter, requestID string, e *apiError) {
if e.challenge != "" {
w.Header().Set("WWW-Authenticate", e.challenge)
}
w.WriteHeader(e.status)
_ = json.NewEncoder(w).Encode(map[string]any{"error": map[string]string{"code": e.code, "message": e.message, "request_id": requestID}})
}
func decode(r *http.Request, dst any) *apiError {
if strings.ToLower(strings.TrimSpace(strings.Split(r.Header.Get("Content-Type"), ";")[0])) != "application/json" {
return &apiError{status: 415, code: "unsupported_media_type", message: "Требуется application/json"}
}
if r.ContentLength > 1<<20 {
return &apiError{status: 413, code: "payload_too_large", message: "Слишком большой запрос"}
}
data, err := io.ReadAll(io.LimitReader(r.Body, (1<<20)+1))
if err != nil {
return bad("Не удалось прочитать JSON")
}
if len(data) > 1<<20 {
return &apiError{status: 413, code: "payload_too_large", message: "Слишком большой запрос"}
}
if !utf8.Valid(data) {
return bad("JSON должен быть в UTF-8")
}
d := json.NewDecoder(bytes.NewReader(data))
d.DisallowUnknownFields()
if err = d.Decode(dst); err != nil {
return bad("Неверный JSON или поля")
}
var rest any
if !errors.Is(d.Decode(&rest), io.EOF) {
return bad("Ожидался один JSON-объект")
}
return nil
}
func validText(s string, max int) bool {
return utf8.ValidString(s) && strings.TrimSpace(s) != "" && utf8.RuneCountInString(s) <= max
}
func validPassword(s string) bool {
n := utf8.RuneCountInString(s)
return utf8.ValidString(s) && n >= 12 && n <= 128
}
func formatTime(t time.Time) string { return t.UTC().Format(time.RFC3339Nano) }
func newID() string {
var b [16]byte
if _, err := rand.Read(b[:]); err != nil {
panic(err)
}
b[6] = (b[6] & 0x0f) | 0x40
b[8] = (b[8] & 0x3f) | 0x80
return fmt.Sprintf("%x-%x-%x-%x-%x", b[:4], b[4:6], b[6:8], b[8:10], b[10:])
}
func newToken() string {
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {
panic(err)
}
return base64.RawURLEncoding.EncodeToString(b)
}
func tokenKey(token string) string {
sum := sha256.Sum256([]byte(token))
return "session:" + hex.EncodeToString(sum[:])
}
func cursorFor(resource string, created time.Time, id string) string {
b, _ := json.Marshal([]string{resource, formatTime(created), id})
return base64.RawURLEncoding.EncodeToString(b)
}
func parsePage(r *http.Request, resource string) (int, *time.Time, string, *apiError) {
q, err := url.ParseQuery(r.URL.RawQuery)
if err != nil {
return 0, nil, "", bad("Неверные параметры страницы")
}
for k, v := range q {
if (k != "limit" && k != "cursor") || len(v) != 1 {
return 0, nil, "", bad("Неверные параметры страницы")
}
}
limit := 50
if raw := q.Get("limit"); raw != "" {
n, e := strconv.Atoi(raw)
if e != nil || n < 1 || n > 100 {
return 0, nil, "", bad("limit должен быть от 1 до 100")
}
limit = n
} else if _, ok := q["limit"]; ok {
return 0, nil, "", bad("Пустой limit")
}
if _, ok := q["cursor"]; !ok {
return limit, nil, "", nil
}
raw := q.Get("cursor")
if raw == "" || len(raw) > 1024 {
return 0, nil, "", bad("Неверный cursor")
}
b, e := base64.RawURLEncoding.DecodeString(raw)
if e != nil {
return 0, nil, "", bad("Неверный cursor")
}
var p []string
if json.Unmarshal(b, &p) != nil || len(p) != 3 || p[0] != resource || !uuidRE.MatchString(p[2]) {
return 0, nil, "", bad("Неверный cursor")
}
t, e := time.Parse(time.RFC3339Nano, p[1])
if e != nil {
return 0, nil, "", bad("Неверный cursor")
}
return limit, &t, p[2], nil
}
func (a *API) dbContext(parent context.Context) (context.Context, context.CancelFunc) {
return context.WithTimeout(parent, a.dbTimeout)
}
func (a *API) authenticate(r *http.Request) (string, string, *apiError) {
h := r.Header.Get("Authorization")
if !strings.HasPrefix(h, "Bearer ") || strings.TrimSpace(strings.TrimPrefix(h, "Bearer ")) == "" {
return "", "", unauthorized("Bearer")
}
token := strings.TrimSpace(strings.TrimPrefix(h, "Bearer "))
ctx, cancel := context.WithTimeout(r.Context(), a.dbTimeout)
defer cancel()
uid, err := a.sessions.Get(ctx, tokenKey(token)).Result()
if errors.Is(err, redis.Nil) {
return "", "", unauthorized("Bearer")
}
if err != nil {
a.logger.Printf("session store unavailable: %v", err)
return "", "", unavailable()
}
if !uuidRE.MatchString(uid) {
return "", "", unauthorized("Bearer")
}
return uid, token, nil
}
func scanUser(row pgx.Row) (User, string, error) {
var u User
var created time.Time
var password string
err := row.Scan(&u.ID, &u.Username, &u.DisplayName, &created, &password)
u.CreatedAt = formatTime(created)
return u, password, err
}
func (a *API) route(r *http.Request) (response, *apiError) {
p := r.URL.Path
if r.Method == "GET" && p == "/health/live" {
return response{status: 200, body: map[string]string{"status": "ok"}}, nil
}
if r.Method == "GET" && p == "/health/ready" {
ctx, cancel := context.WithTimeout(r.Context(), time.Second)
defer cancel()
if a.db.Ping(ctx) != nil || a.sessions.Ping(ctx).Err() != nil {
return response{}, unavailable()
}
return response{status: 200, body: map[string]string{"status": "ok"}}, nil
}
if r.Method == "POST" && p == "/api/v1/users" {
return a.createUser(r)
}
if r.Method == "POST" && p == "/api/v1/auth/sessions" {
return a.createSession(r)
}
if !strings.HasPrefix(p, "/api/v1/") {
return response{}, missing()
}
uid, token, authErr := a.authenticate(r)
if authErr != nil {
return response{}, authErr
}
if r.Method == "DELETE" && p == "/api/v1/auth/sessions/current" {
ctx, cancel := context.WithTimeout(r.Context(), a.dbTimeout)
defer cancel()
if err := a.sessions.Del(ctx, tokenKey(token)).Err(); err != nil {
return response{}, unavailable()
}
return response{status: 204, noBody: true}, nil
}
if r.Method == "GET" && p == "/api/v1/users/me" {
return a.currentUser(r, uid)
}
if p == "/api/v1/chats" {
if r.Method == "POST" {
return a.createChat(r, uid)
}
if r.Method == "GET" {
return a.listChats(r, uid)
}
}
if !strings.HasPrefix(p, "/api/v1/chats/") {
return response{}, missing()
}
parts := strings.Split(strings.TrimPrefix(p, "/api/v1/chats/"), "/")
if len(parts) < 1 || !uuidRE.MatchString(parts[0]) {
return response{}, bad("Неверный chat_id")
}
chatID := parts[0]
if len(parts) == 1 && r.Method == "GET" {
return a.getChat(r, uid, chatID)
}
if len(parts) == 2 && parts[1] == "messages" {
if r.Method == "POST" {
return a.createMessage(r, uid, chatID)
}
if r.Method == "GET" {
return a.listMessages(r, uid, chatID)
}
}
return response{}, missing()
}
func (a *API) createUser(r *http.Request) (response, *apiError) {
var in struct {
Username string `json:"username"`
DisplayName string `json:"display_name"`
Password string `json:"password"`
}
if e := decode(r, &in); e != nil {
return response{}, e
}
if !usernameRE.MatchString(in.Username) || !validText(in.DisplayName, 100) || !validPassword(in.Password) {
return response{}, bad("Неверные данные пользователя")
}
passwordHash, err := hashPassword(in.Password)
if err != nil {
return response{}, &apiError{status: 500, code: "internal_error", message: "Внутренняя ошибка"}
}
ctx, cancel := a.dbContext(r.Context())
defer cancel()
var u User
var created time.Time
err = a.db.QueryRow(ctx, `INSERT INTO users(id,username,display_name,password_hash) VALUES($1,$2,$3,$4) RETURNING id::text,username,display_name,created_at`, newID(), in.Username, in.DisplayName, passwordHash).Scan(&u.ID, &u.Username, &u.DisplayName, &created)
if err != nil {
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) && pgErr.Code == "23505" {
return response{}, &apiError{status: 409, code: "conflict", message: "Имя пользователя занято"}
}
return response{}, unavailable()
}
u.CreatedAt = formatTime(created)
return response{status: 201, body: u}, nil
}
func (a *API) createSession(r *http.Request) (response, *apiError) {
if r.ContentLength > 0 {
return response{}, bad("Тело запроса не допускается")
}
h := r.Header.Get("Authorization")
if !strings.HasPrefix(h, "Basic ") {
return response{}, unauthorized("Basic")
}
raw, err := base64.StdEncoding.DecodeString(strings.TrimSpace(strings.TrimPrefix(h, "Basic ")))
if err != nil {
return response{}, unauthorized("Basic")
}
parts := strings.SplitN(string(raw), ":", 2)
if len(parts) != 2 || !usernameRE.MatchString(parts[0]) {
return response{}, unauthorized("Basic")
}
ctx, cancel := a.dbContext(r.Context())
defer cancel()
u, passwordHash, err := scanUser(a.db.QueryRow(ctx, `SELECT id::text,username,display_name,created_at,password_hash FROM users WHERE username=$1`, parts[0]))
if errors.Is(err, pgx.ErrNoRows) {
return response{}, unauthorized("Basic")
}
if err != nil {
return response{}, unavailable()
}
if !verifyPassword(passwordHash, parts[1]) {
return response{}, unauthorized("Basic")
}
token := newToken()
expires := time.Now().UTC().Add(a.ttl)
if err = a.sessions.Set(ctx, tokenKey(token), u.ID, a.ttl).Err(); err != nil {
return response{}, unavailable()
}
return response{status: 201, body: Session{Token: token, TokenType: "Bearer", ExpiresAt: formatTime(expires), User: u}}, nil
}
func (a *API) currentUser(r *http.Request, uid string) (response, *apiError) {
ctx, cancel := a.dbContext(r.Context())
defer cancel()
u, _, err := scanUser(a.db.QueryRow(ctx, `SELECT id::text,username,display_name,created_at,password_hash FROM users WHERE id=$1`, uid))
if errors.Is(err, pgx.ErrNoRows) {
return response{}, unauthorized("Bearer")
}
if err != nil {
return response{}, unavailable()
}
return response{status: 200, body: u}, nil
}
func (a *API) createChat(r *http.Request, uid string) (response, *apiError) {
var in struct {
Title string `json:"title"`
MemberIDs []string `json:"member_ids"`
}
if e := decode(r, &in); e != nil {
return response{}, e
}
if !validText(in.Title, 100) || len(in.MemberIDs) < 1 || len(in.MemberIDs) > 100 {
return response{}, bad("Неверные данные чата")
}
seen := map[string]bool{}
for _, id := range in.MemberIDs {
if !uuidRE.MatchString(id) || seen[id] {
return response{}, bad("Неверные участники чата")
}
seen[id] = true
}
if !seen[uid] {
return response{}, bad("Создатель должен быть участником")
}
ctx, cancel := a.dbContext(r.Context())
defer cancel()
tx, err := a.db.Begin(ctx)
if err != nil {
return response{}, unavailable()
}
defer tx.Rollback(ctx)
c := Chat{ID: newID(), Title: in.Title, MemberIDs: append([]string{}, in.MemberIDs...)}
var created time.Time
if err = tx.QueryRow(ctx, `INSERT INTO chats(id,title) VALUES($1,$2) RETURNING created_at`, c.ID, c.Title).Scan(&created); err != nil {
return response{}, unavailable()
}
for _, id := range c.MemberIDs {
if _, err = tx.Exec(ctx, `INSERT INTO chat_members(chat_id,user_id) VALUES($1,$2)`, c.ID, id); err != nil {
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) && pgErr.Code == "23503" {
return response{}, bad("Неверные участники чата")
}
return response{}, unavailable()
}
}
if err = tx.Commit(ctx); err != nil {
return response{}, unavailable()
}
c.CreatedAt = formatTime(created)
return response{status: 201, body: c}, nil
}
func scanChat(row pgx.Row) (Chat, time.Time, error) {
var c Chat
var t time.Time
err := row.Scan(&c.ID, &c.Title, &c.MemberIDs, &t)
c.CreatedAt = formatTime(t)
return c, t, err
}
func chatSelect() string {
return `SELECT c.id::text,c.title,ARRAY(SELECT cm2.user_id::text FROM chat_members cm2 WHERE cm2.chat_id=c.id ORDER BY cm2.user_id),c.created_at FROM chats c`
}
func (a *API) getChat(r *http.Request, uid, chatID string) (response, *apiError) {
ctx, cancel := a.dbContext(r.Context())
defer cancel()
c, _, err := scanChat(a.db.QueryRow(ctx, chatSelect()+` WHERE c.id=$1 AND EXISTS(SELECT 1 FROM chat_members cm WHERE cm.chat_id=c.id AND cm.user_id=$2)`, chatID, uid))
if errors.Is(err, pgx.ErrNoRows) {
return response{}, missing()
}
if err != nil {
return response{}, unavailable()
}
return response{status: 200, body: c}, nil
}
func (a *API) listChats(r *http.Request, uid string) (response, *apiError) {
limit, afterTime, afterID, e := parsePage(r, "chats:"+uid)
if e != nil {
return response{}, e
}
ctx, cancel := a.dbContext(r.Context())
defer cancel()
if afterTime != nil {
var ok bool
err := a.db.QueryRow(ctx, `SELECT EXISTS(SELECT 1 FROM chats c JOIN chat_members cm ON cm.chat_id=c.id WHERE cm.user_id=$1 AND c.id=$2 AND c.created_at=$3)`, uid, afterID, *afterTime).Scan(&ok)
if err != nil {
return response{}, unavailable()
}
if !ok {
return response{}, bad("Неверный cursor")
}
}
var rows pgx.Rows
var err error
base := chatSelect() + ` JOIN chat_members mine ON mine.chat_id=c.id WHERE mine.user_id=$1`
if afterTime == nil {
rows, err = a.db.Query(ctx, base+` ORDER BY c.created_at,c.id LIMIT $2`, uid, limit+1)
} else {
rows, err = a.db.Query(ctx, base+` AND (c.created_at,c.id)>($2,$3) ORDER BY c.created_at,c.id LIMIT $4`, uid, *afterTime, afterID, limit+1)
}
if err != nil {
return response{}, unavailable()
}
defer rows.Close()
items := make([]Chat, 0, limit+1)
times := make([]time.Time, 0, limit+1)
for rows.Next() {
c, t, scanErr := scanChat(rows)
if scanErr != nil {
return response{}, unavailable()
}
items = append(items, c)
times = append(times, t)
}
if rows.Err() != nil {
return response{}, unavailable()
}
var next any
if len(items) > limit {
last := items[limit-1]
next = cursorFor("chats:"+uid, times[limit-1], last.ID)
items = items[:limit]
}
return response{status: 200, body: map[string]any{"items": items, "next_cursor": next}}, nil
}
func (a *API) createMessage(r *http.Request, uid, chatID string) (response, *apiError) {
key := r.Header.Get("Idempotency-Key")
if !idempotencyKeyRE.MatchString(key) {
return response{}, bad("Требуется корректный Idempotency-Key")
}
var in struct {
Text string `json:"text"`
}
if e := decode(r, &in); e != nil {
return response{}, e
}
if !validText(in.Text, 4000) {
return response{}, bad("Текст сообщения должен содержать от 1 до 4000 символов")
}
ctx, cancel := a.dbContext(r.Context())
defer cancel()
tx, err := a.db.Begin(ctx)
if err != nil {
return response{}, unavailable()
}
defer tx.Rollback(ctx)
var member bool
if err = tx.QueryRow(ctx, `SELECT EXISTS(SELECT 1 FROM chat_members WHERE chat_id=$1 AND user_id=$2)`, chatID, uid).Scan(&member); err != nil {
return response{}, unavailable()
}
if !member {
return response{}, missing()
}
requestHash := sha256.Sum256([]byte(in.Text))
if _, err = tx.Exec(ctx, `DELETE FROM message_idempotency WHERE user_id=$1 AND method=$2 AND path=$3 AND idempotency_key=$4 AND created_at < clock_timestamp() - ($5 * interval '1 second')`, uid, r.Method, r.URL.Path, key, int64(a.idempotencyTTL/time.Second)); err != nil {
return response{}, unavailable()
}
result, err := tx.Exec(ctx, `INSERT INTO message_idempotency(user_id,method,path,idempotency_key,request_hash) VALUES($1,$2,$3,$4,$5) ON CONFLICT DO NOTHING`, uid, r.Method, r.URL.Path, key, requestHash[:])
if err != nil {
return response{}, unavailable()
}
inserted := result.RowsAffected() == 1
if !inserted {
var storedHash []byte
var messageID string
if err = tx.QueryRow(ctx, `SELECT request_hash,message_id::text FROM message_idempotency WHERE user_id=$1 AND method=$2 AND path=$3 AND idempotency_key=$4 FOR UPDATE`, uid, r.Method, r.URL.Path, key).Scan(&storedHash, &messageID); err != nil {
return response{}, unavailable()
}
if !bytes.Equal(storedHash, requestHash[:]) {
return response{}, &apiError{status: 409, code: "conflict", message: "Idempotency-Key уже использован с другим запросом"}
}
var m Message
var created time.Time
if err = tx.QueryRow(ctx, `SELECT id::text,chat_id::text,sender_id::text,text,created_at FROM messages WHERE id=$1`, messageID).Scan(&m.ID, &m.ChatID, &m.SenderID, &m.Text, &created); err != nil {
return response{}, unavailable()
}
m.CreatedAt = formatTime(created)
if err = tx.Commit(ctx); err != nil {
return response{}, unavailable()
}
return response{status: 201, body: m}, nil
}
m := Message{ID: newID(), ChatID: chatID, SenderID: uid, Text: in.Text}
var created time.Time
if err = tx.QueryRow(ctx, `INSERT INTO messages(id,chat_id,sender_id,text) VALUES($1,$2,$3,$4) RETURNING created_at`, m.ID, m.ChatID, m.SenderID, m.Text).Scan(&created); err != nil {
return response{}, unavailable()
}
if _, err = tx.Exec(ctx, `UPDATE message_idempotency SET message_id=$1 WHERE user_id=$2 AND method=$3 AND path=$4 AND idempotency_key=$5`, m.ID, uid, r.Method, r.URL.Path, key); err != nil {
return response{}, unavailable()
}
if err = tx.Commit(ctx); err != nil {
return response{}, unavailable()
}
m.CreatedAt = formatTime(created)
return response{status: 201, body: m}, nil
}
func (a *API) listMessages(r *http.Request, uid, chatID string) (response, *apiError) {
limit, afterTime, afterID, e := parsePage(r, "messages:"+chatID)
if e != nil {
return response{}, e
}
ctx, cancel := a.dbContext(r.Context())
defer cancel()
var member bool
if err := a.db.QueryRow(ctx, `SELECT EXISTS(SELECT 1 FROM chat_members WHERE chat_id=$1 AND user_id=$2)`, chatID, uid).Scan(&member); err != nil {
return response{}, unavailable()
}
if !member {
return response{}, missing()
}
if afterTime != nil {
var ok bool
err := a.db.QueryRow(ctx, `SELECT EXISTS(SELECT 1 FROM messages WHERE chat_id=$1 AND id=$2 AND created_at=$3)`, chatID, afterID, *afterTime).Scan(&ok)
if err != nil {
return response{}, unavailable()
}
if !ok {
return response{}, bad("Неверный cursor")
}
}
var rows pgx.Rows
var err error
base := `SELECT id::text,chat_id::text,sender_id::text,text,created_at FROM messages WHERE chat_id=$1`
if afterTime == nil {
rows, err = a.db.Query(ctx, base+` ORDER BY created_at,id LIMIT $2`, chatID, limit+1)
} else {
rows, err = a.db.Query(ctx, base+` AND (created_at,id)>($2,$3) ORDER BY created_at,id LIMIT $4`, chatID, *afterTime, afterID, limit+1)
}
if err != nil {
return response{}, unavailable()
}
defer rows.Close()
items := make([]Message, 0, limit+1)
times := make([]time.Time, 0, limit+1)
for rows.Next() {
var m Message
var t time.Time
if rows.Scan(&m.ID, &m.ChatID, &m.SenderID, &m.Text, &t) != nil {
return response{}, unavailable()
}
m.CreatedAt = formatTime(t)
items = append(items, m)
times = append(times, t)
}
if rows.Err() != nil {
return response{}, unavailable()
}
var next any
if len(items) > limit {
last := items[limit-1]
next = cursorFor("messages:"+chatID, times[limit-1], last.ID)
items = items[:limit]
}
return response{status: 200, body: map[string]any{"items": items, "next_cursor": next}}, nil
}
+42
View File
@@ -0,0 +1,42 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
mkdir -p evidence
tmp_file=$(mktemp)
trap 'rm -f "$tmp_file"' EXIT
docker compose --env-file .env exec -T postgres psql -U messenger -d messenger -v ON_ERROR_STOP=1 >"$tmp_file" <<'SQL'
INSERT INTO users(id, username, display_name, password_hash)
VALUES ('00000000-0000-4000-8000-000000000001', 'explain_user', 'EXPLAIN fixture', 'fixture-not-for-login')
ON CONFLICT (id) DO NOTHING;
INSERT INTO chats(id, title)
VALUES ('00000000-0000-4000-8000-000000000002', 'EXPLAIN history fixture')
ON CONFLICT (id) DO NOTHING;
INSERT INTO chat_members(chat_id, user_id)
VALUES ('00000000-0000-4000-8000-000000000002', '00000000-0000-4000-8000-000000000001')
ON CONFLICT DO NOTHING;
DELETE FROM messages WHERE chat_id = '00000000-0000-4000-8000-000000000002';
INSERT INTO messages(id, chat_id, sender_id, text, created_at)
SELECT md5('history-message-' || n::text)::uuid,
'00000000-0000-4000-8000-000000000002',
'00000000-0000-4000-8000-000000000001',
'message ' || n::text,
TIMESTAMPTZ '2026-01-01 00:00:00+00' + n * INTERVAL '1 microsecond'
FROM generate_series(1, 10000) AS n;
ANALYZE messages;
SELECT count(*) AS fixture_messages
FROM messages
WHERE chat_id = '00000000-0000-4000-8000-000000000002';
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, chat_id, sender_id, text, created_at
FROM messages
WHERE chat_id = '00000000-0000-4000-8000-000000000002'
ORDER BY created_at, id
LIMIT 50;
SQL
cp "$tmp_file" evidence/explain-history.txt
cat "$tmp_file"
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
set -eu
mkdir -p secrets
if [ ! -f secrets/postgres_password ]; then
umask 077
openssl rand -hex 24 > secrets/postgres_password
fi
if [ ! -f secrets/redis_password ]; then
umask 077
openssl rand -hex 24 > secrets/redis_password
fi
postgres_password=$(cat secrets/postgres_password)
redis_password=$(cat secrets/redis_password)
umask 077
printf 'postgres://messenger:%s@postgres:5432/messenger?sslmode=disable\n' "$postgres_password" > secrets/database_url
printf 'redis://:%s@redis:6379/0\n' "$redis_password" > secrets/cache_url
if [ ! -f .env ]; then cp .env.example .env; fi
printf '%s\n' 'Local .env and secret files are ready.'
+26
View File
@@ -0,0 +1,26 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
wait_healthy() {
service=$1
container_id=$(docker compose --env-file .env ps -q "$service")
attempts=0
until [ "$(docker inspect --format '{{.State.Health.Status}}' "$container_id" 2>/dev/null || true)" = healthy ]; do
attempts=$((attempts + 1))
if [ "$attempts" -ge 60 ]; then
printf '%s\n' "$service did not become healthy" >&2
exit 1
fi
sleep 1
done
}
docker compose --env-file .env restart postgres redis >/dev/null
wait_healthy postgres
wait_healthy redis
docker compose --env-file .env restart api1 api2 >/dev/null
wait_healthy api1
wait_healthy api2
docker compose --env-file .env restart nginx >/dev/null
wait_healthy nginx
+25
View File
@@ -0,0 +1,25 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
wait_healthy() {
service=$1
container_id=$(docker compose --env-file .env ps -q "$service")
attempts=0
until [ "$(docker inspect --format '{{.State.Health.Status}}' "$container_id" 2>/dev/null || true)" = healthy ]; do
attempts=$((attempts + 1))
if [ "$attempts" -ge 60 ]; then
printf '%s\n' "$service did not become healthy" >&2
exit 1
fi
sleep 1
done
}
docker compose --env-file .env build migrate >/dev/null
docker compose --env-file .env run --rm --no-deps migrate >/dev/null
for service in api1 api2; do
docker compose --env-file .env up -d --no-deps --force-recreate "$service" >/dev/null
wait_healthy "$service"
docker compose --env-file .env exec -T nginx nginx -s reload >/dev/null
done
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
docker compose --env-file .env start api1 api2 >/dev/null
for service in api1 api2; do
container_id=$(docker compose --env-file .env ps -q "$service")
attempts=0
until [ "$(docker inspect --format '{{.State.Health.Status}}' "$container_id" 2>/dev/null || true)" = healthy ]; do
attempts=$((attempts + 1))
if [ "$attempts" -ge 60 ]; then
printf '%s\n' "$service did not become healthy" >&2
exit 1
fi
sleep 1
done
done
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
case "$TARGET_INSTANCE_ID" in
api-1) service=api1 ;;
api-2) service=api2 ;;
*) printf '%s\n' "Unknown API instance alias" >&2; exit 2 ;;
esac
docker compose --env-file .env stop --timeout 3 "$service" >/dev/null
+105
View File
@@ -0,0 +1,105 @@
"""Stop api-1 and api-2 in turn and record bounded failover metrics."""
import json
import os
import subprocess
import sys
import time
import uuid
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure, arguments, check, run # noqa: E402
def command(*args, env=None):
subprocess.run(args, cwd=ROOT, env=env, check=True,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
def wait_instances(client, expected, timeout=60):
deadline = time.monotonic() + timeout
found = set()
while time.monotonic() < deadline:
try:
found.add(client.call("GET", "/health/ready", schema="Health")
["headers"]["X-Instance-Id"])
if expected.issubset(found):
return
except Failure:
pass
time.sleep(0.1)
raise Failure("expected API replicas did not become visible")
def main():
args = arguments("Sequential API failover metrics")
client = Client(args.base_url, args.ca_file, timeout=2)
command(str(ROOT / "scripts" / "start-api.sh"))
wait_instances(client, {"api-1", "api-2"})
alice, _, _, chat = client.bootstrap()
path = "/api/v1/chats/" + chat["id"] + "/messages"
results = []
for target in ("api-1", "api-2"):
command(str(ROOT / "scripts" / "start-api.sh"))
wait_instances(client, {"api-1", "api-2"})
env = dict(os.environ)
env["TARGET_INSTANCE_ID"] = target
started = time.monotonic()
process = subprocess.Popen([str(ROOT / "scripts" / "stop-one.sh")],
cwd=ROOT, env=env,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL)
successes = failures = 0
first_survivor = None
deadline = started + 8
key = uuid.uuid4().hex
text = "failover-" + target + "-" + key
while time.monotonic() < deadline:
try:
response = client.call("POST", path, 201, actor=alice,
body={"text": text},
headers={"Idempotency-Key": key},
schema="Message")
successes += 1
if (process.poll() is not None
and response["headers"]["X-Instance-Id"] != target
and first_survivor is None):
first_survivor = time.monotonic() - started
key = uuid.uuid4().hex
text = "failover-" + target + "-" + key
except Failure:
failures += 1
# Repeat the same body and key after an uncertain result.
code = process.wait(timeout=10)
elapsed = time.monotonic() - started
check(code == 0, "stop-one.sh failed")
check(successes > 0 and first_survivor is not None,
"surviving replica did not accept writes")
check(first_survivor <= 10, "failover exceeded ten seconds")
results.append({
"stopped_instance": target,
"duration_seconds": round(elapsed, 3),
"successful_requests": successes,
"failed_requests": failures,
"successful_rps": round(successes / elapsed, 3),
"first_survivor_success_seconds": round(first_survivor, 3),
})
command(str(ROOT / "scripts" / "start-api.sh"))
wait_instances(client, {"api-1", "api-2"})
output = {"workload": "single-client closed-loop POST messages", "runs": results}
(ROOT / "evidence").mkdir(exist_ok=True)
(ROOT / "evidence" / "failover-grade4.json").write_text(
json.dumps(output, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print("PASS:", json.dumps(output, ensure_ascii=False))
if __name__ == "__main__":
try:
run(main)
finally:
subprocess.run([str(ROOT / "scripts" / "start-api.sh")], cwd=ROOT,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False)
@@ -0,0 +1,91 @@
"""Run concurrent writes while both API replicas are replaced one at a time."""
import concurrent.futures
import json
import math
import subprocess
import sys
import threading
import time
import uuid
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure, arguments, check, run # noqa: E402
def main():
args = arguments("Rolling update under write load")
base = Client(args.base_url, args.ca_file)
alice, _, _, chat = base.bootstrap()
path = "/api/v1/chats/" + chat["id"] + "/messages"
workers = 4
start_barrier = threading.Barrier(workers + 1)
stop = threading.Event()
def worker(worker_id):
client = Client(args.base_url, args.ca_file, timeout=3)
successes = failures = 0
latencies = []
instances = set()
index = 0
start_barrier.wait()
while not stop.is_set():
key = uuid.uuid4().hex
started = time.monotonic()
try:
result = client.call("POST", path, 201, actor=alice,
body={"text": "rolling-%d-%d" % (worker_id, index)},
headers={"Idempotency-Key": key}, schema="Message")
successes += 1
latencies.append(time.monotonic() - started)
instances.add(result["headers"]["X-Instance-Id"])
except Failure:
failures += 1
index += 1
return successes, failures, latencies, instances
started = time.monotonic()
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as pool:
futures = [pool.submit(worker, i) for i in range(workers)]
start_barrier.wait()
update = subprocess.run([str(ROOT / "scripts" / "rolling-update.sh")],
cwd=ROOT, stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE, text=True, timeout=120)
time.sleep(3)
stop.set()
results = [future.result() for future in futures]
elapsed = time.monotonic() - started
check(update.returncode == 0, "rolling update failed: " + update.stderr[-500:])
successes = sum(item[0] for item in results)
failures = sum(item[1] for item in results)
latencies = sorted(value for item in results for value in item[2])
instances = sorted(set().union(*(item[3] for item in results)))
check(successes > 0, "no successful writes during rolling update")
check(set(instances) == {"api-1", "api-2"},
"both replicas were not observed during rolling update")
def percentile(fraction):
if not latencies:
return None
pos = max(0, math.ceil(fraction * len(latencies)) - 1)
return round(latencies[pos] * 1000, 3)
output = {
"workload": "four-client closed-loop POST messages",
"elapsed_seconds": round(elapsed, 3),
"successful_requests": successes,
"failed_requests": failures,
"successful_rps": round(successes / elapsed, 3),
"latency_ms_successes": {"p50": percentile(.5), "p95": percentile(.95),
"p99": percentile(.99)},
"instances": instances,
}
(ROOT / "evidence").mkdir(exist_ok=True)
(ROOT / "evidence" / "rolling-update.json").write_text(
json.dumps(output, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print("PASS:", json.dumps(output, ensure_ascii=False))
if __name__ == "__main__":
run(main)
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env python3
"""Stop Redis and PostgreSQL separately and check bounded API failures."""
import subprocess
import sys
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure, check # noqa: E402
def compose(*args):
subprocess.run(["docker", "compose", "--env-file", ".env", *args], cwd=ROOT,
check=True, stdout=subprocess.DEVNULL)
def wait_ready(client, timeout=60):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
client.call("GET", "/health/ready", schema="Health")
return
except Failure:
time.sleep(0.5)
raise Failure("stack did not recover")
def main():
client = Client(timeout=5)
actor = client.user("dependency")
try:
compose("stop", "redis")
client.call("GET", "/health/ready", 503)
client.call("GET", "/api/v1/users/me", 503, actor=actor)
client.call("GET", "/api/v1/users/me", 401)
finally:
compose("start", "redis")
wait_ready(client)
client.call("GET", "/api/v1/users/me", actor=actor, schema="User")
try:
compose("stop", "postgres")
client.call("GET", "/health/ready", 503)
client.call("GET", "/api/v1/users/me", 503, actor=actor)
finally:
compose("start", "postgres")
wait_ready(client)
client.call("GET", "/api/v1/users/me", actor=actor, schema="User")
print("PASS: Redis/PostgreSQL failures returned 503; anonymous access stayed 401; state recovered")
if __name__ == "__main__":
try:
main()
except (Failure, OSError, subprocess.CalledProcessError) as exc:
print("FAIL:", exc)
raise SystemExit(1)
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env python3
"""Recreate the API with a two-second session TTL and prove expiry."""
import os
import subprocess
import sys
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure, check # noqa: E402
def compose(env, *args):
subprocess.run(["docker", "compose", "--env-file", ".env", *args], cwd=ROOT,
env=env, check=True, stdout=subprocess.DEVNULL)
def wait_ready(timeout=60):
client = Client(timeout=2)
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
client.call("GET", "/health/ready", schema="Health")
return
except Failure:
time.sleep(0.5)
raise Failure("API did not become ready")
def main():
normal_env = dict(os.environ)
normal_env.pop("SESSION_TTL_SECONDS", None)
short_env = dict(normal_env)
short_env["SESSION_TTL_SECONDS"] = "2"
try:
compose(short_env, "up", "-d", "--force-recreate", "--no-deps", "--wait", "api1", "api2")
compose(short_env, "exec", "-T", "nginx", "nginx", "-s", "reload")
wait_ready()
client = Client()
actor = client.user("expiry")
time.sleep(3)
client.call("GET", "/api/v1/users/me", 401, actor=actor)
print("PASS: two-second Redis session expired and was rejected")
finally:
compose(normal_env, "up", "-d", "--force-recreate", "--no-deps", "--wait", "api1", "api2")
compose(normal_env, "exec", "-T", "nginx", "nginx", "-s", "reload")
wait_ready()
if __name__ == "__main__":
try:
main()
except (Failure, OSError, subprocess.CalledProcessError) as exc:
print("FAIL:", exc)
raise SystemExit(1)
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""Black-box checks for the grade 4 registration, transaction and auth cases."""
import base64
import concurrent.futures
import sys
import threading
import uuid
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tests"))
from client import Client, arguments, check, run # noqa: E402
def basic(username, password):
raw = (username + ":" + password).encode("utf-8")
return {"Authorization": "Basic " + base64.b64encode(raw).decode("ascii")}
def main():
args = arguments("Grade 4 API checks")
base = Client(args.base_url, args.ca_file)
suffix = uuid.uuid4().hex[:12]
username = "race_" + suffix
password = "Race-password-" + uuid.uuid4().hex
registration = {"username": username, "display_name": "Гонка регистрации", "password": password}
workers = 4
barrier = threading.Barrier(workers)
def register(_):
client = Client(args.base_url, args.ca_file)
barrier.wait()
return client.call("POST", "/api/v1/users", [201, 409], body=registration)["status"]
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as pool:
statuses = list(pool.map(register, range(workers)))
check(statuses.count(201) == 1 and statuses.count(409) == workers - 1,
"concurrent registration must create exactly one user")
base.call("POST", "/api/v1/auth/sessions", 401,
headers=basic(username, "definitely-wrong-password"))
session = base.call("POST", "/api/v1/auth/sessions", 201,
headers=basic(username, password), schema="Session")["json"]
actor = {"id": session["user"]["id"], "token": session["token"]}
title = "rollback-" + suffix
base.call("POST", "/api/v1/chats", 400, actor=actor,
body={"title": title, "member_ids": [actor["id"], str(uuid.uuid4())]})
page = base.call("GET", "/api/v1/chats?limit=100", actor=actor, schema="ChatPage")["json"]
check(all(chat["title"] != title for chat in page["items"]),
"failed composite operation left a visible chat")
base.call("DELETE", "/api/v1/auth/sessions/current", 204, actor=actor)
base.call("GET", "/api/v1/users/me", 401, actor=actor)
print("PASS: registration race, transaction rollback, bad password and logout")
if __name__ == "__main__":
run(main)
+58
View File
@@ -0,0 +1,58 @@
"""Prove that one idempotency key creates one message across both replicas."""
import concurrent.futures
import json
import sys
import threading
import uuid
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, arguments, check, run # noqa: E402
def main():
args = arguments("Concurrent cross-replica idempotency check")
base = Client(args.base_url, args.ca_file)
alice, _, _, chat = base.bootstrap()
path = "/api/v1/chats/" + chat["id"] + "/messages"
key = "parallel-" + uuid.uuid4().hex
body = {"text": "one logical concurrent message"}
workers = 12
barrier = threading.Barrier(workers)
def send(_):
client = Client(args.base_url, args.ca_file)
barrier.wait()
result = client.call("POST", path, 201, actor=alice, body=body,
headers={"Idempotency-Key": key}, schema="Message")
return result["json"], result["headers"]["X-Instance-Id"]
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as pool:
responses = list(pool.map(send, range(workers)))
messages = [item[0] for item in responses]
instances = sorted(set(item[1] for item in responses))
ids = sorted(set(item["id"] for item in messages))
check(len(ids) == 1, "one key created more than one message")
check(all(item == messages[0] for item in messages),
"idempotent responses differ")
check(len(instances) >= 2, "requests did not cross two API replicas")
base.call("POST", path, 409, actor=alice, body={"text": "different body"},
headers={"Idempotency-Key": key})
page = base.call("GET", path + "?limit=100", actor=alice,
schema="MessagePage")["json"]
check(sum(item["id"] == ids[0] for item in page["items"]) == 1,
"history contains a duplicate message")
result = {"parallel_requests": workers, "instances": instances,
"unique_message_ids": len(ids), "conflicting_reuse_status": 409}
(ROOT / "evidence").mkdir(exist_ok=True)
(ROOT / "evidence" / "idempotency.json").write_text(
json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print("PASS:", json.dumps(result, ensure_ascii=False))
if __name__ == "__main__":
run(main)
+71
View File
@@ -0,0 +1,71 @@
"""Check LB failover when api1 still runs but cannot reach its stores."""
import json
import subprocess
import sys
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure, check, run # noqa: E402
def docker(*args):
return subprocess.run(["docker", *args], cwd=ROOT, check=True,
capture_output=True, text=True).stdout.strip()
def compose(*args):
return docker("compose", "--env-file", ".env", *args)
def details(container):
return json.loads(docker("inspect", container))[0]
def wait_health(container, expected, timeout=90):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if details(container)["State"]["Health"]["Status"] == expected:
return
time.sleep(1)
raise Failure("api1 did not become " + expected)
def main():
client = Client(timeout=8)
seen = set()
for _ in range(20):
seen.add(client.call("GET", "/health/ready", schema="Health")
["headers"]["X-Instance-Id"])
check(seen == {"api-1", "api-2"}, "both instances must be ready initially")
actor, _, _, chat = client.bootstrap()
container = compose("ps", "-q", "api1")
check(bool(container), "api1 container is missing")
network = next((name for name in details(container)["NetworkSettings"]["Networks"]
if name.endswith("_backend")), None)
check(network is not None, "api1 backend network is missing")
disconnected = False
try:
docker("network", "disconnect", network, container)
disconnected = True
wait_health(container, "unhealthy")
for _ in range(20):
for path, schema in (("/api/v1/users/me", "User"),
("/health/ready", "Health")):
response = client.call("GET", path, actor=actor, schema=schema)
check(response["headers"]["X-Instance-Id"] == "api-2",
path + " was not served by healthy api2")
for number in range(10):
response = client.message(actor, chat, "unready check " + str(number))
check(response["headers"]["X-Instance-Id"] == "api-2",
"message was not accepted by healthy api2")
print("PASS: api1 unhealthy with frontend connection; 40 GETs and 10 POSTs served by api2")
finally:
if disconnected:
docker("network", "connect", "--alias", "api1", network, container)
wait_health(container, "healthy")
if __name__ == "__main__":
run(main)