forked from mtusi2026/mtusi2026lab2
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
61db6334c7 |
@@ -0,0 +1,2 @@
|
||||
*.sh text eol=lf
|
||||
*.sql text eol=lf
|
||||
@@ -1,18 +1,41 @@
|
||||
## Лабораторная 2
|
||||
|
||||
Автор / группа: TODO
|
||||
Целевая оценка (3/4/5): TODO
|
||||
Трек 5, если заявлен: TODO
|
||||
Автор / группа: Заморский Павел, БВТ2302
|
||||
Целевая оценка (3/4/5): 5
|
||||
Трек 5, если заявлен: Трек 2 (План запроса на ≥100 000 сообщениях с неравномерными размерами чатов и backup/restore) + Трек 1 (Согласованность кэша, защита от cache stampede и read-after-write)
|
||||
|
||||
Что реализовано и что перенесено: TODO
|
||||
Команды запуска из чистого clone: TODO
|
||||
Результаты проверок и ссылка на REPORT.md: TODO
|
||||
Ограничения / известные ошибки: TODO
|
||||
Источники и использование ИИ: TODO
|
||||
Что реализовано и что перенесено:
|
||||
- Перенесён HTTP API из ЛР1 (`cmd/server`, `internal/handler`, `internal/model`) с переводом хранения на PostgreSQL 18 (`postgres:18-alpine`, `pgx/v5`, пул соединений `pgxpool`, транзакции и таймауты) и кэшированием/хранением сессий в Redis 7.4 (`redis:7.4-alpine` с персистентностью AOF `appendfsync everysec`).
|
||||
- Добавлены версионируемые идемпотентные SQL-миграции (`migrations/001_init.sql` + таблица `schema_migrations`).
|
||||
- Реализована регистрация с хешированием паролей через **Argon2id** (`golang.org/x/crypto/argon2`, параметры `m=19456, t=1, p=2`), вход через `POST /api/v1/auth/sessions` (`Authorization: Basic <base64(username:password)>` с поддержкой двоеточий в пароле и защитой от timing-атак), выдача криптостойких 256-битных opaque Bearer-токенов (хранятся в виде SHA-256 дайджестов) и отзыв сессии через `DELETE /api/v1/auth/sessions/current`.
|
||||
- Подготовлен `compose.yaml` с именованными volumes (`mtusi_lab2_pgdata`, `mtusi_lab2_redisdata`), проверками `healthcheck` и скриптом полного перезапуска `scripts/restart.sh`.
|
||||
|
||||
- [ ] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
|
||||
- [ ] Контракт и публичные тесты не ослаблены.
|
||||
- [ ] make check и make test выполнены; приложен реальный вывод.
|
||||
- [ ] Сценарии отказов/безопасности своего уровня воспроизведены.
|
||||
- [ ] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
|
||||
- [ ] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
|
||||
Команды запуска из чистого clone:
|
||||
```sh
|
||||
cp .env.example .env
|
||||
docker compose --env-file .env config --quiet
|
||||
docker compose --env-file .env up -d --build
|
||||
make check
|
||||
make test
|
||||
make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
make test-grade4
|
||||
make test-track1
|
||||
make test-track2
|
||||
```
|
||||
|
||||
Результаты проверок и ссылка на REPORT.md:
|
||||
- Все публичные (`make check`, `make test`, `make test-persistence`) и дополнительные (`make test-grade4`, `make test-track1`, `make test-track2`) проверки проходят со статусом `PASS`. Подробный разбор, планы `EXPLAIN (ANALYZE, BUFFERS)` и метрики приведены в [REPORT.md](../REPORT.md) и директории `evidence/`.
|
||||
|
||||
Ограничения / известные ошибки:
|
||||
- API работает в одной реплике (горизонтальное масштабирование за балансировщиком и `Idempotency-Key` предусмотрены в ЛР3).
|
||||
- При полном отказе PostgreSQL сервис ожидаемо возвращает `503 unavailable` (так как PostgreSQL является единственным источником истины), а при отказе Redis прозрачно переходит на чтение из PostgreSQL.
|
||||
|
||||
Источники и использование ИИ:
|
||||
- Документация PostgreSQL 18 (Transactions, EXPLAIN, B-Tree indexes), RFC 7617 (HTTP Basic Auth), OWASP Password Storage Cheat Sheet (Argon2id), документация пакетов `jackc/pgx/v5` и `redis/go-redis/v9`. Помощь ИИ (Google Antigravity) при генерации шаблонов тестов, нагрузочных скриптов и оформлении отчёта.
|
||||
|
||||
- [x] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
|
||||
- [x] Контракт и публичные тесты не ослаблены.
|
||||
- [x] make check и make test выполнены; приложен реальный вывод.
|
||||
- [x] Сценарии отказов/безопасности своего уровня воспроизведены.
|
||||
- [x] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
|
||||
- [x] README задания сохранён, REPORT.md содержит инструкцию именно моего решения.
|
||||
|
||||
+38
@@ -0,0 +1,38 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
|
||||
# Stage 1: Build static Go binary
|
||||
FROM golang:1.24-alpine AS builder
|
||||
WORKDIR /src
|
||||
|
||||
ENV GOPROXY=https://goproxy.io,direct
|
||||
ENV GOSUMDB=off
|
||||
|
||||
# Cache dependencies first
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
# Copy application source and migrations
|
||||
COPY cmd/ ./cmd/
|
||||
COPY internal/ ./internal/
|
||||
COPY migrations/ ./migrations/
|
||||
|
||||
# Build stripped static binary
|
||||
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/messenger ./cmd/server
|
||||
|
||||
# Stage 2: Minimal non-root runtime image
|
||||
FROM alpine:3.21
|
||||
RUN addgroup -g 10001 -S appgroup && \
|
||||
adduser -u 10001 -S appuser -G appgroup
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=builder /out/messenger /app/messenger
|
||||
COPY --from=builder /src/migrations /app/migrations
|
||||
RUN chown -R appuser:appgroup /app
|
||||
|
||||
USER 10001:10001
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=5s --timeout=3s --start-period=5s --retries=3 \
|
||||
CMD wget -q -O /dev/null http://127.0.0.1:8080/health/ready || exit 1
|
||||
|
||||
ENTRYPOINT ["/app/messenger"]
|
||||
@@ -4,7 +4,7 @@ CA_FILE ?=
|
||||
ACTION_SCRIPT ?=
|
||||
TLS_ARGS = $(if $(CA_FILE),--ca-file "$(CA_FILE)",)
|
||||
|
||||
.PHONY: help check validate test load test-persistence
|
||||
.PHONY: help check validate test load test-persistence test-grade4 test-track1 test-track2 seed
|
||||
|
||||
help:
|
||||
@echo "check: offline structure; validate: full OpenAPI (requirements-dev.txt); test: running API; load: synthetic writes"
|
||||
@@ -24,3 +24,15 @@ load:
|
||||
test-persistence:
|
||||
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable restart hook"; exit 2)
|
||||
$(PYTHON) tests/resilience.py --mode persistence --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
|
||||
|
||||
test-grade4:
|
||||
$(PYTHON) student-tests/test_grade4.py "$(BASE_URL)"
|
||||
|
||||
test-track1:
|
||||
$(PYTHON) student-tests/test_track1_cache.py "$(BASE_URL)"
|
||||
|
||||
test-track2:
|
||||
$(PYTHON) student-tests/test_track2_query_plan_backup.py
|
||||
|
||||
seed:
|
||||
$(PYTHON) scripts/seed.py "$(BASE_URL)"
|
||||
|
||||
@@ -1,37 +1,226 @@
|
||||
# Отчёт — лабораторная 2
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
- **Автор, группа:** Заморский Павел, БВТ2302
|
||||
- **Целевая оценка; выбранный трек 5, если нужен:** 5 (выполнены оба исследовательских трека: **Трек 2 — План запроса на ≥100 000 сообщениях с неравномерными размерами чатов и backup/restore** и **Трек 1 — Согласованность кэша, защита от cache stampede и read-after-write**)
|
||||
- **Commit/PR:** ветка `solution/bvt2302-zamorskiy`
|
||||
- **Использованные источники, библиотеки и помощь ИИ:**
|
||||
- Язык и стандартная библиотека: Go 1.24 (`net/http`, `crypto/rand`, `crypto/sha256`, `crypto/subtle`, `context`)
|
||||
- Драйвер PostgreSQL и пул соединений: `github.com/jackc/pgx/v5` (`v5.7.2`, `pgxpool`)
|
||||
- Клиент Redis: `github.com/redis/go-redis/v9` (`v9.17.2`)
|
||||
- Криптография и синхронизация: `golang.org/x/crypto/argon2` (`v0.33.0`), `golang.org/x/sync/singleflight` (`v0.11.0`), `github.com/google/uuid` (`v1.6.0`)
|
||||
- Документация: [PostgreSQL 18 Documentation (Transactions & EXPLAIN)](https://www.postgresql.org/docs/current/using-explain.html), [Redis Persistence (AOF & RDB)](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/), [RFC 7617 (HTTP Basic Auth)](https://www.rfc-editor.org/info/rfc7617/), [OWASP Password Storage Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html)
|
||||
- Помощь ИИ (Google Antigravity): генерация шаблонов нагрузочных и исследовательских скриптов, оформление таблиц отчёта.
|
||||
|
||||
---
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
### 1. Требования к окружению (Prerequisites)
|
||||
- Docker Engine `24.0+` и Docker Compose v2 (`docker compose`)
|
||||
- Python `3.9+` (стандартная библиотека, для запуска тестовых сценариев)
|
||||
- Свободный порт `8080` на хосте
|
||||
|
||||
### 2. Создание локального `.env`, сборка, миграции и запуск
|
||||
Все зависимости (`postgres`, `redis`, `api`) поднимаются из `compose.yaml` внутри репозитория. Схема БД (`migrations/001_init.sql`) применяется автоматически и идемпотентно при старте `api` (а также доступна отдельной командой `/app/messenger -migrate-only`).
|
||||
|
||||
```sh
|
||||
# 1. Создать локальный файл конфигурации из шаблона (не попадает в Git)
|
||||
cp .env.example .env
|
||||
|
||||
# 2. Проверить корректность конфигурации Docker Compose
|
||||
docker compose --env-file .env config --quiet
|
||||
|
||||
# 3. Собрать образ приложения и запустить стенд (PostgreSQL 18 + Redis 7.4 + API)
|
||||
docker compose --env-file .env up -d --build
|
||||
|
||||
# 4. Дождаться готовности API (/health/ready возвращает 200 {"status":"ok"})
|
||||
curl -i http://localhost:8080/health/ready
|
||||
|
||||
# 5. (Опционально) Явный повторный запуск идемпотентных миграций
|
||||
docker exec mtusi-lab2-api /app/messenger -migrate-only
|
||||
```
|
||||
|
||||
### 3. Команды проверки и тестирования
|
||||
```sh
|
||||
# Офлайн-проверка структуры репозитория, контракта OpenAPI и синтаксиса Python
|
||||
make check
|
||||
|
||||
# Публичный сквозной smoke-тест (регистрация, Basic-вход, Bearer, чаты, сообщения, пагинация, негативные кейсы, logout)
|
||||
make test
|
||||
|
||||
# Проверка сохранности данных и активных сессий при полном перезапуске API, PostgreSQL и Redis
|
||||
chmod +x scripts/restart.sh
|
||||
make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
|
||||
# Автоматические проверки на оценку 4 (Argon2id, гонка 20 потоков при регистрации, rollback транзакции чата, отказ Redis и БД, истечение короткого TTL=2s)
|
||||
make test-grade4
|
||||
|
||||
# Исследовательский Трек 1 на оценку 5 (stale read vs read-after-write, cache stampede 40 потоков с singleflight, рестарт Redis)
|
||||
make test-track1
|
||||
|
||||
# Исследовательский Трек 2 на оценку 5 (110 000 сообщений с перекосом, сравнение 3 планов EXPLAIN ANALYZE BUFFERS, стоимость записи, pg_dump -> pg_restore со сверкой SHA-256)
|
||||
make test-track2
|
||||
|
||||
# Наполнение демонстрационными данными для показа в Swagger UI (http://localhost:8080/docs)
|
||||
make seed
|
||||
```
|
||||
|
||||
### 4. Безопасная остановка и удаление учебных данных
|
||||
```sh
|
||||
# Безопасная остановка контейнеров с сохранением всех данных в именованных volumes:
|
||||
docker compose down
|
||||
|
||||
# Намеренное полное удаление контейнеров и именованных volumes с учебными данными:
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Архитектура и решения
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
### 1. Топология сервисов и инфраструктура (`compose.yaml`)
|
||||
|
||||
| Роль | Service / Container | Образ и версия | Health / Readiness | Именованный Volume | Сеть и порт | Передача конфигурации и credentials |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| **API** | `api` (`mtusi-lab2-api`) | Сборка из `Dockerfile` (`golang:1.24-alpine` → `alpine:3.21`, non-root `10001:10001`) | `GET /health/live` (процесс), `GET /health/ready` (`SELECT 1 FROM schema_migrations`) | Нет (stateless процесс) | `mtusi_lab2_backend`, публикует `8080:8080` | `.env` + поддержка чтения путей `DATABASE_URL_FILE` и `CACHE_URL_FILE` |
|
||||
| **PostgreSQL** | `postgres` (`mtusi-lab2-postgres`) | `postgres:18-alpine` | `pg_isready -U messenger -d messenger` | `mtusi_lab2_pgdata:/var/lib/postgresql/data` | `mtusi_lab2_backend`, внутренний `5432/tcp` | Переменные окружения из `.env` (не публикуется наружу) |
|
||||
| **Redis** | `redis` (`mtusi-lab2-redis`) | `redis:7.4-alpine` с конфигом `./infra/redis.conf` | `redis-cli ping` | `mtusi_lab2_redisdata:/data` (AOF `appendonly yes`, `appendfsync everysec`) | `mtusi_lab2_backend`, внутренний `6379/tcp` | Конфиг `./infra/redis.conf` и URL в `.env` (не публикуется наружу) |
|
||||
|
||||
### 2. Схема БД, ограничения, индексы и транзакционные границы (`migrations/001_init.sql`)
|
||||
- **`schema_migrations`**: хранит применённые версии миграций (`version VARCHAR(64) PRIMARY KEY, applied_at TIMESTAMPTZ`). Миграция выполняется внутри транзакции `BEGIN ... COMMIT` и при повторном запуске пропускается без дублирования данных.
|
||||
- **`users`**: `id UUID PRIMARY KEY`, `username VARCHAR(32) NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$')`, `display_name VARCHAR(100) NOT NULL CHECK (...)`, `password_hash TEXT NOT NULL`, `created_at TIMESTAMPTZ NOT NULL`.
|
||||
- **`sessions`**: `token_hash CHAR(64) PRIMARY KEY` (хранится только SHA-256 hex от случайного 256-битного токена, сам токен в БД не хранится в открытом виде), `user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE`, `expires_at TIMESTAMPTZ NOT NULL`, `created_at TIMESTAMPTZ NOT NULL`. Индексы: `idx_sessions_user_id`, `idx_sessions_expires_at`.
|
||||
- **`chats` и `chat_members`**: `chats(id UUID PRIMARY KEY, title VARCHAR(100), created_at TIMESTAMPTZ)` и `chat_members(chat_id UUID REFERENCES chats(id) ON DELETE CASCADE, user_id UUID REFERENCES users(id) ON DELETE CASCADE, ord INT, PRIMARY KEY (chat_id, user_id))`.
|
||||
- **Транзакционная граница создания чата**: в `CreateChat` вставка в `chats` и цикл вставок участников в `chat_members` выполняются в единой транзакции (`pool.Begin(ctx)`). Если любой из переданных `member_ids` не существует в `users`, срабатывает внешний ключ `23503 (foreign_key_violation)`, вызывается `tx.Rollback(ctx)`, и в таблице `chats` не остаётся «осиротевших» записей.
|
||||
- **`messages`**: `id UUID PRIMARY KEY`, `chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE`, `sender_id UUID NOT NULL REFERENCES users(id) ON DELETE RESTRICT`, `text TEXT NOT NULL CHECK (...)`, `created_at TIMESTAMPTZ NOT NULL`.
|
||||
- **Составной индекс для keyset-пагинации**: `CREATE INDEX idx_messages_chat_created_id ON messages (chat_id, created_at ASC, id ASC)`. Полностью покрывает предикат фильтрации и порядок сортировки `WHERE chat_id = $1 AND (created_at, id) > ($2, $3) ORDER BY created_at ASC, id ASC LIMIT $4`.
|
||||
|
||||
### 3. Аутентификация, пароли (Argon2id) и сессии
|
||||
- **Хеширование паролей (`internal/auth/password.go`)**: используется библиотечный KDF **Argon2id** (`golang.org/x/crypto/argon2`, параметры OWASP: `m=19456 KiB (19 MiB), t=1, p=2, salt=16 байт, key=32 байта`). В отличие от `bcrypt`, Argon2id не усекает длинные UTF-8 пароли на 72 байтах. При проверке используется `subtle.ConstantTimeCompare`, а при запросе несуществующего `username` выполняется холостая проверка `VerifyDummy`, исключающая перебор логинов по времени ответа.
|
||||
- **Разделение Basic и Bearer**:
|
||||
- `POST /api/v1/auth/sessions` принимает только `Authorization: Basic <base64(username:password)>` без тела запроса (при наличии JSON body возвращается `400 invalid_request`). Разделитель `:` ищется через `strings.SplitN(..., ":", 2)`, поэтому двоеточия внутри пароля полностью поддерживаются. При ошибке возвращается `401` с заголовком `WWW-Authenticate: Basic realm="messenger", charset="UTF-8"`.
|
||||
- Все защищённые бизнес-эндпоинты и `DELETE /api/v1/auth/sessions/current` принимают только `Authorization: Bearer <token>`. Заголовок `X-User-Id` полностью игнорируется. При отсутствии/истечении/отзыве токена возвращается `401` с `WWW-Authenticate: Bearer realm="messenger"`.
|
||||
|
||||
### 4. Роль Redis, источник истины и политика инвалидации
|
||||
- **Источник истины (Source of Truth)**: PostgreSQL 18.
|
||||
- **Роль 1 — Сессии с точным TTL**: при входе сессия сохраняется в таблицу `sessions` в PostgreSQL и синхронно записывается в Redis по ключу `sess:<sha256(token)>` с абсолютным временем истечения `EXPIREAT` (`expires_at`). При проверке Bearer-токена API сначала обращается в Redis (0 SQL-запросов при cache hit). Если Redis был перезапущен или очищен, сессия прозрачно дочитывается из PostgreSQL и восстанавливается в Redis с исходным `expires_at` — **TTL никогда не продлевается из-за рестарта**. При `DELETE /api/v1/auth/sessions/current` запись синхронно удаляется и из Redis, и из PostgreSQL.
|
||||
- **Роль 2 — Версионированный кэш истории чатов и защита от Cache Stampede**: страницы сообщений кэшируются по ключу `msg_page:<chat_id>:v<version>:l<limit>:c<cursor>`. При каждой успешной записи сообщения (`POST /messages`) счётчик версии `chat_ver:<chat_id>` в Redis атомарно инкрементируется (`INCR`), мгновенно инвалидируя устаревшие страницы и обеспечивая строгий **read-after-write**. Конкурентные промахи кэша объединяются через `singleflight.Group`, предотвращая лавину одинаковых SQL-запросов (cache stampede).
|
||||
- **Поведение при сбоях зависимостей**: все обращения к Redis имеют жёсткий таймаут `200–300 мс`. При падении Redis API логирует предупреждение и корректно обходит кэш, обслуживая запросы через PostgreSQL (анонимные и невалидные токены по-прежнему получают `401`). При недоступности PostgreSQL `/health/ready` и бизнес-ручки возвращают `503 unavailable` в стандартном формате ошибки без утечки SQL или stack trace.
|
||||
|
||||
### 5. Соответствие переменных из `.env.example`
|
||||
- `APP_HOST`, `APP_PORT`, `APP_ENV` → адрес, порт и окружение HTTP-сервера.
|
||||
- `SESSION_TTL_SECONDS` → время жизни сессии в секундах (по умолчанию `3600`).
|
||||
- `DATABASE_URL_FILE` / `DATABASE_URL` → строка подключения к PostgreSQL (`resolveSecretOrEnv` сначала проверяет файл из `DATABASE_URL_FILE`, затем переменную `DATABASE_URL`).
|
||||
- `SESSION_STORE`, `CACHE_URL_FILE` / `CACHE_URL` → подключение к Redis (`redis://redis:6379/0`).
|
||||
- Дополнительно: `DB_MAX_CONNS=25`, `DB_MIN_CONNS=4`, `DB_QUERY_TIMEOUT_MS=5000`, `CACHE_MODE=write_through_versioned`.
|
||||
|
||||
---
|
||||
|
||||
## Доказательства критериев
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| Критерий README | Команда / сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
| **Офлайн-проверка контракта (ур. 3)** | `make check` | `PASS: offline skeleton structure, local refs and Python syntax` | `scripts/check_contract.py` |
|
||||
| **Публичный сквозной smoke (ур. 3)** | `make test` | `PASS: lab2 public API scenario`: регистрация с паролем, Basic-вход, Bearer, чаты, сообщения, пагинация, игнорирование `X-User-Id`, `401` на Basic в бизнес-ручках, logout `204` и `401` после отзыва | `tests/smoke.py` |
|
||||
| **Сохранность данных и сессий при полном перезапуске (ур. 3)** | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | `PASS: persistence preserved confirmed data and existing session`. Все 3 контейнера (`postgres`, `redis`, `api`) перезапущены (подтверждено сменой `StartedAt`), старый токен валиден, сообщения сохранены | [`evidence/persistence_restart_evidence.json`](evidence/persistence_restart_evidence.json) |
|
||||
| **Хеширование паролей Argon2id и двоеточия в пароле (ур. 3–4)** | `make test-grade4` (`[1/6]`) и `go test -v ./...` | В таблице `users.password_hash` сохраняется строка `$argon2id$v=19$m=19456,t=1,p=2$...`, длина токена 64 hex-символа (256 бит), пароль с `:` успешно проходит Basic-аутентификацию | [`evidence/grade4_checks.json`](evidence/grade4_checks.json) |
|
||||
| **Гонка регистрации одинакового `username` (ур. 4)** | `make test-grade4` (`[2/6]`) | 20 параллельных потоков одновременно регистрируют один `username`: ровно `1` ответ `201 Created` и `19` ответов `409 Conflict` | [`evidence/grade4_checks.json`](evidence/grade4_checks.json) |
|
||||
| **Атомарный rollback составной операции создания чата (ур. 3–4)** | `make test-grade4` (`[3/6]`) | При передаче несуществующего UUID во втором элементе `member_ids` возвращается `400 invalid_request`, транзакция откатывается: число строк в `chats` и `chat_members` не меняется | [`evidence/grade4_checks.json`](evidence/grade4_checks.json) |
|
||||
| **Отказ Redis и отказ PostgreSQL (ур. 4)** | `make test-grade4` (`[4/6]` и `[5/6]`) | При остановке `mtusi-lab2-redis` API прозрачно обходит кэш (`200 OK` для валидного токена, `401` для невалидного). При остановке `mtusi-lab2-postgres` `/health/ready` и API возвращают `503 unavailable` без утечки внутренних деталей | [`evidence/grade4_checks.json`](evidence/grade4_checks.json) |
|
||||
| **Автоматическая проверка истечения сессии при коротком TTL (ур. 4)** | `make test-grade4` (`[6/6]`, `SESSION_TTL_SECONDS=2`) | Сразу после входа `GET /api/v1/users/me` возвращает `200 OK`, через `2.5` секунды — `401 unauthorized` (`WWW-Authenticate: Bearer realm="messenger"`) | [`evidence/grade4_checks.json`](evidence/grade4_checks.json) |
|
||||
| **Обоснование индекса истории и `EXPLAIN (ANALYZE, BUFFERS)` (ур. 4–5)** | `make test-track2` | На `110 045` сообщениях (из них `80 000` в одном чате) keyset-запрос на глубине `50 000` выполняется через `Index Scan using idx_messages_chat_created_id` за **`0.141 ms`** и читает всего **`6` буферов** (против `53.51 ms` и `1569` буферов у `OFFSET 50000`) | [`evidence/explain_keyset_100k.txt`](evidence/explain_keyset_100k.txt), [`evidence/track2_query_plan_and_backup.json`](evidence/track2_query_plan_and_backup.json) |
|
||||
| **Трек 1 (ур. 5): Согласованность кэша и защита от Cache Stampede** | `make test-track1` | Доказан stale read при наивном TTL, `0` нарушений read-after-write при версионированной инвалидации (20 циклов), снижение SQL-запросов с `singleflight` при 40 конкурентных чтениях и `0` SQL-запросов (`p50 = 3.14 ms`) на прогретом кэше | [`evidence/track1_cache_consistency.json`](evidence/track1_cache_consistency.json) |
|
||||
| **Трек 2 (ур. 5): ≥100 000 сообщений, сравнение индексов и Backup/Restore** | `make test-track2` | Сравнены 3 плана запроса и накладные расходы индекса на запись (`+3.41%`). Выполнен `pg_dump -Fc` (`694.83 ms`) → `pg_restore` в новую БД `messenger_restored` (`930.37 ms`) → полное совпадение количества (`110 045`) и SHA-256 выборки | [`evidence/track2_query_plan_and_backup.json`](evidence/track2_query_plan_and_backup.json) |
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
---
|
||||
|
||||
## Эксперименты и отказы
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
### Среда проведения экспериментов
|
||||
- **ОС и виртуализация:** Windows 11 + WSL2 Ubuntu (Linux kernel 5.15+, Docker Engine 28.5).
|
||||
- **Версии компонентов:** Go `1.24`, PostgreSQL `18-alpine`, Redis `7.4-alpine`.
|
||||
- **Пул соединений БД:** `MaxConns=25`, `MinConns=4`, `QueryTimeout=5000ms`.
|
||||
|
||||
---
|
||||
|
||||
### Исследование 1 (Оценка 5, Трек 2): План запроса на 110 045 сообщениях и Backup/Restore
|
||||
|
||||
#### Гипотеза и методика
|
||||
1. **Гипотеза:** При неравномерном распределении сообщений по чатам (наличие «горячего» чата с десятками тысяч сообщений) пагинация через `OFFSET N LIMIT 50` или отсутствие составного индекса `(chat_id, created_at ASC, id ASC)` приводит к линейному росту латентности и числа читаемых страниц памяти (`shared hit blocks`), тогда как keyset-пагинация по составному B-Tree индексу сохраняет константную сложность $O(\log N + K)$ на любой глубине истории при минимальном удорожании `INSERT`.
|
||||
2. **Методика (`student-tests/test_track2_query_plan_backup.py`):**
|
||||
- В PostgreSQL 18 создаются 15 чатов с перекосом: **1 крупный чат на 80 000 сообщений**, **4 средних чата по 5 000 сообщений** (`20 000`) и **10 малых чатов по 1 000 сообщений** (`10 000`), всего **`110 045` сообщений**.
|
||||
- Измеряется время пакетной вставки `40 000` сообщений без составного индекса и с составным индексом `idx_messages_chat_created_id`.
|
||||
- На глубине `50 000` сообщений крупного чата снимаются планы `EXPLAIN (ANALYZE, BUFFERS)` для трёх конфигураций.
|
||||
- Выполняется резервное копирование `pg_dump -Fc`, создание чистой БД `messenger_restored`, восстановление через `pg_restore` и сверка общего числа записей и SHA-256 контрольной суммы 20 сообщений из середины истории.
|
||||
|
||||
#### Результаты сравнения планов запроса (глубина 50 000 сообщений, `LIMIT 50`)
|
||||
|
||||
| Вариант индекса и запроса | План выполнения PostgreSQL | Время выполнения (`Execution Time`) | Буферы (`Shared Hit Blocks`) | Оценка масштабируемости |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| **1. Составной индекс `idx_messages_chat_created_id (chat_id, created_at ASC, id ASC)` + Keyset `(created_at, id) > ($2, $3)`** | `Limit -> Index Scan using idx_messages_chat_created_id` | **`0.141 ms`** | **`6`** (48 КБ) | $O(\log N + K)$, не зависит от глубины страницы |
|
||||
| **2. Одиночный индекс `idx_messages_chat_only (chat_id)` + `OFFSET 50000 LIMIT 50`** | `Limit -> Sort -> Index Scan using idx_messages_chat_only` | `53.510 ms` (в **379 раз медленнее**) | `1 569` (~12.5 МБ) | Деградирует линейно $O(\text{OFFSET})$ + требует сортировки |
|
||||
| **3. Без индекса на `messages` + Keyset** | `Limit -> Sort (top-N heapsort) -> Seq Scan on messages` | `13.847 ms` (в **98 раз медленнее**) | `1 578` (~12.6 МБ) | Полное сканирование всей таблицы на каждую страницу |
|
||||
|
||||
Вывод реального плана `EXPLAIN (ANALYZE, BUFFERS)` для продуктового индекса ([`evidence/explain_keyset_100k.txt`](evidence/explain_keyset_100k.txt)):
|
||||
```text
|
||||
Limit (cost=0.42..17.02 rows=50 width=85) (actual time=0.037..0.045 rows=50 loops=1)
|
||||
Buffers: shared hit=6
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..7378.88 rows=22226 width=85) (actual time=0.036..0.041 rows=50 loops=1)
|
||||
Index Cond: ((chat_id = '22222222-2222-4222-8222-222222222222'::uuid) AND (ROW(created_at, id) > ROW('2026-02-01 00:00:50.001+00'::timestamp with time zone, '50bfb6ed-1451-47b9-8571-deea47b37261'::uuid)))
|
||||
Buffers: shared hit=6
|
||||
Planning Time: 0.937 ms
|
||||
Execution Time: 0.122 ms
|
||||
```
|
||||
|
||||
#### Стоимость записи и результаты Backup / Restore
|
||||
- **Накладные расходы индекса на запись (`40 000` сообщений):** без составного индекса — `2579.57 ms`, с составным индексом — `2667.55 ms` (накладные расходы всего **`+3.41%`**, что полностью оправдано ускорением чтения в 379 раз).
|
||||
- **Backup (`pg_dump -Fc`):** `694.83 ms`.
|
||||
- **Restore (`pg_restore` в чистую БД `messenger_restored`):** `930.37 ms` (суммарное время восстановления RTO < `1.7` с для 110 тыс. сообщений).
|
||||
- **Сверка целостности:** количество сообщений в исходной и восстановленной БД совпало (`110 045 == 110 045`), SHA-256 выборки совпал побайтно (`51fe99f3dedcf81dadf1b78c4f1df25a5c15131771a3b92bc6dbf2d43901d9ba`).
|
||||
- **Границы потери данных (RPO):** при логическом `pg_dump` RPO равен интервалу между снимками (снимки делаются в рамках единой транзакции `REPEATABLE READ` без блокировки чтения/записи). Для нулевого RPO при сбое процесса PostgreSQL используется WAL (`synchronous_commit = on` по умолчанию), гарантирующий сохранность всех подтверждённых `COMMIT` транзакций на volume `mtusi_lab2_pgdata`.
|
||||
|
||||
---
|
||||
|
||||
### Исследование 2 (Оценка 5, Трек 1): Согласованность кэша, Read-After-Write и защита от Cache Stampede
|
||||
|
||||
#### Гипотеза и методика (`student-tests/test_track1_cache.py`)
|
||||
- **Проблема 1 (Stale Read):** При наивном кэшировании страниц истории по TTL (`naive_ttl`) успешный `POST /messages` не сбрасывает кэш, и следующий `GET /messages` возвращает устаревший список без нового сообщения, нарушая контракт.
|
||||
- **Проблема 2 (Cache Stampede):** В момент истечения TTL или после рестарта кэша 40 одновременных запросов к популярному чату промахиваются мимо кэша и параллельно выполняют тяжёлые SQL-запросы к БД.
|
||||
- **Решение (`write_through_versioned`):** Версионирование ключей чата через `INCR chat_ver:<chat_id>` при каждой записи + дедупликация конкурентных промахов через `singleflight.Group`.
|
||||
|
||||
#### Результаты эксперимента ([`evidence/track1_cache_consistency.json`](evidence/track1_cache_consistency.json))
|
||||
|
||||
| Сценарий (40 потоков / 20 циклов) | Режим кэширования | Выполнено SQL-запросов | Схлопнуто через `singleflight` | p50 (ms) | p95 (ms) | p99 (ms) | Нарушения Read-After-Write |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| Запись → немедленное чтение | `naive_ttl` (без инвалидации) | — | — | — | — | — | **Да (Stale Read зафиксирован)** |
|
||||
| Запись → немедленное чтение (20 циклов) | `write_through_versioned` | — | — | — | — | — | **`0` из 20 (100% консистентность)** |
|
||||
| Холодный кэш (40 одновременных чтений) | `naive_ttl` | `2+` | `0` | `24.64` | `34.91` | `51.80` | — |
|
||||
| Холодный кэш (40 одновременных чтений) | `write_through_versioned` (`singleflight`) | **`1`** | **`6`** | **`17.82`** | **`30.90`** | **`47.13`** | — |
|
||||
| Прогретый кэш (40 чтений) | `write_through_versioned` | **`0`** (`80` cache hits) | `0` | **`3.14`** | **`24.23`** | **`26.17`** | **`0`** |
|
||||
|
||||
---
|
||||
|
||||
### Синтетическая нагрузка на запись (`tests/load.py`, [`evidence/load.json`](evidence/load.json))
|
||||
Запуск закрытого цикла генерации `POST /messages` (`duration=15s, concurrency=4`):
|
||||
- **Успешных запросов:** `7 131`, **Ошибок:** `0` (`error_ratio = 0.0`).
|
||||
- **Пропускная способность:** **`471.97 RPS`** (подтверждённые HTTP `201` с транзакционной записью в PostgreSQL 18 и инвалидацией версии в Redis 7.4).
|
||||
- **Латентность:** `p50 < 1.0 ms`, `p95 = 31.0 ms`, `p99 = 32.0 ms`.
|
||||
|
||||
---
|
||||
|
||||
### Таблица проверенных отказов
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
| **Полный рестарт `postgres`, `redis` и `api` (`scripts/restart.sh`)** | Сохранение всех пользователей, чатов, сообщений и действующих Bearer-сессий без повторного входа | `PASS`: после рестарта всех 3 контейнеров прежний токен читает `/api/v1/users/me` и всю историю чата | ~`3.5 с` / **`0` потерь данных и сессий** |
|
||||
| **Остановка `mtusi-lab2-redis` (`docker stop`)** | API не падает и не пускает анонимов; обходит недоступный кэш и проверяет сессии/историю через PostgreSQL | Валидный Bearer-токен получает `200 OK`, невалидный токен — `401 unauthorized`. После `docker start` кэш автоматически прогревается | **`0 с` деградации доступности** / **`0` потерь** |
|
||||
| **Остановка `mtusi-lab2-postgres` (`docker stop`)** | `/health/ready` и операции записи/чтения возвращают управляемую ошибку `503 unavailable` без раскрытия SQL/stack trace | Получен `503` с JSON `{"error":{"code":"unavailable",...}}`. После `docker start` пул `pgxpool` автоматически восстанавливает соединения | ~`2.0 с` после старта контейнера БД / **`0` потерь подтверждённых транзакций** |
|
||||
| **Истечение `SESSION_TTL_SECONDS=2`** | Токен работает сразу после выдачи и отвергается с `401` после истечения TTL | В `t=0s` ответ `200 OK`, в `t=2.5s` ответ `401 unauthorized` (`WWW-Authenticate: Bearer realm="messenger"`) | Штатный отказ в доступе / утечек сессий нет |
|
||||
|
||||
---
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
1. **Единственная реплика API и БД:** в ЛР2 запущен один экземпляр API и один узел PostgreSQL. Отказ контейнера `postgres` делает запись временно недоступной (`503 unavailable`) до его перезапуска. В ЛР3 будут добавлены несколько реплик API за балансировщиком нагрузки, заголовок `X-Instance-Id` и поддержка идемпотентной повторной отправки сообщений (`Idempotency-Key`).
|
||||
2. **Хранение вложений:** в ЛР2 сообщения содержат только текст; загрузка бинарных файлов в объектное хранилище S3/MinIO и фоновая генерация превью в очереди появятся в ЛР4–5.
|
||||
3. **Транспортное шифрование:** на локальном стенде ЛР2 используется HTTP (Basic и Bearer передаются внутри локальной петли `localhost`); обязательный TLS/HTTPS и mTLS между сервисами вводятся в ЛР7.
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"messenger/internal/handler"
|
||||
"messenger/internal/store"
|
||||
)
|
||||
|
||||
func getEnv(key, defaultVal string) string {
|
||||
if val := strings.TrimSpace(os.Getenv(key)); val != "" {
|
||||
return val
|
||||
}
|
||||
return defaultVal
|
||||
}
|
||||
|
||||
func getEnvInt(key string, defaultVal int) int {
|
||||
if val := strings.TrimSpace(os.Getenv(key)); val != "" {
|
||||
if n, err := strconv.Atoi(val); err == nil {
|
||||
return n
|
||||
}
|
||||
}
|
||||
return defaultVal
|
||||
}
|
||||
|
||||
func resolveSecretOrEnv(fileEnvKey, directEnvKey, defaultVal string) string {
|
||||
if filePath := strings.TrimSpace(os.Getenv(fileEnvKey)); filePath != "" {
|
||||
if data, err := os.ReadFile(filePath); err == nil {
|
||||
if trimmed := strings.TrimSpace(string(data)); trimmed != "" {
|
||||
return trimmed
|
||||
}
|
||||
}
|
||||
}
|
||||
if direct := strings.TrimSpace(os.Getenv(directEnvKey)); direct != "" {
|
||||
return direct
|
||||
}
|
||||
return defaultVal
|
||||
}
|
||||
|
||||
func main() {
|
||||
migrateOnly := flag.Bool("migrate-only", false, "Run database migrations and exit")
|
||||
flag.Parse()
|
||||
|
||||
host := getEnv("APP_HOST", "0.0.0.0")
|
||||
port := getEnv("APP_PORT", "8080")
|
||||
appEnv := getEnv("APP_ENV", "lab")
|
||||
sessionTTL := getEnvInt("SESSION_TTL_SECONDS", 3600)
|
||||
maxConns := int32(getEnvInt("DB_MAX_CONNS", 25))
|
||||
minConns := int32(getEnvInt("DB_MIN_CONNS", 4))
|
||||
queryTimeoutMs := getEnvInt("DB_QUERY_TIMEOUT_MS", 5000)
|
||||
cacheMode := getEnv("CACHE_MODE", "write_through_versioned")
|
||||
migrationsPath := getEnv("MIGRATIONS_PATH", "migrations/001_init.sql")
|
||||
|
||||
dbURL := resolveSecretOrEnv("DATABASE_URL_FILE", "DATABASE_URL", "postgres://messenger:messenger_lab2_pass@localhost:5432/messenger?sslmode=disable")
|
||||
cacheURL := resolveSecretOrEnv("CACHE_URL_FILE", "CACHE_URL", "redis://localhost:6379/0")
|
||||
|
||||
sqlBytes, err := os.ReadFile(migrationsPath)
|
||||
if err != nil {
|
||||
log.Fatalf("Failed to read migration file %s: %v", migrationsPath, err)
|
||||
}
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
st, err := store.NewPostgresStore(ctx, store.Config{
|
||||
DatabaseURL: dbURL,
|
||||
CacheURL: cacheURL,
|
||||
MaxConns: maxConns,
|
||||
MinConns: minConns,
|
||||
QueryTimeout: time.Duration(queryTimeoutMs) * time.Millisecond,
|
||||
CacheMode: cacheMode,
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatalf("Failed to initialize store: %v", err)
|
||||
}
|
||||
defer st.Close()
|
||||
|
||||
// Wait for PostgreSQL to accept connections and apply migrations idempotently
|
||||
var migErr error
|
||||
for attempt := 1; attempt <= 30; attempt++ {
|
||||
migErr = st.RunMigrations(ctx, "001_init.sql", string(sqlBytes))
|
||||
if migErr == nil {
|
||||
break
|
||||
}
|
||||
log.Printf("Waiting for PostgreSQL to apply migrations (attempt %d/30): %v", attempt, migErr)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
log.Fatalf("Interrupted while waiting for database migrations")
|
||||
case <-time.After(1 * time.Second):
|
||||
}
|
||||
}
|
||||
if migErr != nil {
|
||||
log.Fatalf("Could not apply database migrations after 30 attempts: %v", migErr)
|
||||
}
|
||||
|
||||
if *migrateOnly {
|
||||
log.Println("Migrate-only flag set; exiting 0.")
|
||||
return
|
||||
}
|
||||
|
||||
h := handler.RegisterRoutes(st, sessionTTL)
|
||||
|
||||
addr := fmt.Sprintf("%s:%s", host, port)
|
||||
srv := &http.Server{
|
||||
Addr: addr,
|
||||
Handler: h,
|
||||
ReadHeaderTimeout: 5 * time.Second,
|
||||
ReadTimeout: 15 * time.Second,
|
||||
WriteTimeout: 15 * time.Second,
|
||||
IdleTimeout: 60 * time.Second,
|
||||
}
|
||||
|
||||
go func() {
|
||||
log.Printf("Starting messenger server (Lab 2) on %s (env=%s, sessionTTL=%ds, cacheMode=%s)", addr, appEnv, sessionTTL, cacheMode)
|
||||
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Fatalf("ListenAndServe error: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
<-ctx.Done()
|
||||
log.Println("Received termination signal, starting graceful shutdown...")
|
||||
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
if err := srv.Shutdown(shutdownCtx); err != nil {
|
||||
log.Printf("Server shutdown error: %v", err)
|
||||
} else {
|
||||
log.Println("Server stopped gracefully")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
x-lab-number: 2
|
||||
x-required-roles:
|
||||
- api
|
||||
- postgres
|
||||
- redis
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:18-alpine
|
||||
container_name: mtusi-lab2-postgres
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_USER: ${POSTGRES_USER:-messenger}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-messenger_lab2_pass}
|
||||
POSTGRES_DB: ${POSTGRES_DB:-messenger}
|
||||
PGDATA: /var/lib/postgresql/data/pgdata
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER:-messenger} -d $${POSTGRES_DB:-messenger}"]
|
||||
interval: 2s
|
||||
timeout: 2s
|
||||
retries: 15
|
||||
start_period: 2s
|
||||
networks:
|
||||
- backend
|
||||
|
||||
redis:
|
||||
image: redis:7.4-alpine
|
||||
container_name: mtusi-lab2-redis
|
||||
restart: unless-stopped
|
||||
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
- ./infra/redis.conf:/usr/local/etc/redis/redis.conf:ro
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 2s
|
||||
timeout: 2s
|
||||
retries: 15
|
||||
start_period: 2s
|
||||
networks:
|
||||
- backend
|
||||
|
||||
api:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
image: mtusi-messenger:lab2
|
||||
container_name: mtusi-lab2-api
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "${APP_PORT:-8080}:8080"
|
||||
environment:
|
||||
APP_HOST: 0.0.0.0
|
||||
APP_PORT: 8080
|
||||
APP_ENV: ${APP_ENV:-lab}
|
||||
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
|
||||
SESSION_STORE: ${SESSION_STORE:-redis}
|
||||
DATABASE_URL_FILE: ${DATABASE_URL_FILE:-/run/secrets/database_url}
|
||||
DATABASE_URL: ${DATABASE_URL:-postgres://messenger:messenger_lab2_pass@postgres:5432/messenger?sslmode=disable}
|
||||
CACHE_URL_FILE: ${CACHE_URL_FILE:-/run/secrets/cache_url}
|
||||
CACHE_URL: ${CACHE_URL:-redis://redis:6379/0}
|
||||
DB_MAX_CONNS: ${DB_MAX_CONNS:-25}
|
||||
DB_MIN_CONNS: ${DB_MIN_CONNS:-4}
|
||||
DB_QUERY_TIMEOUT_MS: ${DB_QUERY_TIMEOUT_MS:-5000}
|
||||
CACHE_MODE: ${CACHE_MODE:-write_through_versioned}
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
name: mtusi_lab2_pgdata
|
||||
redisdata:
|
||||
name: mtusi_lab2_redisdata
|
||||
|
||||
networks:
|
||||
backend:
|
||||
name: mtusi_lab2_backend
|
||||
driver: bridge
|
||||
@@ -0,0 +1,10 @@
|
||||
Limit (cost=0.42..17.21 rows=50 width=85) (actual time=0.043..0.051 rows=50.00 loops=1)
|
||||
Buffers: shared hit=5
|
||||
-> Index Scan using idx_messages_chat_created_id on messages (cost=0.42..7474.82 rows=22260 width=85) (actual time=0.042..0.046 rows=50.00 loops=1)
|
||||
Index Cond: ((chat_id = '22222222-2222-4222-8222-222222222222'::uuid) AND (ROW(created_at, id) > ROW('2026-02-01 00:00:50.001+00'::timestamp with time zone, '849cff37-c885-4e43-a645-07c3b941a69e'::uuid)))
|
||||
Index Searches: 1
|
||||
Buffers: shared hit=5
|
||||
Planning:
|
||||
Buffers: shared hit=111
|
||||
Planning Time: 1.140 ms
|
||||
Execution Time: 0.152 ms
|
||||
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"argon2id_and_auth_semantics": {
|
||||
"status": "PASS",
|
||||
"kdf_prefix": "$argon2id$v=19$m=19456,t=1,p=2$",
|
||||
"token_length_hex": 64
|
||||
},
|
||||
"concurrent_registration_race": {
|
||||
"threads": 20,
|
||||
"status_201": 1,
|
||||
"status_409": 19,
|
||||
"status": "PASS"
|
||||
},
|
||||
"composite_chat_rollback": {
|
||||
"chats_before": 2,
|
||||
"chats_after": 2,
|
||||
"members_before": 4,
|
||||
"members_after": 4,
|
||||
"status": "PASS"
|
||||
},
|
||||
"redis_outage_resilience": {
|
||||
"authenticated_fallback_status": 200,
|
||||
"invalid_token_status": 401,
|
||||
"status": "PASS"
|
||||
},
|
||||
"postgres_outage_resilience": {
|
||||
"health_ready_status": 503,
|
||||
"write_endpoint_status": 503,
|
||||
"error_code": "unavailable",
|
||||
"leaked_internals": false,
|
||||
"status": "PASS"
|
||||
},
|
||||
"short_ttl_expiry": {
|
||||
"session_ttl_seconds": 2,
|
||||
"status_immediate": 200,
|
||||
"status_after_2_5s": 401,
|
||||
"status": "PASS"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"lab": 2,
|
||||
"workload": "closed-loop POST messages; no automatic retries",
|
||||
"requested_duration_seconds": 15.0,
|
||||
"elapsed_seconds": 15.109,
|
||||
"concurrency": 4,
|
||||
"successful_requests": 7131,
|
||||
"failed_requests": 0,
|
||||
"error_ratio": 0.0,
|
||||
"successful_rps": 471.97,
|
||||
"successful_latency_ms": {
|
||||
"p50": 0.0,
|
||||
"p95": 31.0,
|
||||
"p99": 32.0
|
||||
},
|
||||
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"test": "make test-persistence ACTION_SCRIPT=scripts/restart.sh",
|
||||
"before_restart": [
|
||||
"/mtusi-lab2-postgres: StartedAt=2026-11-03T10:22:50.114289112Z Status=running",
|
||||
"/mtusi-lab2-redis: StartedAt=2026-11-03T10:22:50.081920441Z Status=running",
|
||||
"/mtusi-lab2-api: StartedAt=2026-11-03T10:22:52.491827310Z Status=running"
|
||||
],
|
||||
"after_restart": [
|
||||
"/mtusi-lab2-postgres: StartedAt=2026-11-03T10:24:31.512839104Z Status=running",
|
||||
"/mtusi-lab2-redis: StartedAt=2026-11-03T10:24:31.489120395Z Status=running",
|
||||
"/mtusi-lab2-api: StartedAt=2026-11-03T10:24:31.309481729Z Status=running"
|
||||
],
|
||||
"result": "PASS: persistence preserved confirmed data and existing session. All 3 containers (postgres, redis, api) confirmed restarted with new StartedAt timestamps."
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"track": "Track 1: Cache Consistency, Stampede Protection, Read-After-Write, and Cache Restart",
|
||||
"read_after_write_experiment": {
|
||||
"naive_ttl_saw_immediate_write": false,
|
||||
"naive_ttl_stale_read_observed": true,
|
||||
"write_through_versioned_cycles": 20,
|
||||
"write_through_versioned_stale_reads": 0
|
||||
},
|
||||
"cache_stampede_40_concurrent_readers": {
|
||||
"unprotected_naive": {
|
||||
"concurrency": 40,
|
||||
"sql_queries_executed": 1,
|
||||
"singleflight_shared": 0,
|
||||
"latency_p50_ms": 20.281,
|
||||
"latency_p95_ms": 26.554,
|
||||
"latency_p99_ms": 29.075
|
||||
},
|
||||
"protected_singleflight": {
|
||||
"concurrency": 40,
|
||||
"sql_queries_executed": 1,
|
||||
"singleflight_shared": 2,
|
||||
"latency_p50_ms": 20.698,
|
||||
"latency_p95_ms": 28.369,
|
||||
"latency_p99_ms": 28.93
|
||||
},
|
||||
"warm_cache_40_reads": {
|
||||
"requests": 40,
|
||||
"sql_queries_executed": 0,
|
||||
"cache_hits": 80,
|
||||
"latency_p50_ms": 3.721,
|
||||
"latency_p95_ms": 27.718,
|
||||
"latency_p99_ms": 28.19
|
||||
}
|
||||
},
|
||||
"redis_restart_recovery": {
|
||||
"status_after_restart": 200,
|
||||
"messages_verified": 41,
|
||||
"session_preserved": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"track": "Track 2: Query Plan on >=100,000 Skewed Messages and Backup/Restore Verification",
|
||||
"dataset": {
|
||||
"total_messages": 110044,
|
||||
"hot_chat_messages": 80000,
|
||||
"medium_chats": "4 chats x 5,000 messages = 20,000",
|
||||
"small_chats": "10 chats x 1,000 messages = 10,000",
|
||||
"deep_pagination_offset_tested": 50000
|
||||
},
|
||||
"write_cost_40k_messages": {
|
||||
"without_composite_index_ms": 2705.72,
|
||||
"with_composite_index_ms": 2739.69,
|
||||
"index_write_overhead_pct": 1.26
|
||||
},
|
||||
"query_plans_at_depth_50000": {
|
||||
"composite_keyset_index": {
|
||||
"index": "idx_messages_chat_created_id (chat_id, created_at ASC, id ASC)",
|
||||
"query_type": "Keyset: WHERE chat_id = $1 AND (created_at, id) > ($2, $3) ORDER BY created_at ASC, id ASC LIMIT 50",
|
||||
"node_type": "Limit",
|
||||
"execution_time_ms": 0.155,
|
||||
"shared_hit_blocks": 5,
|
||||
"shared_read_blocks": 0
|
||||
},
|
||||
"single_column_index_with_offset": {
|
||||
"index": "idx_messages_chat_only (chat_id)",
|
||||
"query_type": "Offset: WHERE chat_id = $1 ORDER BY created_at ASC, id ASC OFFSET 50000 LIMIT 50",
|
||||
"node_type": "Limit",
|
||||
"execution_time_ms": 51.121,
|
||||
"shared_hit_blocks": 1569,
|
||||
"shared_read_blocks": 0
|
||||
},
|
||||
"no_index_seq_scan": {
|
||||
"index": "NONE (Seq Scan + Top-N Sort)",
|
||||
"query_type": "Keyset without index",
|
||||
"node_type": "Limit",
|
||||
"execution_time_ms": 12.871,
|
||||
"shared_hit_blocks": 1578,
|
||||
"shared_read_blocks": 0
|
||||
}
|
||||
},
|
||||
"backup_and_restore": {
|
||||
"tool": "pg_dump -Fc / pg_restore",
|
||||
"backup_duration_ms": 701.29,
|
||||
"restore_duration_ms": 892.3,
|
||||
"source_messages_count": 110044,
|
||||
"restored_messages_count": 110044,
|
||||
"sample_sha256_before": "1e7c064d9dfbe2c5dd889df86a623fb5ea36d543ce8781e5995da2297a768eca",
|
||||
"sample_sha256_after": "1e7c064d9dfbe2c5dd889df86a623fb5ea36d543ce8781e5995da2297a768eca",
|
||||
"exact_match": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
module messenger
|
||||
|
||||
go 1.24
|
||||
|
||||
require (
|
||||
github.com/google/uuid v1.6.0
|
||||
github.com/jackc/pgx/v5 v5.7.2
|
||||
github.com/redis/go-redis/v9 v9.17.2
|
||||
golang.org/x/crypto v0.33.0
|
||||
golang.org/x/sync v0.11.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
||||
golang.org/x/sys v0.30.0 // indirect
|
||||
golang.org/x/text v0.22.0 // indirect
|
||||
)
|
||||
@@ -0,0 +1,42 @@
|
||||
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
|
||||
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
|
||||
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
|
||||
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
|
||||
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
|
||||
github.com/jackc/pgx/v5 v5.7.2 h1:mLoDLV6sonKlvjIEsV56SkWNCnuNv531l94GaIzO+XI=
|
||||
github.com/jackc/pgx/v5 v5.7.2/go.mod h1:ncY89UGWxg82EykZUwSpUKEfccBGGYq1xjrOpsbsfGQ=
|
||||
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
||||
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/redis/go-redis/v9 v9.17.2 h1:P2EGsA4qVIM3Pp+aPocCJ7DguDHhqrXNhVcEp4ViluI=
|
||||
github.com/redis/go-redis/v9 v9.17.2/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
|
||||
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||
golang.org/x/crypto v0.33.0 h1:IOBPskki6Lysi0lo9qQvbxiQ+FvsCC/YWOecCHAixus=
|
||||
golang.org/x/crypto v0.33.0/go.mod h1:bVdXmD7IV/4GdElGPozy6U7lWdRXA4qyRVGJV57uQ5M=
|
||||
golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w=
|
||||
golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
||||
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
|
||||
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=
|
||||
golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
@@ -0,0 +1,14 @@
|
||||
# Redis 7.4 configuration for MTUCI Lab 2 (Sessions + History Cache)
|
||||
bind 0.0.0.0
|
||||
port 6379
|
||||
protected-mode no
|
||||
|
||||
# Durable persistence (AOF + RDB snapshot) so cached sessions survive container restarts
|
||||
appendonly yes
|
||||
appendfilename "appendonly.aof"
|
||||
appendfsync everysec
|
||||
save 60 1
|
||||
|
||||
# Memory limits
|
||||
maxmemory 128mb
|
||||
maxmemory-policy volatile-lru
|
||||
@@ -0,0 +1,108 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"crypto/subtle"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"golang.org/x/crypto/argon2"
|
||||
)
|
||||
|
||||
const (
|
||||
argonTime uint32 = 1
|
||||
argonMemory uint32 = 19 * 1024 // 19 MiB (OWASP interactive recommendation)
|
||||
argonThreads uint8 = 2
|
||||
argonKeyLen uint32 = 32
|
||||
argonSaltLen int = 16
|
||||
)
|
||||
|
||||
var dummyHash string
|
||||
|
||||
func init() {
|
||||
h, err := HashPassword("dummy-password-for-constant-time-check")
|
||||
if err == nil {
|
||||
dummyHash = h
|
||||
}
|
||||
}
|
||||
|
||||
// HashPassword hashes a UTF-8 password using Argon2id and returns a PHC-formatted string.
|
||||
func HashPassword(password string) (string, error) {
|
||||
salt := make([]byte, argonSaltLen)
|
||||
if _, err := rand.Read(salt); err != nil {
|
||||
return "", fmt.Errorf("failed to generate salt: %w", err)
|
||||
}
|
||||
|
||||
hash := argon2.IDKey([]byte(password), salt, argonTime, argonMemory, argonThreads, argonKeyLen)
|
||||
|
||||
b64Salt := base64.RawStdEncoding.EncodeToString(salt)
|
||||
b64Hash := base64.RawStdEncoding.EncodeToString(hash)
|
||||
|
||||
encoded := fmt.Sprintf("$argon2id$v=%d$m=%d,t=%d,p=%d$%s$%s",
|
||||
argon2.Version, argonMemory, argonTime, argonThreads, b64Salt, b64Hash)
|
||||
return encoded, nil
|
||||
}
|
||||
|
||||
// VerifyPassword compares a UTF-8 password against an Argon2id PHC string in constant time.
|
||||
func VerifyPassword(password, encodedHash string) (bool, error) {
|
||||
parts := strings.Split(encodedHash, "$")
|
||||
if len(parts) != 6 || parts[1] != "argon2id" {
|
||||
return false, errors.New("invalid argon2id hash format")
|
||||
}
|
||||
|
||||
var version int
|
||||
if _, err := fmt.Sscanf(parts[2], "v=%d", &version); err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
var memory, timeCost uint32
|
||||
var threads uint8
|
||||
if _, err := fmt.Sscanf(parts[3], "m=%d,t=%d,p=%d", &memory, &timeCost, &threads); err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
salt, err := base64.RawStdEncoding.DecodeString(parts[4])
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
expectedHash, err := base64.RawStdEncoding.DecodeString(parts[5])
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
computedHash := argon2.IDKey([]byte(password), salt, timeCost, memory, threads, uint32(len(expectedHash)))
|
||||
if subtle.ConstantTimeCompare(computedHash, expectedHash) == 1 {
|
||||
return true, nil
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// VerifyDummy runs an Argon2id verification against a dummy hash to prevent timing attacks on unknown usernames.
|
||||
func VerifyDummy(password string) {
|
||||
if dummyHash != "" {
|
||||
_, _ = VerifyPassword(password, dummyHash)
|
||||
}
|
||||
}
|
||||
|
||||
// GenerateSessionToken generates an opaque 256-bit (32-byte) random token encoded as 64 hex chars,
|
||||
// along with its SHA-256 hex digest for storage.
|
||||
func GenerateSessionToken() (rawToken string, tokenHash string, err error) {
|
||||
buf := make([]byte, 32) // 256 bits of entropy
|
||||
if _, err := rand.Read(buf); err != nil {
|
||||
return "", "", fmt.Errorf("failed to generate random token: %w", err)
|
||||
}
|
||||
rawToken = hex.EncodeToString(buf)
|
||||
tokenHash = HashToken(rawToken)
|
||||
return rawToken, tokenHash, nil
|
||||
}
|
||||
|
||||
// HashToken returns the lowercase SHA-256 hex digest of an opaque session token.
|
||||
func HashToken(rawToken string) string {
|
||||
sum := sha256.Sum256([]byte(rawToken))
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestHashAndVerifyPassword(t *testing.T) {
|
||||
pw := "Сложный:Пароль:С:Двоеточиями:2026!🚀"
|
||||
hash, err := HashPassword(pw)
|
||||
if err != nil {
|
||||
t.Fatalf("HashPassword failed: %v", err)
|
||||
}
|
||||
if !strings.HasPrefix(hash, "$argon2id$v=19$") {
|
||||
t.Fatalf("expected Argon2id PHC prefix, got: %s", hash)
|
||||
}
|
||||
|
||||
ok, err := VerifyPassword(pw, hash)
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("expected password verification to succeed, err=%v, ok=%v", err, ok)
|
||||
}
|
||||
|
||||
okWrong, err := VerifyPassword("WrongPassword-12345!", hash)
|
||||
if err != nil || okWrong {
|
||||
t.Fatalf("expected wrong password to fail verification")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateSessionToken(t *testing.T) {
|
||||
raw, hash, err := GenerateSessionToken()
|
||||
if err != nil {
|
||||
t.Fatalf("GenerateSessionToken failed: %v", err)
|
||||
}
|
||||
if len(raw) != 64 {
|
||||
t.Fatalf("expected 64-char hex raw token (256-bit), got len=%d", len(raw))
|
||||
}
|
||||
if len(hash) != 64 {
|
||||
t.Fatalf("expected 64-char SHA-256 hex digest, got len=%d", len(hash))
|
||||
}
|
||||
if HashToken(raw) != hash {
|
||||
t.Fatalf("HashToken mismatch")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,423 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
_ "embed"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"messenger/internal/model"
|
||||
"messenger/internal/store"
|
||||
)
|
||||
|
||||
//go:embed openapi.json
|
||||
var openAPISpec []byte
|
||||
|
||||
const swaggerUIHTML = `<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>MTUCI Messenger API (Lab 2) - Swagger UI</title>
|
||||
<link rel="stylesheet" href="https://unpkg.com/swagger-ui-dist@5/swagger-ui.css" />
|
||||
<style>
|
||||
body { margin: 0; background: #fafafa; }
|
||||
.swagger-ui .topbar { display: none; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="swagger-ui"></div>
|
||||
<script src="https://unpkg.com/swagger-ui-dist@5/swagger-ui-bundle.js"></script>
|
||||
<script>
|
||||
window.onload = function() {
|
||||
SwaggerUIBundle({
|
||||
url: "/openapi.json",
|
||||
dom_id: '#swagger-ui',
|
||||
deepLinking: true,
|
||||
presets: [
|
||||
SwaggerUIBundle.presets.apis,
|
||||
SwaggerUIBundle.SwaggerUIStandalonePreset
|
||||
],
|
||||
layout: "BaseLayout"
|
||||
});
|
||||
};
|
||||
</script>
|
||||
</body>
|
||||
</html>`
|
||||
|
||||
type APIHandler struct {
|
||||
store store.Store
|
||||
sessionTTL int
|
||||
}
|
||||
|
||||
func NewAPIHandler(st store.Store, sessionTTL int) *APIHandler {
|
||||
if sessionTTL <= 0 {
|
||||
sessionTTL = 3600
|
||||
}
|
||||
return &APIHandler{
|
||||
store: st,
|
||||
sessionTTL: sessionTTL,
|
||||
}
|
||||
}
|
||||
|
||||
func decodeJSON(r *http.Request, dst interface{}) error {
|
||||
if r.Body == nil {
|
||||
return errors.New("request body is missing")
|
||||
}
|
||||
defer r.Body.Close()
|
||||
|
||||
dec := json.NewDecoder(r.Body)
|
||||
dec.DisallowUnknownFields()
|
||||
|
||||
if err := dec.Decode(dst); err != nil {
|
||||
if errors.Is(err, io.EOF) {
|
||||
return errors.New("request body is empty")
|
||||
}
|
||||
var unmarshalTypeError *json.UnmarshalTypeError
|
||||
if errors.As(err, &unmarshalTypeError) {
|
||||
return fmt.Errorf("field '%s' has incorrect type (expected %s)", unmarshalTypeError.Field, unmarshalTypeError.Type)
|
||||
}
|
||||
return fmt.Errorf("malformed JSON or unexpected field: %v", err)
|
||||
}
|
||||
|
||||
if dec.More() {
|
||||
return errors.New("request body contains multiple JSON values")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeStoreError(w http.ResponseWriter, r *http.Request, err error) {
|
||||
switch {
|
||||
case errors.Is(err, store.ErrConflict):
|
||||
WriteError(w, r, http.StatusConflict, "conflict", "resource conflict")
|
||||
case errors.Is(err, store.ErrNotFound):
|
||||
WriteError(w, r, http.StatusNotFound, "not_found", "resource not found or access denied")
|
||||
case errors.Is(err, store.ErrUnauthorized):
|
||||
WriteBearerUnauthorized(w, r, "unauthorized")
|
||||
case errors.Is(err, store.ErrUnavailable):
|
||||
WriteError(w, r, http.StatusServiceUnavailable, "unavailable", "storage dependency is temporarily unavailable")
|
||||
case errors.Is(err, store.ErrBadRequest):
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", err.Error())
|
||||
default:
|
||||
WriteError(w, r, http.StatusInternalServerError, "internal_error", "internal server error")
|
||||
}
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleLiveness(w http.ResponseWriter, r *http.Request) {
|
||||
WriteJSON(w, r, http.StatusOK, model.HealthResponse{Status: "ok"})
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleReadiness(w http.ResponseWriter, r *http.Request) {
|
||||
if err := h.store.Ready(r.Context()); err != nil {
|
||||
WriteError(w, r, http.StatusServiceUnavailable, "unavailable", "database dependency is not ready")
|
||||
return
|
||||
}
|
||||
WriteJSON(w, r, http.StatusOK, model.HealthResponse{Status: "ok"})
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleRegisterUser(w http.ResponseWriter, r *http.Request) {
|
||||
var req model.CreateUserRequest
|
||||
if err := decodeJSON(r, &req); err != nil {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
user, err := h.store.CreateUser(r.Context(), req)
|
||||
if err != nil {
|
||||
writeStoreError(w, r, err)
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, r, http.StatusCreated, user)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleCreateSession(w http.ResponseWriter, r *http.Request) {
|
||||
// Contract: POST /auth/sessions accepts ONLY Authorization: Basic <base64(username:password)>, without JSON body.
|
||||
if r.Body != nil {
|
||||
defer r.Body.Close()
|
||||
var buf [1]byte
|
||||
n, _ := r.Body.Read(buf[:])
|
||||
if n > 0 {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", "POST /api/v1/auth/sessions must not include a request body")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
authHeader := strings.TrimSpace(r.Header.Get("Authorization"))
|
||||
if authHeader == "" {
|
||||
WriteBasicUnauthorized(w, r, "invalid credentials")
|
||||
return
|
||||
}
|
||||
|
||||
parts := strings.SplitN(authHeader, " ", 2)
|
||||
if len(parts) != 2 || !strings.EqualFold(parts[0], "Basic") {
|
||||
WriteBasicUnauthorized(w, r, "invalid credentials")
|
||||
return
|
||||
}
|
||||
|
||||
b64Payload := strings.TrimSpace(parts[1])
|
||||
decoded, err := base64.StdEncoding.DecodeString(b64Payload)
|
||||
if err != nil || !utf8.Valid(decoded) {
|
||||
WriteBasicUnauthorized(w, r, "invalid credentials")
|
||||
return
|
||||
}
|
||||
|
||||
credParts := strings.SplitN(string(decoded), ":", 2)
|
||||
if len(credParts) != 2 {
|
||||
WriteBasicUnauthorized(w, r, "invalid credentials")
|
||||
return
|
||||
}
|
||||
|
||||
username := credParts[0]
|
||||
password := credParts[1]
|
||||
|
||||
sess, err := h.store.AuthenticateAndCreateSession(r.Context(), username, password, h.sessionTTL)
|
||||
if err != nil {
|
||||
if errors.Is(err, store.ErrUnavailable) {
|
||||
WriteError(w, r, http.StatusServiceUnavailable, "unavailable", "storage dependency is temporarily unavailable")
|
||||
return
|
||||
}
|
||||
WriteBasicUnauthorized(w, r, "invalid credentials")
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, r, http.StatusCreated, sess)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleRevokeSession(w http.ResponseWriter, r *http.Request) {
|
||||
rawToken := GetAuthToken(r.Context())
|
||||
if rawToken == "" {
|
||||
WriteBearerUnauthorized(w, r, "unauthenticated")
|
||||
return
|
||||
}
|
||||
|
||||
if err := h.store.RevokeSession(r.Context(), rawToken); err != nil {
|
||||
if errors.Is(err, store.ErrUnavailable) {
|
||||
WriteError(w, r, http.StatusServiceUnavailable, "unavailable", "storage dependency is temporarily unavailable")
|
||||
return
|
||||
}
|
||||
WriteBearerUnauthorized(w, r, "session already revoked or expired")
|
||||
return
|
||||
}
|
||||
|
||||
reqID := GetRequestID(r.Context())
|
||||
w.Header().Set("X-Request-Id", reqID)
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleCurrentUser(w http.ResponseWriter, r *http.Request) {
|
||||
user := GetAuthUser(r.Context())
|
||||
if user == nil {
|
||||
WriteBearerUnauthorized(w, r, "unauthenticated")
|
||||
return
|
||||
}
|
||||
WriteJSON(w, r, http.StatusOK, user)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleCreateChat(w http.ResponseWriter, r *http.Request) {
|
||||
caller := GetAuthUser(r.Context())
|
||||
if caller == nil {
|
||||
WriteBearerUnauthorized(w, r, "unauthenticated")
|
||||
return
|
||||
}
|
||||
|
||||
var req model.CreateChatRequest
|
||||
if err := decodeJSON(r, &req); err != nil {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
chat, err := h.store.CreateChat(r.Context(), caller.ID, req)
|
||||
if err != nil {
|
||||
writeStoreError(w, r, err)
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, r, http.StatusCreated, chat)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleListChats(w http.ResponseWriter, r *http.Request) {
|
||||
caller := GetAuthUser(r.Context())
|
||||
if caller == nil {
|
||||
WriteBearerUnauthorized(w, r, "unauthenticated")
|
||||
return
|
||||
}
|
||||
|
||||
limit := 50
|
||||
limitStr := r.URL.Query().Get("limit")
|
||||
if limitStr != "" {
|
||||
l, err := strconv.Atoi(limitStr)
|
||||
if err != nil || l < 1 || l > 100 {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", "limit must be an integer between 1 and 100")
|
||||
return
|
||||
}
|
||||
limit = l
|
||||
}
|
||||
|
||||
cursor := r.URL.Query().Get("cursor")
|
||||
page, err := h.store.ListChats(r.Context(), caller.ID, limit, cursor)
|
||||
if err != nil {
|
||||
writeStoreError(w, r, err)
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, r, http.StatusOK, page)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleGetChat(w http.ResponseWriter, r *http.Request) {
|
||||
caller := GetAuthUser(r.Context())
|
||||
if caller == nil {
|
||||
WriteBearerUnauthorized(w, r, "unauthenticated")
|
||||
return
|
||||
}
|
||||
|
||||
chatID := r.PathValue("chat_id")
|
||||
if _, err := uuid.Parse(chatID); err != nil {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", "invalid chat id UUID")
|
||||
return
|
||||
}
|
||||
|
||||
chat, err := h.store.GetChat(r.Context(), caller.ID, chatID)
|
||||
if err != nil {
|
||||
writeStoreError(w, r, err)
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, r, http.StatusOK, chat)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleCreateMessage(w http.ResponseWriter, r *http.Request) {
|
||||
caller := GetAuthUser(r.Context())
|
||||
if caller == nil {
|
||||
WriteBearerUnauthorized(w, r, "unauthenticated")
|
||||
return
|
||||
}
|
||||
|
||||
chatID := r.PathValue("chat_id")
|
||||
if _, err := uuid.Parse(chatID); err != nil {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", "invalid chat id UUID")
|
||||
return
|
||||
}
|
||||
|
||||
var req model.CreateMessageRequest
|
||||
if err := decodeJSON(r, &req); err != nil {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
msg, err := h.store.CreateMessage(r.Context(), caller.ID, chatID, req)
|
||||
if err != nil {
|
||||
writeStoreError(w, r, err)
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, r, http.StatusCreated, msg)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleListMessages(w http.ResponseWriter, r *http.Request) {
|
||||
caller := GetAuthUser(r.Context())
|
||||
if caller == nil {
|
||||
WriteBearerUnauthorized(w, r, "unauthenticated")
|
||||
return
|
||||
}
|
||||
|
||||
chatID := r.PathValue("chat_id")
|
||||
if _, err := uuid.Parse(chatID); err != nil {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", "invalid chat id UUID")
|
||||
return
|
||||
}
|
||||
|
||||
limit := 50
|
||||
limitStr := r.URL.Query().Get("limit")
|
||||
if limitStr != "" {
|
||||
l, err := strconv.Atoi(limitStr)
|
||||
if err != nil || l < 1 || l > 100 {
|
||||
WriteError(w, r, http.StatusBadRequest, "invalid_request", "limit must be an integer between 1 and 100")
|
||||
return
|
||||
}
|
||||
limit = l
|
||||
}
|
||||
|
||||
cursor := r.URL.Query().Get("cursor")
|
||||
page, err := h.store.ListMessages(r.Context(), caller.ID, chatID, limit, cursor)
|
||||
if err != nil {
|
||||
writeStoreError(w, r, err)
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, r, http.StatusOK, page)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleInternalStats(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Query().Get("reset") == "1" {
|
||||
h.store.ResetStats()
|
||||
}
|
||||
if mode := r.URL.Query().Get("mode"); mode != "" {
|
||||
h.store.SetCacheMode(mode)
|
||||
}
|
||||
WriteJSON(w, r, http.StatusOK, h.store.Stats(r.Context()))
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleOpenAPISpec(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||
w.Header().Set("X-Request-Id", GetRequestID(r.Context()))
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write(openAPISpec)
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleSwaggerUI(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
w.Header().Set("X-Request-Id", GetRequestID(r.Context()))
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(swaggerUIHTML))
|
||||
}
|
||||
|
||||
func (h *APIHandler) HandleNotFound(w http.ResponseWriter, r *http.Request) {
|
||||
WriteError(w, r, http.StatusNotFound, "not_found", fmt.Sprintf("route %s %s not found", r.Method, r.URL.Path))
|
||||
}
|
||||
|
||||
func RegisterRoutes(st store.Store, sessionTTL int) http.Handler {
|
||||
api := NewAPIHandler(st, sessionTTL)
|
||||
mux := http.NewServeMux()
|
||||
|
||||
// Documentation & schema
|
||||
mux.HandleFunc("GET /docs", api.HandleSwaggerUI)
|
||||
mux.HandleFunc("GET /docs/", api.HandleSwaggerUI)
|
||||
mux.HandleFunc("GET /openapi.json", api.HandleOpenAPISpec)
|
||||
|
||||
// Internal diagnostics for Grade 5 cache & SQL experiments
|
||||
mux.HandleFunc("GET /internal/stats", api.HandleInternalStats)
|
||||
mux.HandleFunc("POST /internal/stats", api.HandleInternalStats)
|
||||
|
||||
// Public routes
|
||||
mux.HandleFunc("GET /health/live", api.HandleLiveness)
|
||||
mux.HandleFunc("GET /health/ready", api.HandleReadiness)
|
||||
mux.HandleFunc("POST /api/v1/users", api.HandleRegisterUser)
|
||||
mux.HandleFunc("POST /api/v1/auth/sessions", api.HandleCreateSession)
|
||||
|
||||
// Protected Bearer routes
|
||||
mux.HandleFunc("DELETE /api/v1/auth/sessions/current", RequireBearerAuth(st, api.HandleRevokeSession))
|
||||
mux.HandleFunc("GET /api/v1/users/me", RequireBearerAuth(st, api.HandleCurrentUser))
|
||||
mux.HandleFunc("POST /api/v1/chats", RequireBearerAuth(st, api.HandleCreateChat))
|
||||
mux.HandleFunc("GET /api/v1/chats", RequireBearerAuth(st, api.HandleListChats))
|
||||
mux.HandleFunc("GET /api/v1/chats/{chat_id}", RequireBearerAuth(st, api.HandleGetChat))
|
||||
mux.HandleFunc("POST /api/v1/chats/{chat_id}/messages", RequireBearerAuth(st, api.HandleCreateMessage))
|
||||
mux.HandleFunc("GET /api/v1/chats/{chat_id}/messages", RequireBearerAuth(st, api.HandleListMessages))
|
||||
|
||||
// Fallback 404 handler
|
||||
mux.HandleFunc("/", api.HandleNotFound)
|
||||
|
||||
var handler http.Handler = mux
|
||||
handler = LoggingMiddleware(handler)
|
||||
handler = RequestIDMiddleware(handler)
|
||||
|
||||
return handler
|
||||
}
|
||||
@@ -0,0 +1,160 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"messenger/internal/model"
|
||||
"messenger/internal/store"
|
||||
)
|
||||
|
||||
type contextKey string
|
||||
|
||||
const (
|
||||
requestIDContextKey contextKey = "request_id"
|
||||
userContextKey contextKey = "user"
|
||||
tokenContextKey contextKey = "token"
|
||||
)
|
||||
|
||||
type loggingResponseWriter struct {
|
||||
http.ResponseWriter
|
||||
statusCode int
|
||||
}
|
||||
|
||||
func (lrw *loggingResponseWriter) WriteHeader(code int) {
|
||||
lrw.statusCode = code
|
||||
lrw.ResponseWriter.WriteHeader(code)
|
||||
}
|
||||
|
||||
func RequestIDMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
reqID := strings.TrimSpace(r.Header.Get("X-Request-Id"))
|
||||
if reqID == "" {
|
||||
reqID = uuid.New().String()
|
||||
}
|
||||
w.Header().Set("X-Request-Id", reqID)
|
||||
ctx := context.WithValue(r.Context(), requestIDContextKey, reqID)
|
||||
next.ServeHTTP(w, r.WithContext(ctx))
|
||||
})
|
||||
}
|
||||
|
||||
func LoggingMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
start := time.Now()
|
||||
lrw := &loggingResponseWriter{ResponseWriter: w, statusCode: http.StatusOK}
|
||||
|
||||
next.ServeHTTP(lrw, r)
|
||||
|
||||
reqID, _ := r.Context().Value(requestIDContextKey).(string)
|
||||
duration := time.Since(start)
|
||||
|
||||
logEntry := map[string]interface{}{
|
||||
"time": time.Now().UTC().Format(time.RFC3339Nano),
|
||||
"method": r.Method,
|
||||
"path": r.URL.Path,
|
||||
"status": lrw.statusCode,
|
||||
"duration_ms": float64(duration.Microseconds()) / 1000.0,
|
||||
"request_id": reqID,
|
||||
"remote_ip": r.RemoteAddr,
|
||||
}
|
||||
logBytes, _ := json.Marshal(logEntry)
|
||||
fmt.Fprintln(os.Stdout, string(logBytes))
|
||||
})
|
||||
}
|
||||
|
||||
func GetRequestID(ctx context.Context) string {
|
||||
if val, ok := ctx.Value(requestIDContextKey).(string); ok && val != "" {
|
||||
return val
|
||||
}
|
||||
return uuid.New().String()
|
||||
}
|
||||
|
||||
func WriteError(w http.ResponseWriter, r *http.Request, status int, code, message string) {
|
||||
reqID := GetRequestID(r.Context())
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Header().Set("X-Request-Id", reqID)
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(model.ErrorResponse{
|
||||
Error: model.ErrorDetail{
|
||||
Code: code,
|
||||
Message: message,
|
||||
RequestID: reqID,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func WriteBasicUnauthorized(w http.ResponseWriter, r *http.Request, message string) {
|
||||
w.Header().Set("WWW-Authenticate", `Basic realm="messenger", charset="UTF-8"`)
|
||||
WriteError(w, r, http.StatusUnauthorized, "unauthorized", message)
|
||||
}
|
||||
|
||||
func WriteBearerUnauthorized(w http.ResponseWriter, r *http.Request, message string) {
|
||||
w.Header().Set("WWW-Authenticate", `Bearer realm="messenger"`)
|
||||
WriteError(w, r, http.StatusUnauthorized, "unauthorized", message)
|
||||
}
|
||||
|
||||
func WriteJSON(w http.ResponseWriter, r *http.Request, status int, data interface{}) {
|
||||
reqID := GetRequestID(r.Context())
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Header().Set("X-Request-Id", reqID)
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(data)
|
||||
}
|
||||
|
||||
func RequireBearerAuth(st store.Store, next http.HandlerFunc) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
// Note: X-User-Id is intentionally ignored in Lab 2+ per contract.
|
||||
authHeader := strings.TrimSpace(r.Header.Get("Authorization"))
|
||||
if authHeader == "" {
|
||||
WriteBearerUnauthorized(w, r, "missing Authorization header")
|
||||
return
|
||||
}
|
||||
|
||||
parts := strings.SplitN(authHeader, " ", 2)
|
||||
if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") {
|
||||
WriteBearerUnauthorized(w, r, "Bearer authentication required")
|
||||
return
|
||||
}
|
||||
|
||||
rawToken := strings.TrimSpace(parts[1])
|
||||
if rawToken == "" || strings.ContainsAny(rawToken, " \t\r\n") {
|
||||
WriteBearerUnauthorized(w, r, "invalid Bearer token")
|
||||
return
|
||||
}
|
||||
|
||||
user, err := st.GetUserByToken(r.Context(), rawToken)
|
||||
if err != nil {
|
||||
if errors.Is(err, store.ErrUnavailable) {
|
||||
WriteError(w, r, http.StatusServiceUnavailable, "unavailable", "storage dependency is temporarily unavailable")
|
||||
return
|
||||
}
|
||||
WriteBearerUnauthorized(w, r, "invalid, expired, or revoked session token")
|
||||
return
|
||||
}
|
||||
|
||||
ctx := context.WithValue(r.Context(), userContextKey, user)
|
||||
ctx = context.WithValue(ctx, tokenContextKey, rawToken)
|
||||
next(w, r.WithContext(ctx))
|
||||
}
|
||||
}
|
||||
|
||||
func GetAuthUser(ctx context.Context) *model.User {
|
||||
if user, ok := ctx.Value(userContextKey).(*model.User); ok {
|
||||
return user
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func GetAuthToken(ctx context.Context) string {
|
||||
if tok, ok := ctx.Value(tokenContextKey).(string); ok {
|
||||
return tok
|
||||
}
|
||||
return ""
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,75 @@
|
||||
package model
|
||||
|
||||
type User struct {
|
||||
ID string `json:"id"`
|
||||
Username string `json:"username"`
|
||||
DisplayName string `json:"display_name"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
}
|
||||
|
||||
type CreateUserRequest struct {
|
||||
Username string `json:"username"`
|
||||
DisplayName string `json:"display_name"`
|
||||
Password string `json:"password"`
|
||||
}
|
||||
|
||||
type Session struct {
|
||||
Token string `json:"token"`
|
||||
TokenType string `json:"token_type"`
|
||||
ExpiresAt string `json:"expires_at"`
|
||||
User User `json:"user"`
|
||||
}
|
||||
|
||||
type Chat struct {
|
||||
ID string `json:"id"`
|
||||
Title string `json:"title"`
|
||||
MemberIDs []string `json:"member_ids"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
}
|
||||
|
||||
type CreateChatRequest struct {
|
||||
Title string `json:"title"`
|
||||
MemberIDs []string `json:"member_ids"`
|
||||
}
|
||||
|
||||
type Message struct {
|
||||
ID string `json:"id"`
|
||||
ChatID string `json:"chat_id"`
|
||||
SenderID string `json:"sender_id"`
|
||||
Text string `json:"text"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
}
|
||||
|
||||
type CreateMessageRequest struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
type ChatPage struct {
|
||||
Items []Chat `json:"items"`
|
||||
NextCursor *string `json:"next_cursor"`
|
||||
}
|
||||
|
||||
type MessagePage struct {
|
||||
Items []Message `json:"items"`
|
||||
NextCursor *string `json:"next_cursor"`
|
||||
}
|
||||
|
||||
type ErrorDetail struct {
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
RequestID string `json:"request_id"`
|
||||
}
|
||||
|
||||
type ErrorResponse struct {
|
||||
Error ErrorDetail `json:"error"`
|
||||
}
|
||||
|
||||
type HealthResponse struct {
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
type CursorData struct {
|
||||
Scope string `json:"s"`
|
||||
Time string `json:"t"`
|
||||
ID string `json:"i"`
|
||||
}
|
||||
@@ -0,0 +1,888 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgconn"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
"github.com/redis/go-redis/v9"
|
||||
"golang.org/x/sync/singleflight"
|
||||
|
||||
"messenger/internal/auth"
|
||||
"messenger/internal/model"
|
||||
)
|
||||
|
||||
type cachedSession struct {
|
||||
User model.User `json:"user"`
|
||||
ExpiresAt time.Time `json:"expires_at"`
|
||||
}
|
||||
|
||||
type PostgresStore struct {
|
||||
pool *pgxpool.Pool
|
||||
rdb *redis.Client
|
||||
queryTimeout time.Duration
|
||||
|
||||
sfGroup singleflight.Group
|
||||
|
||||
mu sync.RWMutex
|
||||
cacheMode string
|
||||
|
||||
sqlQueries atomic.Int64
|
||||
cacheHits atomic.Int64
|
||||
cacheMisses atomic.Int64
|
||||
singleflightShared atomic.Int64
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
DatabaseURL string
|
||||
CacheURL string
|
||||
MaxConns int32
|
||||
MinConns int32
|
||||
QueryTimeout time.Duration
|
||||
CacheMode string
|
||||
}
|
||||
|
||||
func NewPostgresStore(ctx context.Context, cfg Config) (*PostgresStore, error) {
|
||||
poolCfg, err := pgxpool.ParseConfig(cfg.DatabaseURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid database URL: %w", err)
|
||||
}
|
||||
|
||||
if cfg.MaxConns <= 0 {
|
||||
cfg.MaxConns = 25
|
||||
}
|
||||
if cfg.MinConns <= 0 {
|
||||
cfg.MinConns = 4
|
||||
}
|
||||
if cfg.QueryTimeout <= 0 {
|
||||
cfg.QueryTimeout = 5 * time.Second
|
||||
}
|
||||
if cfg.CacheMode == "" {
|
||||
cfg.CacheMode = "write_through_versioned"
|
||||
}
|
||||
|
||||
poolCfg.MaxConns = cfg.MaxConns
|
||||
poolCfg.MinConns = cfg.MinConns
|
||||
poolCfg.MaxConnLifetime = 30 * time.Minute
|
||||
poolCfg.MaxConnIdleTime = 5 * time.Minute
|
||||
poolCfg.HealthCheckPeriod = 15 * time.Second
|
||||
poolCfg.ConnConfig.ConnectTimeout = 3 * time.Second
|
||||
|
||||
pool, err := pgxpool.NewWithConfig(ctx, poolCfg)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create pgx pool: %w", err)
|
||||
}
|
||||
|
||||
var rdb *redis.Client
|
||||
if cfg.CacheURL != "" {
|
||||
opt, err := redis.ParseURL(cfg.CacheURL)
|
||||
if err != nil {
|
||||
pool.Close()
|
||||
return nil, fmt.Errorf("invalid cache URL: %w", err)
|
||||
}
|
||||
opt.DialTimeout = 500 * time.Millisecond
|
||||
opt.ReadTimeout = 500 * time.Millisecond
|
||||
opt.WriteTimeout = 500 * time.Millisecond
|
||||
opt.PoolSize = 30
|
||||
opt.MaxRetries = 1
|
||||
rdb = redis.NewClient(opt)
|
||||
}
|
||||
|
||||
return &PostgresStore{
|
||||
pool: pool,
|
||||
rdb: rdb,
|
||||
queryTimeout: cfg.QueryTimeout,
|
||||
cacheMode: cfg.CacheMode,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) RunMigrations(ctx context.Context, version string, sqlContent string) error {
|
||||
qCtx, cancel := context.WithTimeout(ctx, 15*time.Second)
|
||||
defer cancel()
|
||||
|
||||
tx, err := s.pool.Begin(qCtx)
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin migration tx: %w", err)
|
||||
}
|
||||
defer func() { _ = tx.Rollback(qCtx) }()
|
||||
|
||||
_, err = tx.Exec(qCtx, `
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version VARCHAR(64) PRIMARY KEY,
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
`)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create schema_migrations: %w", err)
|
||||
}
|
||||
|
||||
var alreadyApplied bool
|
||||
err = tx.QueryRow(qCtx, `SELECT EXISTS (SELECT 1 FROM schema_migrations WHERE version = $1)`, version).Scan(&alreadyApplied)
|
||||
if err != nil {
|
||||
return fmt.Errorf("check migration version: %w", err)
|
||||
}
|
||||
|
||||
if !alreadyApplied {
|
||||
if _, err := tx.Exec(qCtx, sqlContent); err != nil {
|
||||
return fmt.Errorf("apply migration %s: %w", version, err)
|
||||
}
|
||||
if _, err := tx.Exec(qCtx, `INSERT INTO schema_migrations (version, applied_at) VALUES ($1, NOW())`, version); err != nil {
|
||||
return fmt.Errorf("record migration %s: %w", version, err)
|
||||
}
|
||||
log.Printf("Migration %s applied successfully", version)
|
||||
} else {
|
||||
log.Printf("Migration %s already applied (idempotent skip)", version)
|
||||
}
|
||||
|
||||
return tx.Commit(qCtx)
|
||||
}
|
||||
|
||||
func (s *PostgresStore) Ready(ctx context.Context) error {
|
||||
qCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
|
||||
defer cancel()
|
||||
|
||||
var ok int
|
||||
if err := s.pool.QueryRow(qCtx, `SELECT 1 FROM schema_migrations LIMIT 1`).Scan(&ok); err != nil {
|
||||
return ErrUnavailable
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) Close() {
|
||||
if s.rdb != nil {
|
||||
_ = s.rdb.Close()
|
||||
}
|
||||
if s.pool != nil {
|
||||
s.pool.Close()
|
||||
}
|
||||
}
|
||||
|
||||
func (s *PostgresStore) getCacheMode() string {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
return s.cacheMode
|
||||
}
|
||||
|
||||
func (s *PostgresStore) SetCacheMode(mode string) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.cacheMode = mode
|
||||
}
|
||||
|
||||
func (s *PostgresStore) ResetStats() {
|
||||
s.sqlQueries.Store(0)
|
||||
s.cacheHits.Store(0)
|
||||
s.cacheMisses.Store(0)
|
||||
s.singleflightShared.Store(0)
|
||||
}
|
||||
|
||||
func (s *PostgresStore) Stats(ctx context.Context) StoreStats {
|
||||
redisOK := false
|
||||
if s.rdb != nil {
|
||||
rCtx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
|
||||
defer cancel()
|
||||
if err := s.rdb.Ping(rCtx).Err(); err == nil {
|
||||
redisOK = true
|
||||
}
|
||||
}
|
||||
return StoreStats{
|
||||
SQLQueries: s.sqlQueries.Load(),
|
||||
CacheHits: s.cacheHits.Load(),
|
||||
CacheMisses: s.cacheMisses.Load(),
|
||||
SingleflightShared: s.singleflightShared.Load(),
|
||||
CacheMode: s.getCacheMode(),
|
||||
RedisConnected: redisOK,
|
||||
}
|
||||
}
|
||||
|
||||
func classifyDBError(err error) error {
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
var pgErr *pgconn.PgError
|
||||
if errors.As(err, &pgErr) {
|
||||
switch pgErr.Code {
|
||||
case "23505": // unique_violation
|
||||
return fmt.Errorf("%w: resource already exists", ErrConflict)
|
||||
case "23503": // foreign_key_violation
|
||||
return fmt.Errorf("%w: referenced entity does not exist", ErrBadRequest)
|
||||
case "23514": // check_violation
|
||||
return fmt.Errorf("%w: check constraint failed", ErrBadRequest)
|
||||
case "22P02": // invalid_text_representation (e.g. bad UUID)
|
||||
return fmt.Errorf("%w: invalid identifier format", ErrBadRequest)
|
||||
}
|
||||
}
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return ErrNotFound
|
||||
}
|
||||
return fmt.Errorf("%w: database operation failed", ErrUnavailable)
|
||||
}
|
||||
|
||||
func (s *PostgresStore) CreateUser(ctx context.Context, req model.CreateUserRequest) (*model.User, error) {
|
||||
if !usernameRegex.MatchString(req.Username) {
|
||||
return nil, fmt.Errorf("%w: username must match ^[a-z0-9_]{3,32}$", ErrBadRequest)
|
||||
}
|
||||
dnLen := utf8.RuneCountInString(req.DisplayName)
|
||||
if dnLen < 1 || dnLen > 100 || strings.TrimSpace(req.DisplayName) == "" {
|
||||
return nil, fmt.Errorf("%w: display_name must be 1..100 characters and not only whitespace", ErrBadRequest)
|
||||
}
|
||||
pwLen := utf8.RuneCountInString(req.Password)
|
||||
if pwLen < 12 || pwLen > 128 || strings.TrimSpace(req.Password) == "" {
|
||||
return nil, fmt.Errorf("%w: password must be 12..128 characters and not only whitespace", ErrBadRequest)
|
||||
}
|
||||
|
||||
pwHash, err := auth.HashPassword(req.Password)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: failed to hash password", ErrUnavailable)
|
||||
}
|
||||
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
|
||||
id := uuid.New().String()
|
||||
now := NowUTC()
|
||||
|
||||
s.sqlQueries.Add(1)
|
||||
_, err = s.pool.Exec(qCtx, `
|
||||
INSERT INTO users (id, username, display_name, password_hash, created_at)
|
||||
VALUES ($1::uuid, $2, $3, $4, $5::timestamptz)
|
||||
`, id, req.Username, req.DisplayName, pwHash, now)
|
||||
if err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
|
||||
return &model.User{
|
||||
ID: id,
|
||||
Username: req.Username,
|
||||
DisplayName: req.DisplayName,
|
||||
CreatedAt: FormatTime(now),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) AuthenticateAndCreateSession(ctx context.Context, username, password string, ttlSeconds int) (*model.Session, error) {
|
||||
if !usernameRegex.MatchString(username) {
|
||||
auth.VerifyDummy(password)
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
pwLen := utf8.RuneCountInString(password)
|
||||
if pwLen < 1 || pwLen > 256 {
|
||||
auth.VerifyDummy(password)
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
|
||||
var user model.User
|
||||
var pwHash string
|
||||
var createdAt time.Time
|
||||
|
||||
s.sqlQueries.Add(1)
|
||||
err := s.pool.QueryRow(qCtx, `
|
||||
SELECT id::text, username, display_name, password_hash, created_at
|
||||
FROM users
|
||||
WHERE username = $1
|
||||
`, username).Scan(&user.ID, &user.Username, &user.DisplayName, &pwHash, &createdAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
auth.VerifyDummy(password)
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
user.CreatedAt = FormatTime(createdAt)
|
||||
|
||||
valid, err := auth.VerifyPassword(password, pwHash)
|
||||
if err != nil || !valid {
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
|
||||
rawToken, tokenHash, err := auth.GenerateSessionToken()
|
||||
if err != nil {
|
||||
return nil, ErrUnavailable
|
||||
}
|
||||
|
||||
if ttlSeconds <= 0 {
|
||||
ttlSeconds = 3600
|
||||
}
|
||||
now := NowUTC()
|
||||
expiresAt := now.Add(time.Duration(ttlSeconds) * time.Second)
|
||||
|
||||
s.sqlQueries.Add(1)
|
||||
_, err = s.pool.Exec(qCtx, `
|
||||
INSERT INTO sessions (token_hash, user_id, expires_at, created_at)
|
||||
VALUES ($1, $2::uuid, $3::timestamptz, $4::timestamptz)
|
||||
`, tokenHash, user.ID, expiresAt, now)
|
||||
if err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
|
||||
s.cacheSession(ctx, tokenHash, user, expiresAt)
|
||||
|
||||
return &model.Session{
|
||||
Token: rawToken,
|
||||
TokenType: "Bearer",
|
||||
ExpiresAt: FormatTime(expiresAt),
|
||||
User: user,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) cacheSession(ctx context.Context, tokenHash string, user model.User, expiresAt time.Time) {
|
||||
if s.rdb == nil {
|
||||
return
|
||||
}
|
||||
remaining := time.Until(expiresAt)
|
||||
if remaining <= 0 {
|
||||
return
|
||||
}
|
||||
payload, err := json.Marshal(cachedSession{
|
||||
User: user,
|
||||
ExpiresAt: expiresAt,
|
||||
})
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
rCtx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
|
||||
defer cancel()
|
||||
key := "sess:" + tokenHash
|
||||
if err := s.rdb.Set(rCtx, key, payload, remaining).Err(); err != nil {
|
||||
log.Printf("WARN: redis session write failed (graceful bypass): %v", err)
|
||||
return
|
||||
}
|
||||
_ = s.rdb.ExpireAt(rCtx, key, expiresAt).Err()
|
||||
}
|
||||
|
||||
func (s *PostgresStore) GetUserByToken(ctx context.Context, rawToken string) (*model.User, error) {
|
||||
if len(rawToken) < 32 || len(rawToken) > 256 {
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
tokenHash := auth.HashToken(rawToken)
|
||||
now := NowUTC()
|
||||
|
||||
// 1. Check Valkey/Redis session cache first
|
||||
if s.rdb != nil {
|
||||
rCtx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
|
||||
val, err := s.rdb.Get(rCtx, "sess:"+tokenHash).Bytes()
|
||||
cancel()
|
||||
if err == nil {
|
||||
var cs cachedSession
|
||||
if jsonErr := json.Unmarshal(val, &cs); jsonErr == nil {
|
||||
if now.Before(cs.ExpiresAt) {
|
||||
s.cacheHits.Add(1)
|
||||
return &cs.User, nil
|
||||
}
|
||||
}
|
||||
} else if !errors.Is(err, redis.Nil) {
|
||||
log.Printf("WARN: redis session read failed (falling back to PostgreSQL): %v", err)
|
||||
}
|
||||
s.cacheMisses.Add(1)
|
||||
}
|
||||
|
||||
// 2. Fallback to PostgreSQL source of truth
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
|
||||
var user model.User
|
||||
var userCreatedAt, expiresAt time.Time
|
||||
|
||||
s.sqlQueries.Add(1)
|
||||
err := s.pool.QueryRow(qCtx, `
|
||||
SELECT u.id::text, u.username, u.display_name, u.created_at, s.expires_at
|
||||
FROM sessions s
|
||||
JOIN users u ON u.id = s.user_id
|
||||
WHERE s.token_hash = $1
|
||||
`, tokenHash).Scan(&user.ID, &user.Username, &user.DisplayName, &userCreatedAt, &expiresAt)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
|
||||
expiresAt = expiresAt.UTC().Truncate(time.Microsecond)
|
||||
if !now.Before(expiresAt) {
|
||||
// Session expired: clean up lazily and reject
|
||||
_, _ = s.pool.Exec(qCtx, `DELETE FROM sessions WHERE token_hash = $1`, tokenHash)
|
||||
return nil, ErrUnauthorized
|
||||
}
|
||||
|
||||
user.CreatedAt = FormatTime(userCreatedAt)
|
||||
// Repopulate Valkey/Redis with original expires_at (never extending TTL!)
|
||||
s.cacheSession(ctx, tokenHash, user, expiresAt)
|
||||
|
||||
return &user, nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) RevokeSession(ctx context.Context, rawToken string) error {
|
||||
if len(rawToken) < 32 || len(rawToken) > 256 {
|
||||
return ErrUnauthorized
|
||||
}
|
||||
tokenHash := auth.HashToken(rawToken)
|
||||
|
||||
if s.rdb != nil {
|
||||
rCtx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
|
||||
if err := s.rdb.Del(rCtx, "sess:"+tokenHash).Err(); err != nil {
|
||||
log.Printf("WARN: redis session delete failed: %v", err)
|
||||
}
|
||||
cancel()
|
||||
}
|
||||
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
|
||||
s.sqlQueries.Add(1)
|
||||
tag, err := s.pool.Exec(qCtx, `
|
||||
DELETE FROM sessions
|
||||
WHERE token_hash = $1 AND expires_at > $2::timestamptz
|
||||
`, tokenHash, NowUTC())
|
||||
if err != nil {
|
||||
return classifyDBError(err)
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return ErrUnauthorized
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) CreateChat(ctx context.Context, callerID string, req model.CreateChatRequest) (*model.Chat, error) {
|
||||
titleLen := utf8.RuneCountInString(req.Title)
|
||||
if titleLen < 1 || titleLen > 100 || strings.TrimSpace(req.Title) == "" {
|
||||
return nil, fmt.Errorf("%w: title must be 1..100 characters and not only whitespace", ErrBadRequest)
|
||||
}
|
||||
if len(req.MemberIDs) < 1 || len(req.MemberIDs) > 100 {
|
||||
return nil, fmt.Errorf("%w: member_ids must contain 1..100 items", ErrBadRequest)
|
||||
}
|
||||
|
||||
seen := make(map[string]struct{}, len(req.MemberIDs))
|
||||
callerIncluded := false
|
||||
for _, mid := range req.MemberIDs {
|
||||
if _, err := uuid.Parse(mid); err != nil {
|
||||
return nil, fmt.Errorf("%w: invalid UUID in member_ids", ErrBadRequest)
|
||||
}
|
||||
if _, exists := seen[mid]; exists {
|
||||
return nil, fmt.Errorf("%w: duplicate user in member_ids", ErrBadRequest)
|
||||
}
|
||||
seen[mid] = struct{}{}
|
||||
if mid == callerID {
|
||||
callerIncluded = true
|
||||
}
|
||||
}
|
||||
if !callerIncluded {
|
||||
return nil, fmt.Errorf("%w: caller must be included in member_ids", ErrBadRequest)
|
||||
}
|
||||
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
|
||||
// Composite atomic transaction: insert chat + insert all chat_members
|
||||
tx, err := s.pool.Begin(qCtx)
|
||||
if err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
defer func() { _ = tx.Rollback(qCtx) }()
|
||||
|
||||
chatID := uuid.New().String()
|
||||
now := NowUTC()
|
||||
|
||||
s.sqlQueries.Add(1)
|
||||
_, err = tx.Exec(qCtx, `
|
||||
INSERT INTO chats (id, title, created_at)
|
||||
VALUES ($1::uuid, $2, $3::timestamptz)
|
||||
`, chatID, req.Title, now)
|
||||
if err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
|
||||
for idx, mid := range req.MemberIDs {
|
||||
s.sqlQueries.Add(1)
|
||||
_, err = tx.Exec(qCtx, `
|
||||
INSERT INTO chat_members (chat_id, user_id, ord)
|
||||
VALUES ($1::uuid, $2::uuid, $3)
|
||||
`, chatID, mid, idx)
|
||||
if err != nil {
|
||||
// If any member does not exist in users, FK constraint 23503 triggers and rolls back the entire chat creation!
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
}
|
||||
|
||||
if err := tx.Commit(qCtx); err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
|
||||
membersCopy := make([]string, len(req.MemberIDs))
|
||||
copy(membersCopy, req.MemberIDs)
|
||||
|
||||
return &model.Chat{
|
||||
ID: chatID,
|
||||
Title: req.Title,
|
||||
MemberIDs: membersCopy,
|
||||
CreatedAt: FormatTime(now),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) ListChats(ctx context.Context, callerID string, limit int, cursorStr string) (*model.ChatPage, error) {
|
||||
if limit < 1 || limit > 100 {
|
||||
return nil, fmt.Errorf("%w: limit must be between 1 and 100", ErrBadRequest)
|
||||
}
|
||||
|
||||
scope := "chats:" + callerID
|
||||
var hasCursor bool
|
||||
var cursorTime time.Time
|
||||
var cursorID string
|
||||
if cursorStr != "" {
|
||||
var err error
|
||||
cursorTime, cursorID, err = DecodeCursor(cursorStr, scope)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
hasCursor = true
|
||||
}
|
||||
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
|
||||
var rows pgx.Rows
|
||||
var err error
|
||||
s.sqlQueries.Add(1)
|
||||
if !hasCursor {
|
||||
rows, err = s.pool.Query(qCtx, `
|
||||
SELECT c.id::text, c.title, c.created_at,
|
||||
array_agg(cm_all.user_id::text ORDER BY cm_all.ord ASC) AS member_ids
|
||||
FROM chats c
|
||||
JOIN chat_members cm_me ON cm_me.chat_id = c.id AND cm_me.user_id = $1::uuid
|
||||
JOIN chat_members cm_all ON cm_all.chat_id = c.id
|
||||
GROUP BY c.id, c.title, c.created_at
|
||||
ORDER BY c.created_at ASC, c.id ASC
|
||||
LIMIT $2
|
||||
`, callerID, limit+1)
|
||||
} else {
|
||||
rows, err = s.pool.Query(qCtx, `
|
||||
SELECT c.id::text, c.title, c.created_at,
|
||||
array_agg(cm_all.user_id::text ORDER BY cm_all.ord ASC) AS member_ids
|
||||
FROM chats c
|
||||
JOIN chat_members cm_me ON cm_me.chat_id = c.id AND cm_me.user_id = $1::uuid
|
||||
JOIN chat_members cm_all ON cm_all.chat_id = c.id
|
||||
WHERE (c.created_at, c.id) > ($2::timestamptz, $3::uuid)
|
||||
GROUP BY c.id, c.title, c.created_at
|
||||
ORDER BY c.created_at ASC, c.id ASC
|
||||
LIMIT $4
|
||||
`, callerID, cursorTime, cursorID, limit+1)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := make([]model.Chat, 0, limit)
|
||||
for rows.Next() {
|
||||
var ch model.Chat
|
||||
var createdAt time.Time
|
||||
if err := rows.Scan(&ch.ID, &ch.Title, &createdAt, &ch.MemberIDs); err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
ch.CreatedAt = FormatTime(createdAt)
|
||||
items = append(items, ch)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
|
||||
var nextCursor *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
last := items[limit-1]
|
||||
c := EncodeCursor(scope, last.CreatedAt, last.ID)
|
||||
nextCursor = &c
|
||||
}
|
||||
|
||||
return &model.ChatPage{
|
||||
Items: items,
|
||||
NextCursor: nextCursor,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) GetChat(ctx context.Context, callerID, chatID string) (*model.Chat, error) {
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
|
||||
var ch model.Chat
|
||||
var createdAt time.Time
|
||||
|
||||
s.sqlQueries.Add(1)
|
||||
err := s.pool.QueryRow(qCtx, `
|
||||
SELECT c.id::text, c.title, c.created_at,
|
||||
array_agg(cm.user_id::text ORDER BY cm.ord ASC) AS member_ids
|
||||
FROM chats c
|
||||
JOIN chat_members cm ON cm.chat_id = c.id
|
||||
WHERE c.id = $1::uuid
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM chat_members WHERE chat_id = c.id AND user_id = $2::uuid
|
||||
)
|
||||
GROUP BY c.id, c.title, c.created_at
|
||||
`, chatID, callerID).Scan(&ch.ID, &ch.Title, &createdAt, &ch.MemberIDs)
|
||||
if err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
ch.CreatedAt = FormatTime(createdAt)
|
||||
return &ch, nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) CreateMessage(ctx context.Context, callerID, chatID string, req model.CreateMessageRequest) (*model.Message, error) {
|
||||
textLen := utf8.RuneCountInString(req.Text)
|
||||
if textLen < 1 || textLen > 4000 || strings.TrimSpace(req.Text) == "" {
|
||||
return nil, fmt.Errorf("%w: text must be 1..4000 characters and not only whitespace", ErrBadRequest)
|
||||
}
|
||||
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
|
||||
msgID := uuid.New().String()
|
||||
now := NowUTC()
|
||||
|
||||
s.sqlQueries.Add(1)
|
||||
tag, err := s.pool.Exec(qCtx, `
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT $1::uuid, $2::uuid, $3::uuid, $4, $5::timestamptz
|
||||
WHERE EXISTS (
|
||||
SELECT 1 FROM chat_members WHERE chat_id = $2::uuid AND user_id = $3::uuid
|
||||
)
|
||||
`, msgID, chatID, callerID, req.Text, now)
|
||||
if err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
|
||||
// Invalidate chat history cache immediately on write (unless in naive_ttl mode for Track 1 experiment)
|
||||
if s.rdb != nil && s.getCacheMode() == "write_through_versioned" {
|
||||
rCtx, rCancel := context.WithTimeout(ctx, 300*time.Millisecond)
|
||||
verKey := "chat_ver:" + chatID
|
||||
if err := s.rdb.Incr(rCtx, verKey).Err(); err != nil {
|
||||
log.Printf("WARN: redis cache version increment failed: %v", err)
|
||||
} else {
|
||||
_ = s.rdb.Expire(rCtx, verKey, 10*time.Minute).Err()
|
||||
}
|
||||
rCancel()
|
||||
}
|
||||
|
||||
return &model.Message{
|
||||
ID: msgID,
|
||||
ChatID: chatID,
|
||||
SenderID: callerID,
|
||||
Text: req.Text,
|
||||
CreatedAt: FormatTime(now),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *PostgresStore) checkChatMembership(ctx context.Context, chatID, callerID, mode string) (bool, error) {
|
||||
memKey := fmt.Sprintf("chat_mem:%s:%s", chatID, callerID)
|
||||
if s.rdb != nil && mode != "disabled" {
|
||||
rCtx, rCancel := context.WithTimeout(ctx, 200*time.Millisecond)
|
||||
val, err := s.rdb.Get(rCtx, memKey).Result()
|
||||
rCancel()
|
||||
if err == nil {
|
||||
return val == "1", nil
|
||||
}
|
||||
}
|
||||
|
||||
queryMembership := func() (bool, error) {
|
||||
qCtx, cancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer cancel()
|
||||
var isMember bool
|
||||
s.sqlQueries.Add(1)
|
||||
err := s.pool.QueryRow(qCtx, `
|
||||
SELECT EXISTS (
|
||||
SELECT 1 FROM chat_members WHERE chat_id = $1::uuid AND user_id = $2::uuid
|
||||
)
|
||||
`, chatID, callerID).Scan(&isMember)
|
||||
if err != nil {
|
||||
return false, classifyDBError(err)
|
||||
}
|
||||
if s.rdb != nil && mode != "disabled" {
|
||||
rCtx, rCancel := context.WithTimeout(ctx, 200*time.Millisecond)
|
||||
strVal := "0"
|
||||
if isMember {
|
||||
strVal = "1"
|
||||
}
|
||||
_ = s.rdb.Set(rCtx, memKey, strVal, 10*time.Minute).Err()
|
||||
rCancel()
|
||||
}
|
||||
return isMember, nil
|
||||
}
|
||||
|
||||
if mode == "write_through_versioned" {
|
||||
res, err, _ := s.sfGroup.Do(memKey, func() (interface{}, error) {
|
||||
return queryMembership()
|
||||
})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return res.(bool), nil
|
||||
}
|
||||
return queryMembership()
|
||||
}
|
||||
|
||||
func (s *PostgresStore) ListMessages(ctx context.Context, callerID, chatID string, limit int, cursorStr string) (*model.MessagePage, error) {
|
||||
if limit < 1 || limit > 100 {
|
||||
return nil, fmt.Errorf("%w: limit must be between 1 and 100", ErrBadRequest)
|
||||
}
|
||||
|
||||
scope := "msgs:" + chatID
|
||||
var hasCursor bool
|
||||
var cursorTime time.Time
|
||||
var cursorID string
|
||||
if cursorStr != "" {
|
||||
var err error
|
||||
cursorTime, cursorID, err = DecodeCursor(cursorStr, scope)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
hasCursor = true
|
||||
}
|
||||
|
||||
mode := s.getCacheMode()
|
||||
|
||||
// 1. Always enforce membership check before returning cached or DB history
|
||||
isMember, err := s.checkChatMembership(ctx, chatID, callerID, mode)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !isMember {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
|
||||
// 2. Determine cache key
|
||||
var cacheKey string
|
||||
if s.rdb != nil && mode != "disabled" {
|
||||
if mode == "naive_ttl" {
|
||||
cacheKey = fmt.Sprintf("msg_page:%s:naive:l%d:c%s", chatID, limit, cursorStr)
|
||||
} else {
|
||||
rCtx, rCancel := context.WithTimeout(ctx, 200*time.Millisecond)
|
||||
ver, err := s.rdb.Get(rCtx, "chat_ver:"+chatID).Int64()
|
||||
rCancel()
|
||||
if err != nil && !errors.Is(err, redis.Nil) {
|
||||
log.Printf("WARN: redis version read failed (bypassing cache): %v", err)
|
||||
cacheKey = ""
|
||||
} else {
|
||||
cacheKey = fmt.Sprintf("msg_page:%s:v%d:l%d:c%s", chatID, ver, limit, cursorStr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Check cache
|
||||
if cacheKey != "" {
|
||||
rCtx, rCancel := context.WithTimeout(ctx, 200*time.Millisecond)
|
||||
cachedBytes, err := s.rdb.Get(rCtx, cacheKey).Bytes()
|
||||
rCancel()
|
||||
if err == nil {
|
||||
var page model.MessagePage
|
||||
if json.Unmarshal(cachedBytes, &page) == nil {
|
||||
s.cacheHits.Add(1)
|
||||
return &page, nil
|
||||
}
|
||||
} else if !errors.Is(err, redis.Nil) {
|
||||
log.Printf("WARN: redis page read failed (falling back to PostgreSQL): %v", err)
|
||||
}
|
||||
s.cacheMisses.Add(1)
|
||||
}
|
||||
|
||||
fetchFromDB := func() (*model.MessagePage, error) {
|
||||
dbCtx, dbCancel := context.WithTimeout(ctx, s.queryTimeout)
|
||||
defer dbCancel()
|
||||
|
||||
var rows pgx.Rows
|
||||
var qErr error
|
||||
s.sqlQueries.Add(1)
|
||||
if !hasCursor {
|
||||
rows, qErr = s.pool.Query(dbCtx, `
|
||||
SELECT id::text, chat_id::text, sender_id::text, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = $1::uuid
|
||||
ORDER BY created_at ASC, id ASC
|
||||
LIMIT $2
|
||||
`, chatID, limit+1)
|
||||
} else {
|
||||
rows, qErr = s.pool.Query(dbCtx, `
|
||||
SELECT id::text, chat_id::text, sender_id::text, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = $1::uuid
|
||||
AND (created_at, id) > ($2::timestamptz, $3::uuid)
|
||||
ORDER BY created_at ASC, id ASC
|
||||
LIMIT $4
|
||||
`, chatID, cursorTime, cursorID, limit+1)
|
||||
}
|
||||
if qErr != nil {
|
||||
return nil, classifyDBError(qErr)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
items := make([]model.Message, 0, limit)
|
||||
for rows.Next() {
|
||||
var m model.Message
|
||||
var createdAt time.Time
|
||||
if err := rows.Scan(&m.ID, &m.ChatID, &m.SenderID, &m.Text, &createdAt); err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
m.CreatedAt = FormatTime(createdAt)
|
||||
items = append(items, m)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, classifyDBError(err)
|
||||
}
|
||||
|
||||
var nextCursor *string
|
||||
if len(items) > limit {
|
||||
items = items[:limit]
|
||||
last := items[limit-1]
|
||||
c := EncodeCursor(scope, last.CreatedAt, last.ID)
|
||||
nextCursor = &c
|
||||
}
|
||||
|
||||
page := &model.MessagePage{
|
||||
Items: items,
|
||||
NextCursor: nextCursor,
|
||||
}
|
||||
|
||||
if cacheKey != "" && s.rdb != nil {
|
||||
if payload, mErr := json.Marshal(page); mErr == nil {
|
||||
rCtx, rCancel := context.WithTimeout(ctx, 200*time.Millisecond)
|
||||
ttl := 30 * time.Second
|
||||
if mode == "naive_ttl" {
|
||||
ttl = 5 * time.Second
|
||||
}
|
||||
_ = s.rdb.Set(rCtx, cacheKey, payload, ttl).Err()
|
||||
rCancel()
|
||||
}
|
||||
}
|
||||
|
||||
return page, nil
|
||||
}
|
||||
|
||||
// 4. Use singleflight in write_through_versioned mode to prevent cache stampede!
|
||||
if mode == "write_through_versioned" && cacheKey != "" {
|
||||
res, err, shared := s.sfGroup.Do(cacheKey, func() (interface{}, error) {
|
||||
return fetchFromDB()
|
||||
})
|
||||
if shared {
|
||||
s.singleflightShared.Add(1)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return res.(*model.MessagePage), nil
|
||||
}
|
||||
|
||||
return fetchFromDB()
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"regexp"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"messenger/internal/model"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrConflict = errors.New("conflict")
|
||||
ErrNotFound = errors.New("not found")
|
||||
ErrBadRequest = errors.New("bad request")
|
||||
ErrUnauthorized = errors.New("unauthorized")
|
||||
ErrUnavailable = errors.New("service unavailable")
|
||||
|
||||
usernameRegex = regexp.MustCompile(`^[a-z0-9_]{3,32}$`)
|
||||
)
|
||||
|
||||
const TimestampFormat = "2006-01-02T15:04:05.000000Z"
|
||||
|
||||
func FormatTime(t time.Time) string {
|
||||
return t.UTC().Truncate(time.Microsecond).Format(TimestampFormat)
|
||||
}
|
||||
|
||||
func NowUTC() time.Time {
|
||||
return time.Now().UTC().Truncate(time.Microsecond)
|
||||
}
|
||||
|
||||
type StoreStats struct {
|
||||
SQLQueries int64 `json:"sql_queries"`
|
||||
CacheHits int64 `json:"cache_hits"`
|
||||
CacheMisses int64 `json:"cache_misses"`
|
||||
SingleflightShared int64 `json:"singleflight_shared"`
|
||||
CacheMode string `json:"cache_mode"`
|
||||
RedisConnected bool `json:"redis_connected"`
|
||||
}
|
||||
|
||||
type Store interface {
|
||||
Ready(ctx context.Context) error
|
||||
CreateUser(ctx context.Context, req model.CreateUserRequest) (*model.User, error)
|
||||
AuthenticateAndCreateSession(ctx context.Context, username, password string, ttlSeconds int) (*model.Session, error)
|
||||
GetUserByToken(ctx context.Context, rawToken string) (*model.User, error)
|
||||
RevokeSession(ctx context.Context, rawToken string) error
|
||||
CreateChat(ctx context.Context, callerID string, req model.CreateChatRequest) (*model.Chat, error)
|
||||
ListChats(ctx context.Context, callerID string, limit int, cursorStr string) (*model.ChatPage, error)
|
||||
GetChat(ctx context.Context, callerID, chatID string) (*model.Chat, error)
|
||||
CreateMessage(ctx context.Context, callerID, chatID string, req model.CreateMessageRequest) (*model.Message, error)
|
||||
ListMessages(ctx context.Context, callerID, chatID string, limit int, cursorStr string) (*model.MessagePage, error)
|
||||
Stats(ctx context.Context) StoreStats
|
||||
ResetStats()
|
||||
SetCacheMode(mode string)
|
||||
Close()
|
||||
}
|
||||
|
||||
func EncodeCursor(scope, timestamp, id string) string {
|
||||
data := model.CursorData{
|
||||
Scope: scope,
|
||||
Time: timestamp,
|
||||
ID: id,
|
||||
}
|
||||
b, _ := json.Marshal(data)
|
||||
return base64.RawURLEncoding.EncodeToString(b)
|
||||
}
|
||||
|
||||
func DecodeCursor(cursorStr, expectedScope string) (time.Time, string, error) {
|
||||
b, err := base64.RawURLEncoding.DecodeString(cursorStr)
|
||||
if err != nil {
|
||||
b, err = base64.URLEncoding.DecodeString(cursorStr)
|
||||
if err != nil {
|
||||
b, err = base64.StdEncoding.DecodeString(cursorStr)
|
||||
if err != nil {
|
||||
return time.Time{}, "", ErrBadRequest
|
||||
}
|
||||
}
|
||||
}
|
||||
var data model.CursorData
|
||||
if err := json.Unmarshal(b, &data); err != nil {
|
||||
return time.Time{}, "", ErrBadRequest
|
||||
}
|
||||
if data.Scope != expectedScope || data.Time == "" || data.ID == "" {
|
||||
return time.Time{}, "", ErrBadRequest
|
||||
}
|
||||
if _, err := uuid.Parse(data.ID); err != nil {
|
||||
return time.Time{}, "", ErrBadRequest
|
||||
}
|
||||
t, err := time.Parse(time.RFC3339Nano, data.Time)
|
||||
if err != nil {
|
||||
return time.Time{}, "", ErrBadRequest
|
||||
}
|
||||
return t.UTC().Truncate(time.Microsecond), data.ID, nil
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
-- Migration 001: Initial schema for Lab 2 (PostgreSQL, Sessions, Chats, Messages)
|
||||
-- Idempotent DDL with primary/foreign keys, constraints, and keyset pagination indexes.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id UUID PRIMARY KEY,
|
||||
username VARCHAR(32) NOT NULL UNIQUE
|
||||
CONSTRAINT chk_users_username CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||
display_name VARCHAR(100) NOT NULL
|
||||
CONSTRAINT chk_users_display_name CHECK (char_length(btrim(display_name)) >= 1 AND char_length(display_name) <= 100),
|
||||
password_hash TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
token_hash CHAR(64) PRIMARY KEY,
|
||||
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
expires_at TIMESTAMPTZ NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_user_id ON sessions (user_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_expires_at ON sessions (expires_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS chats (
|
||||
id UUID PRIMARY KEY,
|
||||
title VARCHAR(100) NOT NULL
|
||||
CONSTRAINT chk_chats_title CHECK (char_length(btrim(title)) >= 1 AND char_length(title) <= 100),
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_chats_created_id ON chats (created_at ASC, id ASC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS chat_members (
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
ord INT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_chat_members_user_chat ON chat_members (user_id, chat_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id UUID PRIMARY KEY,
|
||||
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
sender_id UUID NOT NULL REFERENCES users(id) ON DELETE RESTRICT,
|
||||
text TEXT NOT NULL
|
||||
CONSTRAINT chk_messages_text CHECK (char_length(btrim(text)) >= 1 AND char_length(text) <= 4000),
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
-- Composite B-Tree index supporting keyset pagination:
|
||||
-- WHERE chat_id = $1 AND (created_at, id) > ($2, $3) ORDER BY created_at ASC, id ASC LIMIT $4
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_chat_created_id ON messages (chat_id, created_at ASC, id ASC);
|
||||
@@ -0,0 +1,8 @@
|
||||
@echo off
|
||||
cd /d "%~dp0\.."
|
||||
if exist .env (
|
||||
docker compose --env-file .env restart postgres redis api
|
||||
) else (
|
||||
docker compose restart postgres redis api
|
||||
)
|
||||
exit /b %ERRORLEVEL%
|
||||
Executable
+12
@@ -0,0 +1,12 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
|
||||
ROOT_DIR="$(dirname -- "$SCRIPT_DIR")"
|
||||
cd "$ROOT_DIR"
|
||||
|
||||
if [ -f .env ]; then
|
||||
docker compose --env-file .env restart postgres redis api
|
||||
else
|
||||
docker compose restart postgres redis api
|
||||
fi
|
||||
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Helper script to populate synthetic demo data for Lab 2 defense in Swagger UI."""
|
||||
import base64
|
||||
import json
|
||||
import sys
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
BASE_URL = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:8080"
|
||||
|
||||
|
||||
def req(method, path, body=None, headers=None):
|
||||
h = {"Accept": "application/json"}
|
||||
data = None
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
h["Content-Type"] = "application/json"
|
||||
if headers:
|
||||
h.update(headers)
|
||||
r = urllib.request.Request(BASE_URL + path, data=data, headers=h, method=method)
|
||||
with urllib.request.urlopen(r, timeout=10) as resp:
|
||||
raw = resp.read()
|
||||
return json.loads(raw.decode("utf-8")) if raw else None
|
||||
|
||||
|
||||
def main():
|
||||
suffix = uuid.uuid4().hex[:6]
|
||||
alice_username = f"alice_{suffix}"
|
||||
bob_username = f"bob_{suffix}"
|
||||
password = "DemoPassword-2026!"
|
||||
|
||||
alice = req("POST", "/api/v1/users", {
|
||||
"username": alice_username,
|
||||
"display_name": "Alice Zamorskaya",
|
||||
"password": password,
|
||||
})
|
||||
bob = req("POST", "/api/v1/users", {
|
||||
"username": bob_username,
|
||||
"display_name": "Bob Bobrov",
|
||||
"password": password,
|
||||
})
|
||||
|
||||
alice_basic = base64.b64encode(f"{alice_username}:{password}".encode("utf-8")).decode("ascii")
|
||||
bob_basic = base64.b64encode(f"{bob_username}:{password}".encode("utf-8")).decode("ascii")
|
||||
|
||||
alice_sess = req("POST", "/api/v1/auth/sessions", headers={"Authorization": f"Basic {alice_basic}"})
|
||||
bob_sess = req("POST", "/api/v1/auth/sessions", headers={"Authorization": f"Basic {bob_basic}"})
|
||||
|
||||
alice_bearer = {"Authorization": f"Bearer {alice_sess['token']}"}
|
||||
bob_bearer = {"Authorization": f"Bearer {bob_sess['token']}"}
|
||||
|
||||
chat = req("POST", "/api/v1/chats", {
|
||||
"title": "Lab 2 Defense Chat",
|
||||
"member_ids": [alice["id"], bob["id"]],
|
||||
}, headers=alice_bearer)
|
||||
|
||||
req("POST", f"/api/v1/chats/{chat['id']}/messages", {
|
||||
"text": "Hello Bob! Stored in PostgreSQL 18 and cached in Redis 7.4.",
|
||||
}, headers=alice_bearer)
|
||||
req("POST", f"/api/v1/chats/{chat['id']}/messages", {
|
||||
"text": "Hi Alice! Authenticated via Bearer session token.",
|
||||
}, headers=bob_bearer)
|
||||
|
||||
print("=== LAB 2 DEMO DATA CREATED ===")
|
||||
print(f"Swagger UI: {BASE_URL}/docs")
|
||||
print(f"Password: {password}")
|
||||
print(f"Alice Username: {alice_username}")
|
||||
print(f"Alice User ID: {alice['id']}")
|
||||
print(f"Alice Token: {alice_sess['token']}")
|
||||
print(f"Bob Username: {bob_username}")
|
||||
print(f"Bob User ID: {bob['id']}")
|
||||
print(f"Bob Token: {bob_sess['token']}")
|
||||
print(f"Chat ID: {chat['id']}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,299 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Grade 4 automated verification suite for Lab 2:
|
||||
- Password KDF & colon-in-password support
|
||||
- Strict Basic/Bearer separation & WWW-Authenticate headers
|
||||
- Concurrent registration race on identical username (20 threads -> 1x 201, 19x 409)
|
||||
- Atomic transaction rollback on composite chat creation with unknown member_id
|
||||
- Session TTL expiration with short SESSION_TTL_SECONDS=2
|
||||
- Graceful cache bypass when Valkey/Redis is down & bounded 503 when PostgreSQL is down
|
||||
"""
|
||||
import base64
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
BASE_URL = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:8080"
|
||||
|
||||
|
||||
def call_api(method, path, body=None, headers=None, raw_body=None, timeout=5):
|
||||
h = {"Accept": "application/json"}
|
||||
data = raw_body
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
h["Content-Type"] = "application/json"
|
||||
if headers:
|
||||
h.update(headers)
|
||||
req = urllib.request.Request(BASE_URL + path, data=data, headers=h, method=method)
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=timeout)
|
||||
except urllib.error.HTTPError as err:
|
||||
resp = err
|
||||
with resp:
|
||||
status = resp.code
|
||||
resp_headers = dict(resp.headers.items())
|
||||
raw = resp.read()
|
||||
payload = json.loads(raw.decode("utf-8")) if raw else None
|
||||
return status, resp_headers, payload
|
||||
|
||||
|
||||
def assert_true(cond, msg):
|
||||
if not cond:
|
||||
raise AssertionError(msg)
|
||||
|
||||
|
||||
def run_cmd(cmd):
|
||||
res = subprocess.run(cmd, shell=True, capture_output=True, text=True)
|
||||
if res.returncode != 0:
|
||||
raise RuntimeError(f"Command failed ({cmd}): {res.stderr}")
|
||||
return res.stdout.strip()
|
||||
|
||||
|
||||
def wait_ready(timeout_sec=30):
|
||||
deadline = time.monotonic() + timeout_sec
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
st, _, _ = call_api("GET", "/health/ready", timeout=2)
|
||||
if st == 200:
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
time.sleep(0.5)
|
||||
raise RuntimeError("API did not become ready in time")
|
||||
|
||||
|
||||
def main():
|
||||
results = {}
|
||||
wait_ready()
|
||||
|
||||
# 1. Negative validation & colon in password
|
||||
print("[1/6] Testing negative inputs, colon-in-password, and auth headers...")
|
||||
u_name = f"u_col_{uuid.uuid4().hex[:8]}"
|
||||
colon_pw = "My:Secret:Pass:2026!"
|
||||
|
||||
# Too short password -> 400
|
||||
st, _, _ = call_api("POST", "/api/v1/users", {
|
||||
"username": u_name,
|
||||
"display_name": "Test User",
|
||||
"password": "short",
|
||||
})
|
||||
assert_true(st == 400, f"Expected 400 for short password, got {st}")
|
||||
|
||||
# Valid registration with colons in password -> 201
|
||||
st, _, user = call_api("POST", "/api/v1/users", {
|
||||
"username": u_name,
|
||||
"display_name": "Colon Password User",
|
||||
"password": colon_pw,
|
||||
})
|
||||
assert_true(st == 201 and "password" not in user, f"Expected 201 User without password field, got {st}")
|
||||
|
||||
# Verify password is stored as Argon2id hash in PostgreSQL
|
||||
pg_hash = run_cmd(
|
||||
f'docker exec mtusi-lab2-postgres psql -U messenger -d messenger -tAc "SELECT password_hash FROM users WHERE username=\'{u_name}\';"'
|
||||
)
|
||||
assert_true(pg_hash.startswith("$argon2id$v=19$"), f"Expected Argon2id hash in DB, got: {pg_hash[:20]}")
|
||||
|
||||
# POST /api/v1/auth/sessions with JSON body -> 400
|
||||
cred = base64.b64encode(f"{u_name}:{colon_pw}".encode("utf-8")).decode("ascii")
|
||||
st, _, _ = call_api(
|
||||
"POST", "/api/v1/auth/sessions",
|
||||
body={"unexpected": "body"},
|
||||
headers={"Authorization": f"Basic {cred}"},
|
||||
)
|
||||
assert_true(st == 400, f"Expected 400 when body is sent to POST /auth/sessions, got {st}")
|
||||
|
||||
# Login with colon password -> 201
|
||||
st, _, sess = call_api(
|
||||
"POST", "/api/v1/auth/sessions",
|
||||
headers={"Authorization": f"Basic {cred}"},
|
||||
)
|
||||
assert_true(st == 201 and len(sess["token"]) >= 64, f"Expected 201 Session with 256-bit token, got {st}")
|
||||
token = sess["token"]
|
||||
results["argon2id_and_auth_semantics"] = {
|
||||
"status": "PASS",
|
||||
"kdf_prefix": "$argon2id$v=19$m=19456,t=1,p=2$",
|
||||
"token_length_hex": len(token),
|
||||
}
|
||||
|
||||
# 2. Concurrent registration race (20 parallel workers with identical username)
|
||||
print("[2/6] Testing concurrent registration race on identical username (20 threads)...")
|
||||
race_username = f"race_{uuid.uuid4().hex[:10]}"
|
||||
|
||||
def register_worker(_):
|
||||
s, _, _ = call_api("POST", "/api/v1/users", {
|
||||
"username": race_username,
|
||||
"display_name": "Race Condition Tester",
|
||||
"password": "RacePassword-12345!",
|
||||
})
|
||||
return s
|
||||
|
||||
with ThreadPoolExecutor(max_workers=20) as pool:
|
||||
codes = list(pool.map(register_worker, range(20)))
|
||||
|
||||
created_count = codes.count(201)
|
||||
conflict_count = codes.count(409)
|
||||
assert_true(
|
||||
created_count == 1 and conflict_count == 19,
|
||||
f"Expected 1x 201 and 19x 409, got {codes}",
|
||||
)
|
||||
results["concurrent_registration_race"] = {
|
||||
"threads": 20,
|
||||
"status_201": created_count,
|
||||
"status_409": conflict_count,
|
||||
"status": "PASS",
|
||||
}
|
||||
|
||||
# 3. Composite operation rollback on POST /api/v1/chats
|
||||
print("[3/6] Testing atomic transaction rollback on composite chat creation...")
|
||||
chats_before = int(run_cmd(
|
||||
'docker exec mtusi-lab2-postgres psql -U messenger -d messenger -tAc "SELECT count(*) FROM chats;"'
|
||||
))
|
||||
members_before = int(run_cmd(
|
||||
'docker exec mtusi-lab2-postgres psql -U messenger -d messenger -tAc "SELECT count(*) FROM chat_members;"'
|
||||
))
|
||||
non_existent_user = str(uuid.uuid4())
|
||||
st, _, _ = call_api(
|
||||
"POST", "/api/v1/chats",
|
||||
body={
|
||||
"title": "Should Rollback Atomically",
|
||||
"member_ids": [user["id"], non_existent_user],
|
||||
},
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
)
|
||||
assert_true(st == 400, f"Expected 400 for unknown member_id in chat creation, got {st}")
|
||||
chats_after = int(run_cmd(
|
||||
'docker exec mtusi-lab2-postgres psql -U messenger -d messenger -tAc "SELECT count(*) FROM chats;"'
|
||||
))
|
||||
members_after = int(run_cmd(
|
||||
'docker exec mtusi-lab2-postgres psql -U messenger -d messenger -tAc "SELECT count(*) FROM chat_members;"'
|
||||
))
|
||||
assert_true(
|
||||
chats_before == chats_after and members_before == members_after,
|
||||
f"Orphan rows left after failed chat creation! chats: {chats_before}->{chats_after}, members: {members_before}->{members_after}",
|
||||
)
|
||||
results["composite_chat_rollback"] = {
|
||||
"chats_before": chats_before,
|
||||
"chats_after": chats_after,
|
||||
"members_before": members_before,
|
||||
"members_after": members_after,
|
||||
"status": "PASS",
|
||||
}
|
||||
|
||||
# 4. Cache outage test (Redis stopped -> graceful fallback to PostgreSQL, anonymous still 401)
|
||||
print("[4/6] Testing Redis outage (graceful cache bypass & no anonymous access)...")
|
||||
run_cmd("docker stop mtusi-lab2-redis")
|
||||
try:
|
||||
st_auth, _, me_data = call_api(
|
||||
"GET", "/api/v1/users/me",
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
)
|
||||
assert_true(st_auth == 200 and me_data["id"] == user["id"], f"Expected 200 fallback when Redis is down, got {st_auth}")
|
||||
|
||||
st_anon, hdr_anon, _ = call_api(
|
||||
"GET", "/api/v1/users/me",
|
||||
headers={"Authorization": "Bearer invalid_token_value_12345678901234567890"},
|
||||
)
|
||||
assert_true(st_anon == 401, f"Expected 401 for invalid token when Redis is down, got {st_anon}")
|
||||
finally:
|
||||
run_cmd("docker start mtusi-lab2-redis")
|
||||
time.sleep(1)
|
||||
|
||||
results["redis_outage_resilience"] = {
|
||||
"authenticated_fallback_status": 200,
|
||||
"invalid_token_status": 401,
|
||||
"status": "PASS",
|
||||
}
|
||||
|
||||
# 5. Database outage test (PostgreSQL stopped -> bounded 503 unavailable without SQL/stacktrace leak)
|
||||
print("[5/6] Testing PostgreSQL outage (bounded 503 error & readiness check)...")
|
||||
run_cmd("docker stop mtusi-lab2-postgres")
|
||||
try:
|
||||
st_ready, _, err_ready = call_api("GET", "/health/ready")
|
||||
assert_true(
|
||||
st_ready == 503 and err_ready["error"]["code"] == "unavailable",
|
||||
f"Expected 503 unavailable on /health/ready when DB is down, got {st_ready}",
|
||||
)
|
||||
st_reg, _, err_reg = call_api("POST", "/api/v1/users", {
|
||||
"username": f"u_dbdown_{uuid.uuid4().hex[:6]}",
|
||||
"display_name": "DB Down User",
|
||||
"password": "ValidPassword-12345!",
|
||||
})
|
||||
assert_true(
|
||||
st_reg == 503 and err_reg["error"]["code"] == "unavailable",
|
||||
f"Expected 503 unavailable on POST /users when DB is down, got {st_reg}",
|
||||
)
|
||||
msg_lower = err_reg["error"]["message"].lower()
|
||||
assert_true(
|
||||
"select" not in msg_lower and "pgx" not in msg_lower and "panic" not in msg_lower,
|
||||
"Error message leaked internal details",
|
||||
)
|
||||
finally:
|
||||
run_cmd("docker start mtusi-lab2-postgres")
|
||||
wait_ready()
|
||||
|
||||
results["postgres_outage_resilience"] = {
|
||||
"health_ready_status": 503,
|
||||
"write_endpoint_status": 503,
|
||||
"error_code": "unavailable",
|
||||
"leaked_internals": False,
|
||||
"status": "PASS",
|
||||
}
|
||||
|
||||
# 6. Short SESSION_TTL_SECONDS=2 expiry test
|
||||
print("[6/6] Testing short SESSION_TTL_SECONDS=2 expiration...")
|
||||
import os
|
||||
with open(".env", "r", encoding="utf-8") as f:
|
||||
base_env = f.read()
|
||||
short_env = base_env.replace("SESSION_TTL_SECONDS=3600", "SESSION_TTL_SECONDS=2")
|
||||
with open(".env.short", "w", encoding="utf-8") as f:
|
||||
f.write(short_env)
|
||||
run_cmd("docker compose --env-file .env.short up -d --force-recreate --no-deps api")
|
||||
try:
|
||||
wait_ready()
|
||||
st, _, short_sess = call_api(
|
||||
"POST", "/api/v1/auth/sessions",
|
||||
headers={"Authorization": f"Basic {cred}"},
|
||||
)
|
||||
assert_true(st == 201, f"Expected 201 on login with 2s TTL, got {st}")
|
||||
short_token = short_sess["token"]
|
||||
|
||||
st_before, _, _ = call_api(
|
||||
"GET", "/api/v1/users/me",
|
||||
headers={"Authorization": f"Bearer {short_token}"},
|
||||
)
|
||||
assert_true(st_before == 200, f"Expected 200 immediately after login, got {st_before}")
|
||||
|
||||
time.sleep(2.5)
|
||||
st_after, _, err_after = call_api(
|
||||
"GET", "/api/v1/users/me",
|
||||
headers={"Authorization": f"Bearer {short_token}"},
|
||||
)
|
||||
assert_true(
|
||||
st_after == 401 and err_after["error"]["code"] == "unauthorized",
|
||||
f"Expected 401 after 2.5s sleep with SESSION_TTL_SECONDS=2, got {st_after}",
|
||||
)
|
||||
finally:
|
||||
if os.path.exists(".env.short"):
|
||||
os.remove(".env.short")
|
||||
run_cmd("docker compose --env-file .env up -d --force-recreate --no-deps api")
|
||||
wait_ready()
|
||||
|
||||
results["short_ttl_expiry"] = {
|
||||
"session_ttl_seconds": 2,
|
||||
"status_immediate": 200,
|
||||
"status_after_2_5s": 401,
|
||||
"status": "PASS",
|
||||
}
|
||||
|
||||
with open("evidence/grade4_checks.json", "w", encoding="utf-8") as f:
|
||||
json.dump(results, f, indent=2)
|
||||
|
||||
print("PASS: All Grade 4 verification scenarios succeeded! Evidence saved to evidence/grade4_checks.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,194 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Grade 5 Track 1 Experiment: Cache Consistency, Stampede Protection, Read-After-Write, and Cache Restart Recovery."""
|
||||
import base64
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
BASE_URL = sys.argv[1] if len(sys.argv) > 1 else "http://localhost:8080"
|
||||
|
||||
|
||||
def call_api(method, path, body=None, headers=None):
|
||||
h = {"Accept": "application/json"}
|
||||
data = None
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
h["Content-Type"] = "application/json"
|
||||
if headers:
|
||||
h.update(headers)
|
||||
req = urllib.request.Request(BASE_URL + path, data=data, headers=h, method=method)
|
||||
t0 = time.perf_counter()
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
raw = resp.read()
|
||||
elapsed_ms = (time.perf_counter() - t0) * 1000.0
|
||||
payload = json.loads(raw.decode("utf-8")) if raw else None
|
||||
return resp.code, payload, elapsed_ms
|
||||
|
||||
|
||||
def run_cmd(cmd):
|
||||
res = subprocess.run(cmd, shell=True, capture_output=True, text=True)
|
||||
if res.returncode != 0:
|
||||
raise RuntimeError(f"Command failed ({cmd}): {res.stderr}")
|
||||
return res.stdout.strip()
|
||||
|
||||
|
||||
def percentile(sorted_vals, pct):
|
||||
if not sorted_vals:
|
||||
return 0.0
|
||||
idx = min(len(sorted_vals) - 1, max(0, int(round((pct / 100.0) * (len(sorted_vals) - 1)))))
|
||||
return round(sorted_vals[idx], 3)
|
||||
|
||||
|
||||
def main():
|
||||
u_name = f"track1_{uuid.uuid4().hex[:8]}"
|
||||
pw = "Track1Password-2026!"
|
||||
_, user, _ = call_api("POST", "/api/v1/users", {
|
||||
"username": u_name,
|
||||
"display_name": "Track 1 Researcher",
|
||||
"password": pw,
|
||||
})
|
||||
cred = base64.b64encode(f"{u_name}:{pw}".encode("utf-8")).decode("ascii")
|
||||
_, sess, _ = call_api("POST", "/api/v1/auth/sessions", headers={"Authorization": f"Basic {cred}"})
|
||||
auth_hdr = {"Authorization": f"Bearer {sess['token']}"}
|
||||
|
||||
_, chat, _ = call_api("POST", "/api/v1/chats", {
|
||||
"title": "Track 1 Cache Consistency Chat",
|
||||
"member_ids": [user["id"]],
|
||||
}, headers=auth_hdr)
|
||||
chat_id = chat["id"]
|
||||
|
||||
# Seed 20 messages
|
||||
for i in range(20):
|
||||
call_api("POST", f"/api/v1/chats/{chat_id}/messages", {
|
||||
"text": f"Seed message #{i + 1}",
|
||||
}, headers=auth_hdr)
|
||||
|
||||
# Experiment 1: Stale Read in naive_ttl vs Strict Read-After-Write in write_through_versioned
|
||||
print("[Track 1 - Step 1] Comparing stale reads (naive_ttl) vs strict read-after-write (write_through_versioned)...")
|
||||
call_api("POST", "/internal/stats?mode=naive_ttl&reset=1")
|
||||
run_cmd("docker exec mtusi-lab2-redis redis-cli FLUSHDB")
|
||||
|
||||
# Warm up cache in naive_ttl
|
||||
_, page_warm, _ = call_api("GET", f"/api/v1/chats/{chat_id}/messages?limit=50", headers=auth_hdr)
|
||||
# Write a new message
|
||||
_, new_msg_naive, _ = call_api("POST", f"/api/v1/chats/{chat_id}/messages", {
|
||||
"text": "Fresh write during naive_ttl",
|
||||
}, headers=auth_hdr)
|
||||
# Immediate read returns stale cached page without new_msg_naive
|
||||
_, page_after_naive, _ = call_api("GET", f"/api/v1/chats/{chat_id}/messages?limit=50", headers=auth_hdr)
|
||||
naive_saw_new = any(m["id"] == new_msg_naive["id"] for m in page_after_naive["items"])
|
||||
|
||||
# Now switch to write_through_versioned and verify 20 consecutive write -> immediate read cycles
|
||||
call_api("POST", "/internal/stats?mode=write_through_versioned&reset=1")
|
||||
run_cmd("docker exec mtusi-lab2-redis redis-cli FLUSHDB")
|
||||
|
||||
raw_violations = 0
|
||||
for i in range(20):
|
||||
_, page_before, _ = call_api("GET", f"/api/v1/chats/{chat_id}/messages?limit=100", headers=auth_hdr)
|
||||
_, sent, _ = call_api("POST", f"/api/v1/chats/{chat_id}/messages", {
|
||||
"text": f"Read-after-write verification #{i + 1}",
|
||||
}, headers=auth_hdr)
|
||||
_, page_imm, _ = call_api("GET", f"/api/v1/chats/{chat_id}/messages?limit=100", headers=auth_hdr)
|
||||
if not any(m["id"] == sent["id"] for m in page_imm["items"]):
|
||||
raw_violations += 1
|
||||
|
||||
if raw_violations != 0:
|
||||
raise AssertionError(f"Read-after-write violated {raw_violations} times!")
|
||||
|
||||
# Experiment 2: Cache Stampede on Cold Cache (40 concurrent readers)
|
||||
print("[Track 1 - Step 2] Measuring cache stampede under 40 concurrent readers (naive_ttl vs singleflight)...")
|
||||
|
||||
def read_worker(_):
|
||||
_, _, ms = call_api("GET", f"/api/v1/chats/{chat_id}/messages?limit=50", headers=auth_hdr)
|
||||
return ms
|
||||
|
||||
# Warm session & membership first, then delete only message page keys to simulate hot chat page expiration
|
||||
call_api("POST", "/internal/stats?mode=naive_ttl&reset=1")
|
||||
call_api("GET", f"/api/v1/chats/{chat_id}/messages?limit=50", headers=auth_hdr)
|
||||
run_cmd(f'docker exec mtusi-lab2-redis redis-cli DEL "msg_page:{chat_id}:naive:l50:c"')
|
||||
call_api("POST", "/internal/stats?mode=naive_ttl&reset=1")
|
||||
|
||||
with ThreadPoolExecutor(max_workers=40) as pool:
|
||||
naive_latencies = sorted(list(pool.map(read_worker, range(40))))
|
||||
_, stats_naive, _ = call_api("GET", "/internal/stats")
|
||||
|
||||
# Protected mode (write_through_versioned with singleflight)
|
||||
call_api("POST", "/internal/stats?mode=write_through_versioned&reset=1")
|
||||
call_api("GET", f"/api/v1/chats/{chat_id}/messages?limit=50", headers=auth_hdr)
|
||||
# Delete the cached page key so all 40 readers miss simultaneously
|
||||
keys_out = run_cmd(f'docker exec mtusi-lab2-redis redis-cli KEYS "msg_page:{chat_id}:*"')
|
||||
for k in keys_out.splitlines():
|
||||
if k.strip():
|
||||
run_cmd(f'docker exec mtusi-lab2-redis redis-cli DEL "{k.strip()}"')
|
||||
call_api("POST", "/internal/stats?mode=write_through_versioned&reset=1")
|
||||
|
||||
with ThreadPoolExecutor(max_workers=40) as pool:
|
||||
protected_latencies = sorted(list(pool.map(read_worker, range(40))))
|
||||
_, stats_protected, _ = call_api("GET", "/internal/stats")
|
||||
|
||||
# Warm cache hit latency (40 requests hitting warm Redis cache -> 0 SQL queries)
|
||||
call_api("POST", "/internal/stats?mode=write_through_versioned&reset=1")
|
||||
warm_latencies = sorted([read_worker(i) for i in range(40)])
|
||||
_, stats_warm, _ = call_api("GET", "/internal/stats")
|
||||
|
||||
# Experiment 3: Redis Restart Recovery & Session Preservation
|
||||
print("[Track 1 - Step 3] Testing Redis container restart and cache recovery...")
|
||||
run_cmd("docker restart mtusi-lab2-redis")
|
||||
time.sleep(2)
|
||||
st_code, page_recovered, _ = call_api("GET", f"/api/v1/chats/{chat_id}/messages?limit=100", headers=auth_hdr)
|
||||
if st_code != 200 or len(page_recovered["items"]) < 40:
|
||||
raise AssertionError("Failed to recover history after Redis restart")
|
||||
|
||||
summary = {
|
||||
"track": "Track 1: Cache Consistency, Stampede Protection, Read-After-Write, and Cache Restart",
|
||||
"read_after_write_experiment": {
|
||||
"naive_ttl_saw_immediate_write": naive_saw_new,
|
||||
"naive_ttl_stale_read_observed": not naive_saw_new,
|
||||
"write_through_versioned_cycles": 20,
|
||||
"write_through_versioned_stale_reads": 0,
|
||||
},
|
||||
"cache_stampede_40_concurrent_readers": {
|
||||
"unprotected_naive": {
|
||||
"concurrency": 40,
|
||||
"sql_queries_executed": stats_naive["sql_queries"],
|
||||
"singleflight_shared": stats_naive["singleflight_shared"],
|
||||
"latency_p50_ms": percentile(naive_latencies, 50),
|
||||
"latency_p95_ms": percentile(naive_latencies, 95),
|
||||
"latency_p99_ms": percentile(naive_latencies, 99),
|
||||
},
|
||||
"protected_singleflight": {
|
||||
"concurrency": 40,
|
||||
"sql_queries_executed": stats_protected["sql_queries"],
|
||||
"singleflight_shared": stats_protected["singleflight_shared"],
|
||||
"latency_p50_ms": percentile(protected_latencies, 50),
|
||||
"latency_p95_ms": percentile(protected_latencies, 95),
|
||||
"latency_p99_ms": percentile(protected_latencies, 99),
|
||||
},
|
||||
"warm_cache_40_reads": {
|
||||
"requests": 40,
|
||||
"sql_queries_executed": stats_warm["sql_queries"],
|
||||
"cache_hits": stats_warm["cache_hits"],
|
||||
"latency_p50_ms": percentile(warm_latencies, 50),
|
||||
"latency_p95_ms": percentile(warm_latencies, 95),
|
||||
"latency_p99_ms": percentile(warm_latencies, 99),
|
||||
},
|
||||
},
|
||||
"redis_restart_recovery": {
|
||||
"status_after_restart": st_code,
|
||||
"messages_verified": len(page_recovered["items"]),
|
||||
"session_preserved": True,
|
||||
},
|
||||
}
|
||||
|
||||
with open("evidence/track1_cache_consistency.json", "w", encoding="utf-8") as f:
|
||||
json.dump(summary, f, indent=2)
|
||||
|
||||
print("PASS: Track 1 experiment completed! Saved to evidence/track1_cache_consistency.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,286 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Grade 5 Track 2 Experiment:
|
||||
- Query Plan & EXPLAIN (ANALYZE, BUFFERS) on >=100,000 messages with skewed chat sizes
|
||||
(1 large chat with 80,000 messages, 4 medium chats with 5,000 messages each, 10 small chats with 1,000 messages each = 110,000 messages).
|
||||
- Comparison of 3 query/index approaches and write cost with vs without secondary index.
|
||||
- Full Backup (pg_dump) -> Restore into a fresh PostgreSQL database (pg_restore) -> Checksum & Row Count verification.
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
|
||||
def run_cmd(cmd):
|
||||
res = subprocess.run(cmd, shell=True, capture_output=True, text=True)
|
||||
if res.returncode != 0:
|
||||
raise RuntimeError(f"Command failed ({cmd}): {res.stderr}")
|
||||
return res.stdout.strip()
|
||||
|
||||
|
||||
def psql(db, sql):
|
||||
res = subprocess.run(
|
||||
f"docker exec -i mtusi-lab2-postgres psql -v ON_ERROR_STOP=1 -U messenger -d {db} -tA",
|
||||
input=sql,
|
||||
shell=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
)
|
||||
if res.returncode != 0:
|
||||
raise RuntimeError(f"PSQL failed on {db}: {res.stderr}\nSQL: {sql[:200]}")
|
||||
return res.stdout.strip()
|
||||
|
||||
|
||||
def main():
|
||||
print("[Track 2 - Step 1] Preparing 110,000 skewed messages across 15 chats...")
|
||||
# Create dedicated synthetic benchmark user and 15 skewed chats
|
||||
setup_sql = """
|
||||
DO $$
|
||||
DECLARE
|
||||
u_id UUID := '11111111-1111-4111-8111-111111111111';
|
||||
hot_chat UUID := '22222222-2222-4222-8222-222222222222';
|
||||
c_id UUID;
|
||||
i INT;
|
||||
BEGIN
|
||||
INSERT INTO users (id, username, display_name, password_hash, created_at)
|
||||
VALUES (u_id, 'track2_bench_user', 'Track 2 Researcher', '$argon2id$v=19$m=19456,t=1,p=2$dGVzdHNhbHQxMjM0NTY3OA$dGVzdGhhc2gxMjM0NTY3ODkwMTIzNDU2Nzg5MDEyMzQ', NOW())
|
||||
ON CONFLICT (username) DO NOTHING;
|
||||
|
||||
SELECT id INTO u_id FROM users WHERE username = 'track2_bench_user';
|
||||
|
||||
-- Delete previous benchmark chats if re-running
|
||||
DELETE FROM chats WHERE title LIKE 'Track2_Skewed_%';
|
||||
|
||||
-- 1 Hot chat (80,000 messages)
|
||||
INSERT INTO chats (id, title, created_at) VALUES (hot_chat, 'Track2_Skewed_Hot_80k', NOW());
|
||||
INSERT INTO chat_members (chat_id, user_id, ord) VALUES (hot_chat, u_id, 0);
|
||||
|
||||
-- 4 Medium chats (5,000 messages each = 20,000)
|
||||
FOR i IN 1..4 LOOP
|
||||
c_id := gen_random_uuid();
|
||||
INSERT INTO chats (id, title, created_at) VALUES (c_id, 'Track2_Skewed_Medium_' || i, NOW());
|
||||
INSERT INTO chat_members (chat_id, user_id, ord) VALUES (c_id, u_id, 0);
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT gen_random_uuid(), c_id, u_id, 'Medium chat message #' || g,
|
||||
TIMESTAMPTZ '2026-01-01 00:00:00Z' + (g * INTERVAL '1 millisecond')
|
||||
FROM generate_series(1, 5000) AS g;
|
||||
END LOOP;
|
||||
|
||||
-- 10 Small chats (1,000 messages each = 10,000)
|
||||
FOR i IN 1..10 LOOP
|
||||
c_id := gen_random_uuid();
|
||||
INSERT INTO chats (id, title, created_at) VALUES (c_id, 'Track2_Skewed_Small_' || i, NOW());
|
||||
INSERT INTO chat_members (chat_id, user_id, ord) VALUES (c_id, u_id, 0);
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT gen_random_uuid(), c_id, u_id, 'Small chat message #' || g,
|
||||
TIMESTAMPTZ '2026-01-01 00:00:00Z' + (g * INTERVAL '1 millisecond')
|
||||
FROM generate_series(1, 1000) AS g;
|
||||
END LOOP;
|
||||
END $$;
|
||||
"""
|
||||
psql("messenger", setup_sql)
|
||||
|
||||
# Measure Write Cost: Inserting 40,000 rows WITHOUT composite index vs 40,000 rows WITH composite index
|
||||
print("[Track 2 - Step 2] Measuring write cost with vs without composite index (80,000 messages into hot chat)...")
|
||||
psql("messenger", "DROP INDEX IF EXISTS idx_messages_chat_created_id;")
|
||||
t0 = time.perf_counter()
|
||||
psql("messenger", """
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT gen_random_uuid(),
|
||||
'22222222-2222-4222-8222-222222222222'::uuid,
|
||||
(SELECT id FROM users WHERE username = 'track2_bench_user'),
|
||||
'Hot chat message batch 1 #' || g,
|
||||
TIMESTAMPTZ '2026-02-01 00:00:00Z' + (g * INTERVAL '1 millisecond')
|
||||
FROM generate_series(1, 40000) AS g;
|
||||
""")
|
||||
write_no_index_ms = round((time.perf_counter() - t0) * 1000.0, 2)
|
||||
|
||||
# Recreate the composite index and measure insertion of the next 40,000 messages
|
||||
psql("messenger", "CREATE INDEX idx_messages_chat_created_id ON messages (chat_id, created_at ASC, id ASC);")
|
||||
t1 = time.perf_counter()
|
||||
psql("messenger", """
|
||||
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
|
||||
SELECT gen_random_uuid(),
|
||||
'22222222-2222-4222-8222-222222222222'::uuid,
|
||||
(SELECT id FROM users WHERE username = 'track2_bench_user'),
|
||||
'Hot chat message batch 2 #' || g,
|
||||
TIMESTAMPTZ '2026-02-01 00:00:00Z' + ((40000 + g) * INTERVAL '1 millisecond')
|
||||
FROM generate_series(1, 40000) AS g;
|
||||
""")
|
||||
write_with_index_ms = round((time.perf_counter() - t1) * 1000.0, 2)
|
||||
|
||||
psql("messenger", "ANALYZE messages;")
|
||||
total_messages = int(psql("messenger", "SELECT count(*) FROM messages;"))
|
||||
hot_chat_messages = int(psql("messenger", "SELECT count(*) FROM messages WHERE chat_id = '22222222-2222-4222-8222-222222222222';"))
|
||||
|
||||
# Find the cursor at offset 50,000 in the hot chat
|
||||
cursor_row = psql("messenger", """
|
||||
SELECT created_at::text || '|' || id::text
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
ORDER BY created_at ASC, id ASC
|
||||
OFFSET 50000 LIMIT 1;
|
||||
""")
|
||||
cursor_ts, cursor_id = cursor_row.split("|")
|
||||
|
||||
# Compare 3 Query Plans using EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)
|
||||
print("[Track 2 - Step 3] Running EXPLAIN (ANALYZE, BUFFERS) across 3 index/query designs...")
|
||||
|
||||
# Plan A: Composite Index (chat_id, created_at ASC, id ASC) + Keyset Pagination at depth 50,000
|
||||
plan_keyset_json = json.loads(psql("messenger", f"""
|
||||
EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
AND (created_at, id) > ('{cursor_ts}'::timestamptz, '{cursor_id}'::uuid)
|
||||
ORDER BY created_at ASC, id ASC
|
||||
LIMIT 50;
|
||||
"""))[0]
|
||||
|
||||
plan_keyset_text = psql("messenger", f"""
|
||||
EXPLAIN (ANALYZE, BUFFERS)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
AND (created_at, id) > ('{cursor_ts}'::timestamptz, '{cursor_id}'::uuid)
|
||||
ORDER BY created_at ASC, id ASC
|
||||
LIMIT 50;
|
||||
""")
|
||||
|
||||
# Plan B: Single-column index on (chat_id) + OFFSET 50000 LIMIT 50
|
||||
psql("messenger", "DROP INDEX idx_messages_chat_created_id;")
|
||||
psql("messenger", "CREATE INDEX idx_messages_chat_only ON messages (chat_id);")
|
||||
psql("messenger", "ANALYZE messages;")
|
||||
|
||||
plan_offset_json = json.loads(psql("messenger", """
|
||||
EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
ORDER BY created_at ASC, id ASC
|
||||
OFFSET 50000 LIMIT 50;
|
||||
"""))[0]
|
||||
|
||||
# Plan C: No index on messages (Seq Scan + Top-N Heap Sort) with Keyset filter
|
||||
psql("messenger", "DROP INDEX idx_messages_chat_only;")
|
||||
psql("messenger", "ANALYZE messages;")
|
||||
|
||||
plan_seqscan_json = json.loads(psql("messenger", f"""
|
||||
EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)
|
||||
SELECT id, chat_id, sender_id, text, created_at
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
AND (created_at, id) > ('{cursor_ts}'::timestamptz, '{cursor_id}'::uuid)
|
||||
ORDER BY created_at ASC, id ASC
|
||||
LIMIT 50;
|
||||
"""))[0]
|
||||
|
||||
# Restore the production composite keyset index!
|
||||
psql("messenger", "CREATE INDEX idx_messages_chat_created_id ON messages (chat_id, created_at ASC, id ASC);")
|
||||
psql("messenger", "ANALYZE messages;")
|
||||
|
||||
# Step 4: Backup -> Restore into a fresh database -> Verify row counts and sampled messages
|
||||
print("[Track 2 - Step 4] Performing pg_dump backup -> pg_restore into clean DB -> checksum verification...")
|
||||
sample_before = psql("messenger", """
|
||||
SELECT id::text || ':' || md5(text) || ':' || created_at::text
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
ORDER BY created_at ASC, id ASC
|
||||
OFFSET 25000 LIMIT 20;
|
||||
""")
|
||||
digest_before = hashlib.sha256(sample_before.encode("utf-8")).hexdigest()
|
||||
|
||||
t_bkp = time.perf_counter()
|
||||
run_cmd("docker exec mtusi-lab2-postgres pg_dump -Fc -U messenger -d messenger -f /tmp/lab2_backup.dump")
|
||||
backup_ms = round((time.perf_counter() - t_bkp) * 1000.0, 2)
|
||||
|
||||
psql("postgres", "DROP DATABASE IF EXISTS messenger_restored;")
|
||||
psql("postgres", "CREATE DATABASE messenger_restored OWNER messenger;")
|
||||
|
||||
t_rst = time.perf_counter()
|
||||
run_cmd("docker exec mtusi-lab2-postgres pg_restore -U messenger -d messenger_restored /tmp/lab2_backup.dump")
|
||||
restore_ms = round((time.perf_counter() - t_rst) * 1000.0, 2)
|
||||
|
||||
restored_total = int(psql("messenger_restored", "SELECT count(*) FROM messages;"))
|
||||
restored_hot = int(psql("messenger_restored", "SELECT count(*) FROM messages WHERE chat_id = '22222222-2222-4222-8222-222222222222';"))
|
||||
sample_after = psql("messenger_restored", """
|
||||
SELECT id::text || ':' || md5(text) || ':' || created_at::text
|
||||
FROM messages
|
||||
WHERE chat_id = '22222222-2222-4222-8222-222222222222'
|
||||
ORDER BY created_at ASC, id ASC
|
||||
OFFSET 25000 LIMIT 20;
|
||||
""")
|
||||
digest_after = hashlib.sha256(sample_after.encode("utf-8")).hexdigest()
|
||||
|
||||
# Clean up temporary restored DB and dump file, and remove synthetic 110k rows so DB stays fast for other tests
|
||||
psql("postgres", "DROP DATABASE messenger_restored;")
|
||||
run_cmd("docker exec mtusi-lab2-postgres rm -f /tmp/lab2_backup.dump")
|
||||
psql("messenger", "DELETE FROM chats WHERE title LIKE 'Track2_Skewed_%';")
|
||||
|
||||
if restored_total != total_messages or digest_before != digest_after:
|
||||
raise AssertionError("Backup/restore verification mismatch!")
|
||||
|
||||
with open("evidence/explain_keyset_100k.txt", "w", encoding="utf-8") as f:
|
||||
f.write(plan_keyset_text + "\n")
|
||||
|
||||
summary = {
|
||||
"track": "Track 2: Query Plan on >=100,000 Skewed Messages and Backup/Restore Verification",
|
||||
"dataset": {
|
||||
"total_messages": total_messages,
|
||||
"hot_chat_messages": hot_chat_messages,
|
||||
"medium_chats": "4 chats x 5,000 messages = 20,000",
|
||||
"small_chats": "10 chats x 1,000 messages = 10,000",
|
||||
"deep_pagination_offset_tested": 50000,
|
||||
},
|
||||
"write_cost_40k_messages": {
|
||||
"without_composite_index_ms": write_no_index_ms,
|
||||
"with_composite_index_ms": write_with_index_ms,
|
||||
"index_write_overhead_pct": round(((write_with_index_ms - write_no_index_ms) / max(1.0, write_no_index_ms)) * 100.0, 2),
|
||||
},
|
||||
"query_plans_at_depth_50000": {
|
||||
"composite_keyset_index": {
|
||||
"index": "idx_messages_chat_created_id (chat_id, created_at ASC, id ASC)",
|
||||
"query_type": "Keyset: WHERE chat_id = $1 AND (created_at, id) > ($2, $3) ORDER BY created_at ASC, id ASC LIMIT 50",
|
||||
"node_type": plan_keyset_json["Plan"]["Node Type"],
|
||||
"execution_time_ms": plan_keyset_json["Execution Time"],
|
||||
"shared_hit_blocks": plan_keyset_json["Plan"].get("Shared Hit Blocks", 0),
|
||||
"shared_read_blocks": plan_keyset_json["Plan"].get("Shared Read Blocks", 0),
|
||||
},
|
||||
"single_column_index_with_offset": {
|
||||
"index": "idx_messages_chat_only (chat_id)",
|
||||
"query_type": "Offset: WHERE chat_id = $1 ORDER BY created_at ASC, id ASC OFFSET 50000 LIMIT 50",
|
||||
"node_type": plan_offset_json["Plan"]["Node Type"],
|
||||
"execution_time_ms": plan_offset_json["Execution Time"],
|
||||
"shared_hit_blocks": plan_offset_json["Plan"].get("Shared Hit Blocks", 0),
|
||||
"shared_read_blocks": plan_offset_json["Plan"].get("Shared Read Blocks", 0),
|
||||
},
|
||||
"no_index_seq_scan": {
|
||||
"index": "NONE (Seq Scan + Top-N Sort)",
|
||||
"query_type": "Keyset without index",
|
||||
"node_type": plan_seqscan_json["Plan"]["Node Type"],
|
||||
"execution_time_ms": plan_seqscan_json["Execution Time"],
|
||||
"shared_hit_blocks": plan_seqscan_json["Plan"].get("Shared Hit Blocks", 0),
|
||||
"shared_read_blocks": plan_seqscan_json["Plan"].get("Shared Read Blocks", 0),
|
||||
},
|
||||
},
|
||||
"backup_and_restore": {
|
||||
"tool": "pg_dump -Fc / pg_restore",
|
||||
"backup_duration_ms": backup_ms,
|
||||
"restore_duration_ms": restore_ms,
|
||||
"source_messages_count": total_messages,
|
||||
"restored_messages_count": restored_total,
|
||||
"sample_sha256_before": digest_before,
|
||||
"sample_sha256_after": digest_after,
|
||||
"exact_match": digest_before == digest_after and total_messages == restored_total,
|
||||
},
|
||||
}
|
||||
|
||||
with open("evidence/track2_query_plan_and_backup.json", "w", encoding="utf-8") as f:
|
||||
json.dump(summary, f, indent=2)
|
||||
|
||||
print("PASS: Track 2 experiment completed! Saved to evidence/track2_query_plan_and_backup.json")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user