ЛР 3: балансировка, идемпотентность и проверки на 4

This commit is contained in:
2026-11-01 17:02:42 +03:00
parent 0ab3a341aa
commit 412b1bb09d
64 changed files with 3904 additions and 53 deletions
+9 -20
View File
@@ -1,20 +1,9 @@
.git
.github
.env
.env.*
!.env.example
secrets
certs
volumes
data
backups
*.key
*.pem
*.p12
*.pfx
*.jks
.venv
__pycache__
node_modules
evidence
.DS_Store
*
!Dockerfile
!requirements.txt
!app/
!app/*.py
!migrations/
!migrations/*.sql
!contracts/
!contracts/openapi.json
+4 -10
View File
@@ -1,13 +1,7 @@
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
# These are logical interface suggestions; map actual framework names in REPORT.md.
APP_HOST=0.0.0.0
APP_PORT=8080
# python scripts/init_local.py creates .env and secret files locally.
COMPOSE_PROJECT_NAME=mtusi-lab3
API_PORT=8080
APP_ENV=lab
APP_VERSION=lab3-v1
SESSION_TTL_SECONDS=3600
# Choose credentials locally. File paths contain no secret values.
DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
API_REPLICAS=2
IDEMPOTENCY_TTL_SECONDS=86400
+5
View File
@@ -0,0 +1,5 @@
*.py text eol=lf
*.sh text eol=lf
*.sql text eol=lf
*.yaml text eol=lf
*.cfg text eol=lf
+27
View File
@@ -0,0 +1,27 @@
FROM docker:28.5.1-cli AS docker_cli
FROM python:3.13.15-slim-bookworm AS runtime
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 \
APP_HOST=0.0.0.0 APP_PORT=8080 APP_ENV=lab
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt \
&& useradd --uid 10001 --create-home messenger
COPY app ./app
COPY migrations ./migrations
COPY contracts/openapi.json ./contracts/openapi.json
USER messenger
EXPOSE 8080
HEALTHCHECK --interval=5s --timeout=3s --start-period=10s --retries=5 \
CMD python -c "import os, urllib.request; urllib.request.urlopen('http://127.0.0.1:' + os.getenv('APP_PORT', '8080') + '/health/ready', timeout=2)"
CMD ["python", "-m", "app"]
# Optional test operator. The API image does not contain the Docker client/socket.
FROM runtime AS checks
USER root
COPY --from=docker_cli /usr/local/bin/docker /usr/local/bin/docker
COPY --from=docker_cli /usr/local/libexec/docker/cli-plugins/docker-compose /usr/local/libexec/docker/cli-plugins/docker-compose
RUN apt-get update && apt-get install -y --no-install-recommends make \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /checks
CMD ["make", "test-student"]
+7
View File
@@ -28,3 +28,10 @@ test-persistence:
test-failover:
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable stop-one hook"; exit 2)
$(PYTHON) tests/resilience.py --mode failover --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
.PHONY: test-student explain
test-student:
$(PYTHON) -m unittest discover -s student-tests -v
explain:
$(PYTHON) scripts/explain_history.py
+103 -23
View File
@@ -1,37 +1,117 @@
# Отчёт — лабораторная 3
# Лабораторная 3 — отчёт
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор: Саламатин, БПИ-2304.
- Целевая оценка: **4**, весь минимум уровня 3 и все пункты уровня 4. Треки на 5 не выполнялись.
- Проверка: 1 ноября 2026 года, Europe/Moscow. В JSON используется UTC.
- Ветка: `solution/bpi-2304-salamatin`. Основа задания — commit `0ab3a34` репозитория `mtusi2026/mtusi2026lab3`.
- Перенесена собственная lab2 из [gwalut/mtusi2026lab2](https://gitea.privcloud.tech/gwalut/mtusi2026lab2), ветка `solution/bpi-2304-salamatin`. README задания, COURSE, contracts и публичные tests lab3 сохранены.
## Запуск из чистого clone
## Запуск
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Нужны Docker Desktop с Linux containers и Docker Compose. Для полного автоматического прогона на хосте нужен Python 3.9+; Python приложения и зависимости устанавливаются в контейнере. На Windows текстовые проверки запускаются с `-X utf8`.
## Архитектура и решения
Из корня clone, PowerShell:
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab2, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
```powershell
python -X utf8 scripts/init_local.py
python -X utf8 scripts/verify.py
```
## Доказательства критериев
`verify.py` собирает образы, поднимает стенд, выполняет публичные и собственные тесты, полный restart, failover, expiry, проверку SQL-плана и нагрузочные эксперименты. Итог — `evidence/verification.json`. Данные не удаляются; стенд остаётся работающим. Тесты создают только синтетических пользователей и сообщения.
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
На текущем компьютере переносимый Python находится на уровень выше репозитория:
```powershell
..\.tools\python\python.exe -X utf8 scripts/verify.py
```
Для обычного запуска на Windows без установленного Python: `./start.ps1`. Скрипт создаёт конфигурацию через Python-контейнер, собирает и запускает Compose. Эквивалентные команды вручную:
```powershell
docker run --rm --mount "type=bind,source=$($PWD.Path),target=/work" --workdir /work python:3.13.15-slim-bookworm python scripts/init_local.py
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build --wait --wait-timeout 120
curl.exe --noproxy "*" http://127.0.0.1:8080/health/ready
```
`init_local.py` создаёт `.env` и локальные случайные пароли/DSN/ключ cursor в `secrets/`. Существующие значения сохраняются и не печатаются. Эти файлы исключены из Git. Не удалять секреты при сохранённых volumes: пароль существующей PostgreSQL не меняется от замены файла.
По умолчанию опубликован только `127.0.0.1:8080` балансировщика. Для другого свободного порта изменить `API_PORT` в `.env` до запуска. `API_REPLICAS=2` задаёт число API, `SESSION_TTL_SECONDS=3600` — срок новых сессий, `APP_VERSION` — метку конфигурации; остальные runtime-настройки и пути секретов заданы в Compose. Приложение всегда слушает `0.0.0.0:8080` внутри контейнера.
Отдельные проверки:
```powershell
docker compose run --rm --no-deps checks make check test
docker compose run --rm --no-deps checks make test-student
docker compose run --rm --no-deps checks sh -ec 'chmod +x scripts/restart.sh; make test-persistence ACTION_SCRIPT=scripts/restart.sh'
docker compose run --rm --no-deps checks sh -ec 'chmod +x scripts/stop-one.sh; make test-failover ACTION_SCRIPT=scripts/stop-one.sh'
docker compose start api
docker compose run --rm --no-deps checks make explain
python -X utf8 scripts/experiments.py --base-url http://127.0.0.1:8080
```
После публичного failover реплика намеренно остаётся остановленной: восстановление выполняется отдельной командой `docker compose start api`. Полный `verify.py` делает это сам. После восстановления нужно дождаться готовности обеих реплик. На Linux для прямого запуска hooks: `chmod +x scripts/restart.sh scripts/stop-one.sh`.
Остановка с сохранением данных: `docker compose stop`, либо `docker compose down` для удаления контейнеров и сети с сохранением volumes. Только для намеренного удаления всех учебных данных: `docker compose down --volumes`. Эта команда не применяется в проверках отказа и restart.
## Реализация
Клиент → HAProxy → API × 2 → общие PostgreSQL и Redis. Используется один масштабируемый сервис `api` без `container_name`. API, БД и Redis не публикуют порты. Балансировка round-robin, без sticky sessions. Лимиты установлены у всех runtime-контейнеров, сервиса миграций и клиента проверок.
Из lab2 сохранены FastAPI, PostgreSQL-миграции, параметризованные запросы, ограниченный pool, scrypt, Basic при входе, Bearer-сессии в Redis с TTL/AOF, проверка членства, подписанные cursor и индекс истории. Добавлены `X-Instance-Id`, обязательный ключ отправки, таблица `message_requests`, HAProxy и сценарии отказа/замены.
Результат отправки и само сообщение сохраняются одной SQL-транзакцией. Блокировка строки чата синхронизирует разные процессы. Повтор валидного тела и ключа в течение 24 часов возвращает тот же Message с 201; другое тело — 409. Область ключа включает пользователя и чат; повтор заново проходит аутентификацию и проверку доступа. Ошибка транзакции не сохраняется как окончательный результат. История не кэшируется.
Readiness API требует обе зависимости. HAProxy проверяет его с `inter 1s`, `fall 2`, `rise 1`, `fastinter 500ms`, `timeout check/connect 1s`; консервативный бюджет обнаружения зависшего backend — до 4 s при штатном планировании. Docker DNS обновляет адреса, HAProxy поддерживает до восьми реплик. API обрабатывает SIGTERM через Uvicorn: закрытие listener, до 5 s на текущие запросы; Compose даёт 8 s. Общий HTTP-бюджет API — 5 s, таймаут HAProxy до backend — 6 s. Прозрачных повторов POST в LB нет.
Миграции выполняет отдельный сервис `migrate`; advisory lock и checksum предотвращают гонку/изменение применённого SQL. Для последовательной замены миграции проверяются один раз перед обновлением. 002 добавляет новую таблицу, 001 сохранена. Подробности: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), сценарий защиты — [docs/DEFENSE.md](docs/DEFENSE.md).
## Проверки критериев
| Критерий | Команда / сценарий | Результат | Доказательство в evidence/ |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Обязательный API и переход lab2 → lab3 | `make test` через LB и smoke с хоста | PASS | `smoke.txt`, `published-smoke.txt` |
| Две настоящие реплики и общая сессия | Повтор `/users/me` с одним токеном; Docker inspect | Не менее двух instance ID и разных контейнеров | `student-tests.txt`, `verification.json` |
| Идемпотентность между репликами | 100 запросов, 10 клиентов, один ключ | Одно сообщение и одна запись результата | `idempotency.json` |
| Конфликт, область ключа, expiry и повтор после logout | Собственные интеграционные тесты | 409 / независимые ключи / новый результат после expiry / 401 | `student-tests.txt` |
| Атомарность сообщения и результата | SQL-trigger вызывает ошибку при INSERT результата | 503; обе записи откатились; повтор после устранения ошибки успешен | `student-tests.txt` |
| Полный перезапуск и сохранность | `make test-persistence` с реализованным hook | Старый токен и данные сохранены; volumes те же, StartedAt изменился у всех пяти контейнеров | `persistence.txt`, `restart.json` |
| Публичный failover | `make test-failover` | Две реплики до отказа; восстановление 2.064 s, 2 ошибки, 117 успешных записей | `failover.txt`, `last-stop.json` |
| Readiness, timeouts и SIGTERM | Healthchecks HAProxy, stop каждой API | Оба процесса завершились с exit code 0, записи продолжились менее чем за 10 s | `failure-api-1.json`, `failure-api-2.json` |
| Отказ каждой реплики с восстановлением предыдущей | `scripts/experiments.py` | Измерены RPS, ошибки, latency и время первой записи после остановки | `experiments.json` |
| Ресурсные лимиты и одна точка входа | Inspect всех runtime-контейнеров | CPU/RAM ограничены; опубликован только LB; API non-root | `verification.json` |
| Поочерёдная замена под нагрузкой | Drain → stop/remove → recreate, отдельно для каждой API | Два новых контейнера; 0 HTTP-ошибок; все 4475 подтверждений сохранены | `rolling-update.json` |
| Прежние проверки уровня 4 lab2 | 12 интеграционных тестов, TTL=2, EXPLAIN | PASS; 401 через 2.3 s; индекс на 10 000 сообщениях | `student-tests.txt`, `expiry.json`, `explain.txt` |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
## Измерения
## Эксперименты и отказы
Стенд: Windows, Docker Desktop 4.50.0/Engine 28.5.1, Linux x86_64. Docker VM: 16 логических CPU, около 7.47 GiB RAM. Лимиты: каждая API — 1 CPU/256 MiB, PostgreSQL — 1 CPU/512 MiB, Redis — 0.5 CPU/192 MiB, HAProxy — 0.5 CPU/128 MiB, checks — 1 CPU/256 MiB. Нагрузочный клиент работает на хосте; CPU/RAM компонентов во время нагрузки сохранены в JSON.
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Версии образов: Python `3.13.15-slim-bookworm`, PostgreSQL `17.6-bookworm`, Redis `7.4.5-bookworm`, HAProxy `3.0.12-alpine`, Docker CLI проверок `28.5.1-cli`. Зависимости закреплены в `requirements.txt`; image ID фактического стенда записаны в JSON.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
Перед каждой нагрузкой создаётся отдельный чат и подтверждённое сообщение; наблюдаются две реплики с одной сессией. Нагрузка closed-loop, четыре клиента, преимущественно отправка сообщений и периодическое чтение текущего пользователя. Воздействие начинается через 4 секунды. При неопределённом результате POST сохраняются ключ и тело. После опыта читаются все страницы истории: сверяются ID и тела всех подтверждений, проверяются дубли. Общий snapshot истории не требуется.
## Ограничения и следующие шаги
| Опыт | Длительность | Успешные RPS | Ошибок попыток | Первая запись после воздействия | p50 / p95 / p99, ms | Подтверждений сверено |
| --- | ---: | ---: | ---: | ---: | --- | ---: |
| Остановка первой API | 24.041 s | 105.235 | 51 (1.976%) | 1.715 s | 32.652 / 75.027 / 96.705 | 2350 |
| Остановка второй API | 24.034 s | 108.762 | 57 (2.134%) | 1.697 s | 32.088 / 65.561 / 97.489 | 2429 |
| Поочерёдная замена обеих API | 45.015 s | 107.276 | 0 | Запросы продолжались; замена заняла около 23.5 s | 28.249 / 82.882 / 127.075 | 4475 |
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
Краткие ошибки при резкой остановке допустимы: HAProxy ещё обнаруживает отказ. Они включены в счётчики, потерянных подтверждений и дублей не обнаружено. При управляемой замене backend сначала переводится в drain; в выполненном опыте ошибок не было. Поле `first_write_after_action_seconds` отсчитывается от начала действия до первой записи после завершения действия; для rolling оно включает длительность всей замены и **не означает 23.5 секунды простоя**.
Это одиночные функциональные опыты, не исследование максимальной производительности. Первые 4 s нагрузки служат коротким прогревом и входят в общий RPS. Percentile рассчитаны только для успешных HTTP-операций; closed-loop ограничен клиентом и не показывает реальную очередь при фиксированной входящей интенсивности. Профиль и лимиты одинаковы для двух отказов. Сырые агрегаты, двухсекундные интервалы нагрузки, ошибки, состояния контейнеров и выборки CPU/RAM находятся в соответствующих JSON.
## Ограничения
Один host, одна БД, один Redis и один LB остаются точками отказа. Проверен отказ одной API-реплики, а не потеря host или любого хранилища. Сессии требуют сохранённого volume Redis. Backup/restore и треки на 5 не выполнялись. Просроченные ключи очищаются при повторном использовании; периодическая фоновая уборка не добавлена.
Проверена эквивалентная поочерёдная замена контейнеров того же приложения с `APP_VERSION=lab3-v2`, без изменения бизнес-кода между версиями. Совместимость произвольных будущих миграций не обещается; изменения схемы должны быть обратно совместимыми либо требовать отдельного окна обслуживания. Смешанный запуск старого lab2 и нового lab3 API не поддерживается из-за изменения обязательного контракта ключа.
Учебный стенд использует локальный HTTP и синтетические данные. У API нет UI; отдельные опыты масштабирования 1/2/4 и отказ нескольких узлов не заявляются.
## Источники
- [Задание lab3](https://gitea.privcloud.tech/mtusi2026/mtusi2026lab3), нормативный OpenAPI и публичные проверки.
- [HAProxy: DNS resolution](https://www.haproxy.com/documentation/haproxy-configuration-tutorials/proxying-essentials/dns-resolution/), [health checks](https://www.haproxy.com/documentation/haproxy-configuration-tutorials/reliability/health-checks/), [Runtime API](https://www.haproxy.com/documentation/haproxy-runtime-api/).
- [Docker Compose services](https://docs.docker.com/reference/compose-file/services/), [PostgreSQL locking](https://www.postgresql.org/docs/17/explicit-locking.html).
- Использованы FastAPI, Pydantic, Uvicorn, Psycopg, redis-py и стандартная библиотека Python. Codex использовался при подготовке кода, проверок и документации.
+1
View File
@@ -0,0 +1 @@
"""Laboratory 1: a single-process, in-memory messenger."""
+17
View File
@@ -0,0 +1,17 @@
import logging
import os
import uvicorn
def main():
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logging.getLogger("psycopg.pool").setLevel(logging.CRITICAL)
logging.info("Starting lab3 environment=%s with shared PostgreSQL and Redis", os.getenv("APP_ENV", "lab"))
uvicorn.run("app.api:app", host=os.getenv("APP_HOST", "0.0.0.0"),
port=int(os.getenv("APP_PORT", "8080")), workers=1,
access_log=False, timeout_graceful_shutdown=5)
if __name__ == "__main__":
main()
+176
View File
@@ -0,0 +1,176 @@
import asyncio
from contextlib import asynccontextmanager
import json
import logging
import os
from pathlib import Path
import re
from time import perf_counter
from uuid import uuid4
from fastapi import Depends, FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from fastapi.responses import Response
import psycopg
from psycopg_pool import PoolTimeout, TooManyRequests
from redis.exceptions import RedisError
from pydantic import ValidationError
from starlette.exceptions import HTTPException
from .config import Settings
from .errors import APIError, invalid, unavailable
from .models import CreateChat, CreateMessage, CreateUser
from .security import basic_credentials, identifier
from .store import Store
logger = logging.getLogger("messenger")
def error_response(request, status, code, message, headers=None):
return JSONResponse(status_code=status, headers=headers, content={"error": {
"code": code, "message": message, "request_id": request.state.request_id}})
async def body(request, model):
content_type = request.headers.get("content-type", "").split(";", 1)[0].strip().lower()
if content_type != "application/json":
raise APIError(415, "unsupported_media_type", "Use application/json")
try:
raw = (await request.body()).decode("utf-8")
def reject_constant(value):
raise ValueError("Nonstandard JSON constant")
data = json.loads(raw, parse_constant=reject_constant)
return model.model_validate(data)
except (ValueError, UnicodeError, ValidationError):
raise invalid("Body must be valid UTF-8 JSON with the required fields and types") from None
def pagination(request):
raw = request.query_params.get("limit", "50")
if re.fullmatch(r"[0-9]+", raw) is None or len(raw) > 3 or not 1 <= int(raw) <= 100:
raise invalid("Limit must be an integer from 1 to 100")
return int(raw), request.query_params.get("cursor")
def create_app(settings=None):
@asynccontextmanager
async def lifespan(app):
store = Store(settings or Settings.load())
app.state.store = store
try:
await store.open()
yield
finally:
await store.close()
instance_id = os.getenv("HOSTNAME", "api") + "-" + uuid4().hex[:12]
app = FastAPI(title="Lab 3 messenger", docs_url=None, redoc_url=None,
openapi_url=None, redirect_slashes=False, lifespan=lifespan)
@app.middleware("http")
async def request_context(request, call_next):
request.state.request_id = str(uuid4())
start = perf_counter()
try:
async with asyncio.timeout(5):
response = await call_next(request)
except (psycopg.Error, PoolTimeout, TooManyRequests, RedisError, TimeoutError):
error = unavailable()
logger.warning("Storage unavailable request_id=%s", request.state.request_id)
response = error_response(request, error.status, error.code, error.message)
except Exception:
logger.error("Unhandled error request_id=%s", request.state.request_id)
response = error_response(request, 500, "internal_error", "Internal server error")
response.headers["X-Request-Id"] = request.state.request_id
response.headers["X-Instance-Id"] = instance_id
response.headers["X-App-Version"] = os.getenv("APP_VERSION", "lab3-v1")
logger.info("request_id=%s method=%s path=%s status=%s duration_ms=%.2f",
request.state.request_id, request.method, request.url.path,
response.status_code, (perf_counter() - start) * 1000)
return response
@app.exception_handler(APIError)
async def api_error(request, error):
return error_response(request, error.status, error.code, error.message, error.headers)
@app.exception_handler(RequestValidationError)
async def validation_error(request, error):
return error_response(request, 400, "invalid_request", "Invalid request parameters")
@app.exception_handler(HTTPException)
async def routing_error(request, error):
code = "not_found" if error.status_code == 404 else "invalid_request"
return error_response(request, error.status_code, code, "Unknown path or unsupported method")
async def current_user(request: Request):
user, token = await app.state.store.authenticate(request.headers.get("authorization"))
request.state.session_token = token
return user
@app.get("/health/live")
async def health():
return {"status": "ok"}
@app.get("/health/ready")
async def readiness():
await app.state.store.ready()
return {"status": "ok"}
@app.post("/api/v1/auth/sessions", status_code=201)
async def login(request: Request):
credentials = basic_credentials(request.headers.get("authorization"))
if await request.body():
raise invalid("Session creation does not accept a body")
return await app.state.store.login(*credentials)
@app.delete("/api/v1/auth/sessions/current", status_code=204)
async def logout(request: Request, user=Depends(current_user)):
await app.state.store.logout(request.state.session_token)
return Response(status_code=204)
@app.post("/api/v1/users", status_code=201)
async def create_user(request: Request):
return await app.state.store.create_user(await body(request, CreateUser))
@app.get("/api/v1/users/me")
async def me(user=Depends(current_user)):
return user
@app.post("/api/v1/chats", status_code=201)
async def create_chat(request: Request, user=Depends(current_user)):
return await app.state.store.create_chat(user["id"], await body(request, CreateChat))
@app.get("/api/v1/chats")
async def list_chats(request: Request, user=Depends(current_user)):
return await app.state.store.list_chats(user["id"], *pagination(request))
@app.get("/api/v1/chats/{chat_id}")
async def get_chat(chat_id: str, user=Depends(current_user)):
return await app.state.store.get_chat(user["id"], identifier(chat_id))
@app.post("/api/v1/chats/{chat_id}/messages", status_code=201)
async def create_message(chat_id: str, request: Request, user=Depends(current_user)):
chat_id = identifier(chat_id)
await app.state.store.get_chat(user["id"], chat_id)
key = request.headers.get("idempotency-key", "")
if re.fullmatch(r"[A-Za-z0-9._:-]{1,128}", key, flags=re.ASCII) is None:
raise invalid("A valid Idempotency-Key is required")
return await app.state.store.create_message(user["id"], chat_id,
await body(request, CreateMessage), key)
@app.get("/api/v1/chats/{chat_id}/messages")
async def list_messages(chat_id: str, request: Request, user=Depends(current_user)):
chat_id = identifier(chat_id)
await app.state.store.get_chat(user["id"], chat_id)
return await app.state.store.list_messages(user["id"], chat_id, *pagination(request))
@app.get("/openapi.json", include_in_schema=False)
async def contract():
path = Path(__file__).resolve().parents[1] / "contracts" / "openapi.json"
return json.loads(path.read_text(encoding="utf-8"))
return app
app = create_app()
+28
View File
@@ -0,0 +1,28 @@
"""Runtime settings; credential values are never logged."""
from dataclasses import dataclass
import os
from pathlib import Path
def secret(name):
return Path(os.environ[name + "_FILE"]).read_text(encoding="utf-8").strip()
@dataclass(frozen=True)
class Settings:
database_url: str
cache_url: str
cursor_key: bytes
session_ttl: int
@classmethod
def load(cls):
ttl = int(os.getenv("SESSION_TTL_SECONDS", "3600"))
if not 1 <= ttl <= 604800:
raise ValueError("SESSION_TTL_SECONDS must be in 1..604800")
if os.getenv("SESSION_STORE", "redis") != "redis":
raise ValueError("This deployment uses SESSION_STORE=redis")
key = secret("CURSOR_KEY").encode()
if len(key) < 32:
raise ValueError("Cursor key must contain at least 32 bytes")
return cls(secret("DATABASE_URL"), secret("CACHE_URL"), key, ttl)
+22
View File
@@ -0,0 +1,22 @@
class APIError(Exception):
def __init__(self, status: int, code: str, message: str, headers=None):
self.status = status
self.code = code
self.message = message
self.headers = headers or {}
super().__init__(message)
def invalid(message="Invalid request"):
return APIError(400, "invalid_request", message)
def unauthorized(basic=False):
challenge = ('Basic realm="messenger", charset="UTF-8"' if basic
else 'Bearer realm="messenger"')
return APIError(401, "unauthorized", "Valid credentials required",
{"WWW-Authenticate": challenge})
def unavailable():
return APIError(503, "unavailable", "Storage temporarily unavailable")
+37
View File
@@ -0,0 +1,37 @@
"""Apply numbered migrations atomically; safe to repeat and serialize."""
import hashlib
from pathlib import Path
import psycopg
from .config import secret
ROOT = Path(__file__).resolve().parents[1]
def migrate():
with psycopg.connect(secret("DATABASE_URL"), connect_timeout=2,
options="-c statement_timeout=10000 -c lock_timeout=5000") as conn:
conn.execute("SELECT pg_advisory_xact_lock(202602)")
conn.execute("""CREATE TABLE IF NOT EXISTS schema_migrations (
version text PRIMARY KEY, sha256 text NOT NULL,
applied_at timestamptz NOT NULL DEFAULT clock_timestamp())""")
for path in sorted((ROOT / "migrations").glob("[0-9]*.sql")):
# Git checkout may change LF to CRLF on Windows; SQL is unchanged.
content = path.read_text(encoding="utf-8").encode("utf-8")
checksum = hashlib.sha256(content).hexdigest()
row = conn.execute("SELECT sha256 FROM schema_migrations WHERE version=%s", (path.name,)).fetchone()
if row:
if row[0] != checksum:
raise RuntimeError("Applied migration checksum differs: " + path.name)
print("Unchanged:", path.name)
continue
conn.execute(content.decode("utf-8"))
conn.execute("INSERT INTO schema_migrations(version, sha256) VALUES (%s, %s)",
(path.name, checksum))
print("Applied:", path.name)
print("PASS: migrations committed")
if __name__ == "__main__":
migrate()
+50
View File
@@ -0,0 +1,50 @@
"""Strict request models; strings retain their original Unicode content."""
import re
from uuid import UUID
from pydantic import BaseModel, ConfigDict, Field, field_validator
class Input(BaseModel):
model_config = ConfigDict(strict=True, extra="forbid")
@field_validator("*", check_fields=False)
@classmethod
def valid_text(cls, value):
if isinstance(value, str):
if not value.strip():
raise ValueError("A string must not be blank")
value.encode("utf-8")
if "\x00" in value:
raise ValueError("NUL is not supported")
return value
class CreateUser(Input):
username: str
display_name: str = Field(min_length=1, max_length=100)
password: str = Field(min_length=12, max_length=128)
@field_validator("username")
@classmethod
def username_format(cls, value):
if re.fullmatch(r"[a-z0-9_]{3,32}", value) is None:
raise ValueError("Invalid username")
return value
class CreateChat(Input):
title: str = Field(min_length=1, max_length=100)
member_ids: list[str] = Field(min_length=1, max_length=100)
@field_validator("member_ids")
@classmethod
def members(cls, values):
normalized = [str(UUID(value)) for value in values]
if len(set(normalized)) != len(normalized):
raise ValueError("Duplicate members")
return normalized
class CreateMessage(Input):
text: str = Field(min_length=1, max_length=4000)
+93
View File
@@ -0,0 +1,93 @@
"""scrypt passwords, strict Basic credentials and persistent signed cursors."""
import base64
from datetime import datetime, timezone
import hashlib
import hmac
import json
import re
import secrets
from uuid import UUID
from .errors import invalid, unauthorized
def password_hash(password, salt=None):
salt = salt or secrets.token_bytes(16)
digest = hashlib.scrypt(password.encode("utf-8"), salt=salt, n=16384,
r=8, p=1, dklen=32, maxmem=64 * 1024 * 1024)
return "scrypt$16384$8$1$" + salt.hex() + "$" + digest.hex()
def password_matches(password, encoded):
try:
algorithm, n, r, p, salt, digest = encoded.split("$")
if (algorithm, n, r, p) != ("scrypt", "16384", "8", "1"):
return False
actual = password_hash(password, bytes.fromhex(salt)).rsplit("$", 1)[1]
return hmac.compare_digest(actual, digest)
except (ValueError, UnicodeError):
return False
def basic_credentials(header):
try:
scheme, encoded = header.split(" ", 1)
if scheme.lower() != "basic" or len(encoded) > 2048:
raise ValueError
value = base64.b64decode(encoded, validate=True).decode("utf-8")
username, password = value.split(":", 1)
if re.fullmatch(r"[a-z0-9_]{3,32}", username) is None:
raise ValueError
if not 12 <= len(password) <= 128:
raise ValueError
return username, password
except (AttributeError, ValueError, UnicodeError):
raise unauthorized(basic=True) from None
def bearer(header):
try:
scheme, token = header.split(" ", 1)
if scheme.lower() != "bearer" or re.fullmatch(r"[A-Za-z0-9_-]{43}", token) is None:
raise ValueError
return token
except (AttributeError, ValueError):
raise unauthorized() from None
def identifier(value):
try:
return UUID(value)
except (ValueError, AttributeError, TypeError):
raise invalid("Invalid UUID") from None
def stamp(value):
return value.astimezone(timezone.utc).isoformat(timespec="microseconds").replace("+00:00", "Z")
class Cursors:
def __init__(self, key):
self.key = key
def encode(self, scope, item):
payload = json.dumps([scope, item["created_at"], item["id"]], separators=(",", ":")).encode()
signature = hmac.digest(self.key, payload, hashlib.sha256)
return base64.urlsafe_b64encode(signature + payload).decode().rstrip("=")
def decode(self, scope, token):
try:
if not token or len(token) > 1024:
raise ValueError
raw = base64.b64decode(token + "=" * (-len(token) % 4), altchars=b"-_", validate=True)
signature, payload = raw[:32], raw[32:]
if not hmac.compare_digest(signature, hmac.digest(self.key, payload, hashlib.sha256)):
raise ValueError
parts = json.loads(payload)
if not isinstance(parts, list) or len(parts) != 3 or parts[0] != scope:
raise ValueError
if not all(isinstance(x, str) for x in parts) or not parts[1].endswith("Z"):
raise ValueError
return datetime.fromisoformat(parts[1].replace("Z", "+00:00")), UUID(parts[2])
except (ValueError, TypeError, UnicodeError):
raise invalid("Invalid cursor for this resource") from None
+228
View File
@@ -0,0 +1,228 @@
"""Persistent business data and durable Redis sessions; no local state cache."""
import asyncio
from datetime import datetime, timezone
import hashlib
import json
import secrets
from uuid import UUID, uuid4
from psycopg.errors import ForeignKeyViolation, UniqueViolation
from psycopg.rows import dict_row
from psycopg_pool import AsyncConnectionPool
import redis.asyncio as redis
from redis.asyncio.retry import Retry
from redis.backoff import NoBackoff
from .errors import APIError, invalid, unauthorized
from .security import Cursors, bearer, password_hash, password_matches, stamp
USER_COLUMNS = "id, username, display_name, created_at"
MESSAGE_COLUMNS = "id, chat_id, sender_id, text, created_at"
def public(row):
return {key: (str(value) if isinstance(value, UUID) else
stamp(value) if isinstance(value, datetime) else value)
for key, value in row.items()}
class Store:
def __init__(self, settings):
self.settings = settings
self.cursors = Cursors(settings.cursor_key)
self.kdf_slots = asyncio.Semaphore(4)
self.dummy_hash = password_hash("dummy-password-for-missing-user", b"0" * 16)
self.pool = AsyncConnectionPool(
settings.database_url, min_size=1, max_size=10, max_waiting=20,
timeout=1, open=False, reconnect_timeout=5,
check=AsyncConnectionPool.check_connection,
kwargs={"row_factory": dict_row, "connect_timeout": 2,
"keepalives": 1, "keepalives_idle": 2, "keepalives_interval": 1,
"keepalives_count": 2, "tcp_user_timeout": 2000,
"options": "-c timezone=UTC -c statement_timeout=2000 -c lock_timeout=1000"})
self.sessions = redis.Redis.from_url(
settings.cache_url, decode_responses=True, max_connections=10,
socket_connect_timeout=1, socket_timeout=1,
retry=Retry(NoBackoff(), 0), retry_on_timeout=False)
async def open(self):
await self.pool.open()
await self.pool.wait(timeout=10)
await self.sessions.ping()
async def close(self):
await self.sessions.aclose()
await self.pool.close()
async def ready(self):
async with self.pool.connection() as conn:
await conn.execute("SELECT 1")
await self.sessions.ping()
async def create_user(self, data):
async with self.kdf_slots:
encoded = await asyncio.to_thread(password_hash, data.password)
try:
async with self.pool.connection() as conn:
cur = await conn.execute(
"INSERT INTO users(id, username, display_name, password_hash) "
"VALUES (%s, %s, %s, %s) RETURNING " + USER_COLUMNS,
(uuid4(), data.username, data.display_name, encoded))
result = public(await cur.fetchone())
return result
except UniqueViolation:
raise APIError(409, "conflict", "Username is already taken") from None
async def login(self, username, password):
async with self.pool.connection() as conn:
cur = await conn.execute("SELECT " + USER_COLUMNS + ", password_hash FROM users WHERE username=%s",
(username,))
row = await cur.fetchone()
async with self.kdf_slots:
valid = await asyncio.to_thread(password_matches, password,
row["password_hash"] if row else self.dummy_hash)
if not valid or row is None:
raise unauthorized(basic=True)
row.pop("password_hash")
token = secrets.token_urlsafe(32)
expires_ms = int(datetime.now(timezone.utc).timestamp() * 1000) + self.settings.session_ttl * 1000
await self.sessions.set(self.session_key(token), json.dumps(
{"user_id": str(row["id"]), "expires_ms": expires_ms}), pxat=expires_ms)
return {"token": token, "token_type": "Bearer",
"expires_at": stamp(datetime.fromtimestamp(expires_ms / 1000, timezone.utc)),
"user": public(row)}
@staticmethod
def session_key(token):
return "session:" + hashlib.sha256(token.encode("ascii")).hexdigest()
async def authenticate(self, header):
token = bearer(header)
value = await self.sessions.get(self.session_key(token))
if value is None:
raise unauthorized()
session = json.loads(value)
if session["expires_ms"] <= int(datetime.now(timezone.utc).timestamp() * 1000):
raise unauthorized()
async with self.pool.connection() as conn:
cur = await conn.execute("SELECT " + USER_COLUMNS + " FROM users WHERE id=%s",
(UUID(session["user_id"]),))
row = await cur.fetchone()
if row is None:
raise unauthorized()
return public(row), token
async def logout(self, token):
await self.sessions.delete(self.session_key(token))
async def create_chat(self, actor, data):
if actor not in data.member_ids:
raise invalid("Members must include the current user")
try:
async with self.pool.connection() as conn:
cur = await conn.execute("INSERT INTO chats(id, title) VALUES (%s, %s) "
"RETURNING id, title, created_at", (uuid4(), data.title))
row = await cur.fetchone()
async with conn.cursor() as cur:
await cur.executemany("INSERT INTO chat_members(chat_id, user_id) VALUES (%s, %s)",
[(row["id"], UUID(member)) for member in data.member_ids])
result = public(row)
result["member_ids"] = sorted(data.member_ids)
return result
except ForeignKeyViolation:
raise invalid("All members must exist") from None
async def accessible_chat(self, conn, actor, chat_id):
cur = await conn.execute("""SELECT c.id, c.title, c.created_at FROM chats c
JOIN chat_members m ON m.chat_id=c.id
WHERE c.id=%s AND m.user_id=%s""", (chat_id, UUID(actor)))
row = await cur.fetchone()
if row is None:
raise APIError(404, "not_found", "Chat not found")
return row
async def get_chat(self, actor, chat_id):
async with self.pool.connection() as conn:
row = await self.accessible_chat(conn, actor, chat_id)
cur = await conn.execute("SELECT user_id FROM chat_members WHERE chat_id=%s ORDER BY user_id", (chat_id,))
result = public(row)
result["member_ids"] = [str(x["user_id"]) for x in await cur.fetchall()]
return result
async def create_message(self, actor, chat_id, data, key):
fingerprint = hashlib.sha256(json.dumps({"text": data.text},
ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8")).hexdigest()
async with self.pool.connection() as conn:
await self.accessible_chat(conn, actor, chat_id)
# Evaluate clock_timestamp AFTER obtaining the per-chat lock.
await conn.execute("SELECT id FROM chats WHERE id=%s FOR UPDATE", (chat_id,))
# All replicas share this lock. Replay and side effect commit together.
await conn.execute("DELETE FROM message_requests WHERE user_id=%s AND chat_id=%s "
"AND request_key=%s AND expires_at <= clock_timestamp()", (UUID(actor), chat_id, key))
replay = await conn.execute("SELECT request_hash, message_id FROM message_requests "
"WHERE user_id=%s AND chat_id=%s AND request_key=%s", (UUID(actor), chat_id, key))
previous = await replay.fetchone()
if previous:
if previous["request_hash"] != fingerprint:
raise APIError(409, "conflict", "Idempotency key was used with a different message")
saved = await conn.execute("SELECT " + MESSAGE_COLUMNS + " FROM messages WHERE id=%s",
(previous["message_id"],))
return public(await saved.fetchone())
cur = await conn.execute("""UPDATE chats SET last_message_at=
greatest(clock_timestamp(), last_message_at + interval '1 microsecond')
WHERE id=%s RETURNING last_message_at""", (chat_id,))
created_at = (await cur.fetchone())["last_message_at"]
cur = await conn.execute("INSERT INTO messages(id, chat_id, sender_id, text, created_at) "
"VALUES (%s, %s, %s, %s, %s) RETURNING " + MESSAGE_COLUMNS,
(uuid4(), chat_id, UUID(actor), data.text, created_at))
result = public(await cur.fetchone())
await conn.execute("INSERT INTO message_requests "
"(user_id, chat_id, request_key, request_hash, message_id) VALUES (%s,%s,%s,%s,%s)",
(UUID(actor), chat_id, key, fingerprint, UUID(result["id"])))
return result
def page(self, rows, scope, limit):
items = rows[:limit]
return {"items": items, "next_cursor": self.cursors.encode(scope, items[-1])
if len(rows) > limit else None}
async def list_messages(self, actor, chat_id, limit, cursor):
scope = "messages:" + str(chat_id)
boundary = self.cursors.decode(scope, cursor) if cursor is not None else None
async with self.pool.connection() as conn:
await self.accessible_chat(conn, actor, chat_id)
query = "SELECT " + MESSAGE_COLUMNS + " FROM messages WHERE chat_id=%s"
params = [chat_id]
if boundary:
query += " AND (created_at, id) > (%s, %s)"
params.extend(boundary)
query += " ORDER BY created_at, id LIMIT %s"
params.append(limit + 1)
cur = await conn.execute(query, params)
rows = [public(row) for row in await cur.fetchall()]
return self.page(rows, scope, limit)
async def list_chats(self, actor, limit, cursor):
scope = "chats:" + actor
boundary = self.cursors.decode(scope, cursor) if cursor is not None else None
query = """SELECT c.id, c.title, c.created_at FROM chats c
JOIN chat_members m ON m.chat_id=c.id WHERE m.user_id=%s"""
params = [UUID(actor)]
if boundary:
query += " AND (c.created_at, c.id) > (%s, %s)"
params.extend(boundary)
query += " ORDER BY c.created_at, c.id LIMIT %s"
params.append(limit + 1)
async with self.pool.connection() as conn:
cur = await conn.execute(query, params)
rows = [public(row) for row in await cur.fetchall()]
if rows:
ids = [UUID(row["id"]) for row in rows]
cur = await conn.execute("SELECT chat_id, user_id FROM chat_members WHERE chat_id=ANY(%s) ORDER BY user_id",
(ids,))
members = {row["id"]: [] for row in rows}
for member in await cur.fetchall():
members[str(member["chat_id"])].append(str(member["user_id"]))
for row in rows:
row["member_ids"] = members[row["id"]]
return self.page(rows, scope, limit)
+122
View File
@@ -0,0 +1,122 @@
name: mtusi-lab3
x-app: &app
image: mtusi-messenger:lab3
build: {context: ., target: runtime}
environment: &environment
APP_HOST: 0.0.0.0
APP_PORT: "8080"
APP_ENV: ${APP_ENV:-lab}
APP_VERSION: ${APP_VERSION:-lab3-v1}
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
SESSION_STORE: redis
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL_FILE: /run/secrets/cache_url
CURSOR_KEY_FILE: /run/secrets/cursor_key
secrets: [database_url, cache_url, cursor_key]
networks: [data]
cpus: 1.0
mem_limit: 256m
stop_grace_period: 8s
services:
postgres:
image: postgres:17.6-bookworm
environment:
POSTGRES_USER: messenger
POSTGRES_DB: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
secrets: [postgres_password]
volumes: [postgres_data:/var/lib/postgresql/data]
networks: [data]
healthcheck:
test: [CMD-SHELL, "pg_isready -U messenger -d messenger"]
interval: 2s
timeout: 2s
retries: 30
cpus: 1.0
mem_limit: 512m
stop_grace_period: 10s
redis:
image: redis:7.4.5-bookworm
command: [sh, -ec, 'exec redis-server /usr/local/etc/redis/redis.conf --requirepass "$$(cat /run/secrets/redis_password)"']
secrets: [redis_password]
volumes:
- redis_data:/data
- ./infra/redis.conf:/usr/local/etc/redis/redis.conf:ro
networks: [data]
healthcheck:
test: [CMD-SHELL, 'REDISCLI_AUTH="$$(cat /run/secrets/redis_password)" redis-cli ping | grep -q PONG']
interval: 2s
timeout: 2s
retries: 30
cpus: 0.5
mem_limit: 192m
stop_grace_period: 10s
migrate:
<<: *app
command: [python, -m, app.migrate]
depends_on:
postgres: {condition: service_healthy}
restart: "no"
api:
<<: *app
deploy:
replicas: ${API_REPLICAS:-2}
depends_on:
migrate: {condition: service_completed_successfully}
redis: {condition: service_healthy}
networks: [edge, data]
lb:
image: haproxy:3.0.12-alpine
volumes:
- ./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
ports:
- "127.0.0.1:${API_PORT:-8080}:8080"
networks: [edge]
depends_on:
api: {condition: service_healthy}
cpus: 0.5
mem_limit: 128m
stop_grace_period: 8s
healthcheck:
test: [CMD, wget, -q, -O, /dev/null, "http://127.0.0.1:8080/health/ready"]
interval: 2s
timeout: 2s
retries: 30
checks:
profiles: [tools]
image: mtusi-messenger:lab3-checks
build: {context: ., target: checks}
environment:
<<: *environment
BASE_URL: http://lb:8080
COMPOSE_PROJECT_NAME: ${COMPOSE_PROJECT_NAME:-mtusi-lab3}
secrets: [database_url, cache_url, cursor_key]
volumes:
- .:/checks
- /var/run/docker.sock:/var/run/docker.sock
networks: [edge, data]
cpus: 1.0
mem_limit: 256m
networks:
edge:
data:
internal: true
volumes:
postgres_data:
redis_data:
secrets:
postgres_password: {file: ./secrets/postgres_password}
redis_password: {file: ./secrets/redis_password}
database_url: {file: ./secrets/database_url}
cache_url: {file: ./secrets/cache_url}
cursor_key: {file: ./secrets/cursor_key}
+61
View File
@@ -0,0 +1,61 @@
# Устройство стенда
```mermaid
flowchart LR
C[HTTP-клиент] --> L[HAProxy :8080]
L --> A[API, реплика 1]
L --> B[API, реплика 2]
A & B --> P[(PostgreSQL)]
A & B --> R[(Redis: сессии)]
M[Одноразовые миграции] --> P
```
Только HAProxy публикует порт на loopback хоста. API подключены к сетям `edge` и `data`; PostgreSQL и Redis — только к внутренней `data`. Балансировщик не получает пароли БД и Redis. Сервис `checks` запускается явно через профиль `tools`; Docker socket нужен ему для сценариев отказа, приложение его не получает.
## Состояние и транзакции
PostgreSQL хранит пользователей, чаты, членство, сообщения и результаты идемпотентных запросов. Redis хранит общие серверные сессии. В API нет локального хранилища бизнес-данных или сессий. Ключ подписи cursor одинаков на всех репликах и сохраняется в файле секрета. Следующую страницу можно читать через другую реплику.
Из lab2 перенесены параметризованные SQL-запросы, scrypt, ограничения пулов, таймауты и keyset-пагинация. Индекс истории: `messages(chat_id, created_at, id)`. Для страницы выбираются максимум `limit + 1` строк после `(created_at, id)`, без OFFSET.
Миграция 001 сохранена без изменения. Новая 002 добавляет `message_requests`: составной PK `(user_id, chat_id, request_key)`, fingerprint валидированного JSON, ID сообщения и срок действия. Метод и путь операции фиксированы: POST сообщения данного чата. Поэтому эта комбинация реализует область ключа «пользователь + метод + путь + ключ».
Отправка сообщения:
1. Проверить действительность сессии и членство, формат ключа и JSON.
2. В транзакции заблокировать строку чата `FOR UPDATE`. Блокировка общая для всех реплик.
3. Удалить прежнюю запись данного ключа, если истекли 24 часа.
4. Для действующего ключа сравнить fingerprint. При совпадении вернуть сохранённое сообщение с 201; при различии — 409.
5. Для нового ключа назначить время, вставить сообщение и результат запроса, затем COMMIT. Только после этого вернуть 201.
Ошибка до COMMIT откатывает обе записи. После потери ответа клиент повторяет тот же ключ и JSON: при состоявшемся COMMIT получает прежний результат, иначе создаёт сообщение. Локальный mutex и sticky sessions для этого не используются. JSON разбирается до хеширования: пробелы и представление Unicode не меняют результат. В контракте lab3 тело содержит только `text`; вложения появляются в lab4.
В одном чате записи сериализуются. Метка времени выбирается после блокировки как максимум часов PostgreSQL и предыдущей метки + 1 микросекунда. Это сохраняет порядок при одинаковом времени или откате часов. Между страницами нет snapshot; новые сообщения могут появиться на следующей странице. Чтение идёт в PostgreSQL, кэша истории нет.
Сессия: случайный Bearer с энтропией 256 бит, SHA-256 токена в ключе Redis, абсолютный expiry и `PXAT`. Чтение и рестарт не продлевают срок. Logout удаляет ключ до 204. Для сохранности Redis настроены volume, AOF и `appendfsync always`. Потеря Redis даёт 503; локального обхода аутентификации нет.
## Балансировка и остановка
HAProxy использует round-robin без affinity. Каждый ответ приложения, включая ошибку и health, содержит `X-Instance-Id`: hostname контейнера и случайный суффикс процесса. Это позволяет отличать и контейнеры, и перезапуски. Дополнительный `X-App-Version` показывает версию конфигурации при последовательной замене.
HAProxy опрашивает `/health/ready` каждую секунду: `fall 2`, `rise 1`, `fastinter 500ms`, `timeout check 1s`, `timeout connect 1s`. Консервативный бюджет исключения зависшего backend — до 4 секунд при штатном планировании; остановившийся процесс обычно обнаруживается быстрее. Docker healthcheck дополняет проверку LB и не заменяет её. Readiness API требует PostgreSQL и Redis, liveness зависит только от процесса.
Docker DNS `127.0.0.11` обновляет адреса `api`; `server-template` резервирует восемь backend-слотов. `hold valid/obsolete` — 1 секунда. При замене контейнера HAProxy получает новый IP без перезапуска. Для этого стенда поддерживаются 2–8 реплик.
API: SQL 2 s, lock 1 s, pool 1 s, Redis 1 s, общий HTTP-бюджет 5 s. HAProxy ждёт backend до 6 s, клиента до 10 s. Автоматических повторов в LB нет: повтор POST выполняет клиент с исходным ключом.
Uvicorn получает SIGTERM как PID 1, закрывает слушающий сокет и завершает текущие запросы в пределах 5 s. Docker даёт API 8 s. Публичный hook оставляет выбранную реплику остановленной и проверяет exit code 0. Нагрузка измеряет доступность через оставшуюся реплику.
## Поочерёдная замена
`scripts/experiments.py` выполняет миграции один раз перед заменой. Для каждой из двух API-реплик находит backend по IP, включает `drain` через внутренний HAProxy Runtime API и ждёт завершения активных запросов. Затем останавливает и удаляет только этот контейнер и восстанавливает недостающую реплику командой `compose up --no-recreate --scale api=2 api`. Вторая продолжает работать. После обнаружения новой реплики через LB операция повторяется для второй.
Эксперимент проверяет поочерёдную замену контейнеров того же образа с новой меткой конфигурации `APP_VERSION=lab3-v2`. Изменение бизнес-логики между версиями не моделируется. Миграции 001/002 добавляют таблицы и совместимы с текущим lab3 API. Обновление с lab2 без промежуточной остановки не заявляется: старый API ещё не требует ключа идемпотентности.
Runtime API HAProxy доступен только внутри Docker-сети, порт 9999 не публикуется. Он используется операторским контейнером для drain; обычный клиент обращается только к опубликованному HTTP-порту.
## Границы решения
Стенд выдерживает остановку одной API-реплики и перезапуск компонентов с сохранением volumes. Единственные PostgreSQL, Redis, HAProxy и Docker-host остаются точками отказа. Потеря диска, удаление volume, Redis flush и отказ всего host не покрываются. Это локальный учебный HTTP-стенд с синтетическими данными, без UI и TLS.
Просроченные записи идемпотентности удаляются при повторном использовании соответствующего ключа. Старые неиспользуемые ключи можно удалять отдельно по индексу `expires_at`; фоновая очистка в этой лабораторной не реализована.
+23
View File
@@ -0,0 +1,23 @@
# Короткий сценарий защиты
1. Открыть `REPORT.md`: цель — оценка 4. Показать `compose.yaml`: две реплики одного сервиса `api`, общие PostgreSQL/Redis, только один published port у `lb`, лимиты ресурсов.
2. Выполнить `docker compose ps`. Показать, что ID из `X-Instance-Id` относятся к разным контейнерам. Одна Bearer-сессия работает через обе реплики без повторного входа.
3. Выполнить `docker compose run --rm --no-deps checks make test`. Показать регистрацию, чат, сообщения, пагинацию и проверки доступа.
4. Выполнить `docker compose run --rm --no-deps checks make test-student`. В `evidence/idempotency.json` — 100 запросов с одним ключом, две реальные реплики и одно сообщение. В тестах также проверяется откат ошибки после INSERT сообщения.
5. Выполнить `docker compose run --rm --no-deps checks sh -ec 'chmod +x scripts/stop-one.sh; make test-failover ACTION_SCRIPT=scripts/stop-one.sh'`. Показать `evidence/last-stop.json` и остановленный контейнер в `docker compose ps --all`. Затем восстановить: `docker compose start api`.
6. Показать результаты отказа каждой реплики и замены под нагрузкой: `evidence/experiments.json`, подробности `failure-api-1.json`, `failure-api-2.json`, `rolling-update.json`. В каждом есть RPS, ошибки, latency и сверка подтверждённых сообщений.
7. Показать полный restart: `evidence/persistence.txt` и `restart.json`. Прежние токены и данные сохраняются; перезапущены API, PostgreSQL, Redis и LB, volumes те же.
## Что объяснить
- **Почему две реплики взаимозаменяемы?** Пользователи/сообщения находятся в PostgreSQL, сессии — в общем Redis; ключ cursor общий. В памяти API нет состояния, необходимого для следующего запроса.
- **Почему одного healthcheck Docker недостаточно?** Он отмечает состояние контейнера. Исключать backend из HTTP-маршрутизации должен балансировщик; HAProxy сам проверяет readiness.
- **Что будет при timeout POST?** COMMIT мог уже завершиться. Повторяем прежний ключ и тело: получаем то же сообщение. Новый ключ может создать новое сообщение.
- **Почему нет дублей при двух одновременных запросах?** Реплики используют одну транзакционную блокировку строки чата, а сообщение и запись результата фиксируются атомарно. PK дополнительно ограничивает ключ.
- **Что означает 409?** Действующий ключ уже был использован для другого текста. Клиенту нужно проверить свою логику повторов.
- **Зачем AOF, если есть TTL?** TTL задаёт срок действия, а AOF сохраняет сессии после рестарта Redis.
- **Почему нельзя говорить об отказоустойчивости всего сервиса?** БД, Redis, LB и host здесь одиночные. Проверен отказ API-процесса, а не потеря любого узла.
- **Чем последовательная замена отличается от остановки всего сервиса?** Одна реплика выводится из балансировки, завершается и заменяется, пока другая обслуживает клиентов. Затем роли меняются.
- **Что ограничивает масштабирование?** БД и Redis остаются общими, запись в один чат сериализуется. Добавление API не убирает эти ограничения.
Перед повтором экспериментов нужны работающий Docker и свободный опубликованный порт. Все тесты создают синтетические данные. Команды `down --volumes` для демонстрации отказа не применяются.
+15
View File
@@ -0,0 +1,15 @@
# Передача работы
Решение находится в ветке `solution/bpi-2304-salamatin`, основанной на skeleton lab3. При публикации нужен fork **третьей** лабораторной; репозиторий lab2 для этого не подходит. Контракт и публичные тесты не изменены.
Заголовок PR: **ЛР 3 — БПИ-2304 — Саламатин — на 4**.
Описание:
> Реализованы две API-реплики за HAProxy с общими PostgreSQL и Redis-сессиями. Отправка сообщения требует Idempotency-Key; сообщение и результат повторного запроса сохраняются атомарно. Добавлены проверки готовности, согласованные таймауты, SIGTERM, лимиты CPU/RAM и обновление адресов backend через Docker DNS.
>
> Публичные smoke/persistence/failover и 12 собственных интеграционных тестов пройдены. Под нагрузкой поочерёдно остановлена каждая реплика, затем выполнена последовательная замена обеих; результаты и ограничения — в REPORT.md и evidence/. Треки на 5 не заявляются.
Для заполнения шаблона преподавателя: автор Саламатин, БПИ-2304, цель 4; база — собственная lab2; команда полной проверки `python -X utf8 scripts/verify.py`. Сведения об источниках и инструментах приведены в конце отчёта.
В публикацию включаются код, конфигурации, миграции, тесты, REPORT.md и небольшие файлы evidence/. `.env`, `secrets/`, Python runtime, Docker volumes и локальные настройки не включаются. Наличие локальной ветки не означает публикацию fork или открытый PR.
+347
View File
@@ -0,0 +1,347 @@
#1 [internal] load local bake definitions
#1 reading from stdin 1.58kB 0.1s done
#1 DONE 0.1s
#2 [checks internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.23kB 0.0s done
#2 DONE 0.1s
#2 [api internal] load build definition from Dockerfile
#2 DONE 0.1s
#3 [checks internal] load metadata for docker.io/library/python:3.13.15-slim-bookworm
#3 ...
#4 [checks internal] load metadata for docker.io/library/docker:28.5.1-cli
#4 DONE 4.4s
#3 [api internal] load metadata for docker.io/library/python:3.13.15-slim-bookworm
#3 DONE 4.4s
#5 [api internal] load .dockerignore
#5 transferring context: 164B done
#5 DONE 0.1s
#5 [checks internal] load .dockerignore
#5 DONE 0.1s
#6 [checks internal] load build context
#6 transferring context: 182.88kB 0.0s done
#6 DONE 0.2s
#7 [api runtime 1/7] FROM docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26
#7 resolve docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26 0.1s done
#7 DONE 0.5s
#8 [checks docker_cli 1/1] FROM docker.io/library/docker:28.5.1-cli@sha256:9190b0613792e658a7783cf14b2d5ace5941bb68ede7276922ea36ee457d76ad
#8 resolve docker.io/library/docker:28.5.1-cli@sha256:9190b0613792e658a7783cf14b2d5ace5941bb68ede7276922ea36ee457d76ad 0.1s done
#8 sha256:2d7ba3afdf47b5c4691580b6f2e2e6e3e2b3bc9bd6a7c60aeacb0d907f06c339 0B / 116B 0.2s
#8 sha256:0a7ca70c232bfd9c484ad87453366b208174bdb0e87e457ce31db9b4511d07e7 0B / 1.01kB 0.2s
#8 sha256:2d7ba3afdf47b5c4691580b6f2e2e6e3e2b3bc9bd6a7c60aeacb0d907f06c339 116B / 116B 0.5s done
#8 sha256:4ca5c7282971aeea368a994cfaadb48e02c3fd851ae30db56495310f53c981ee 0B / 540B 0.2s
#8 sha256:0a7ca70c232bfd9c484ad87453366b208174bdb0e87e457ce31db9b4511d07e7 1.01kB / 1.01kB 0.5s done
#8 sha256:4ca5c7282971aeea368a994cfaadb48e02c3fd851ae30db56495310f53c981ee 540B / 540B 0.6s done
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 0B / 21.74MB 0.2s
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 0B / 22.16MB 0.1s
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 3.15MB / 21.74MB 1.3s
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 6.29MB / 21.74MB 1.8s
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 8.39MB / 21.74MB 2.1s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 0B / 20.43MB 0.2s
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 12.58MB / 21.74MB 3.3s
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 14.68MB / 21.74MB 4.0s
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 16.78MB / 21.74MB 4.6s
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 18.87MB / 21.74MB 5.2s
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 1.05MB / 22.16MB 5.2s
#8 ...
#7 [api runtime 1/7] FROM docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26
#7 sha256:43dcef5051b0d747db428c3890841803fca6b253a38f1f66774f964ee0dbaa33 3.53MB / 3.53MB 1.1s done
#7 sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150 4.19MB / 28.24MB 7.3s
#7 sha256:2e57034c052c6d059f36df62c9e6f4ee3630bb3a653a5f9e4a196159d53abb35 12.59MB / 12.59MB 3.7s done
#7 sha256:5483d2d2a330470f65f1ca5e1c6a9d4167051e32057624e31e13a9ec6392f42b 250B / 250B 1.2s done
#7 sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150 6.29MB / 28.24MB 9.0s
#7 sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150 8.39MB / 28.24MB 10.3s
#7 sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150 10.49MB / 28.24MB 11.1s
#7 ...
#8 [checks docker_cli 1/1] FROM docker.io/library/docker:28.5.1-cli@sha256:9190b0613792e658a7783cf14b2d5ace5941bb68ede7276922ea36ee457d76ad
#8 sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 21.74MB / 21.74MB 8.5s done
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 8.39MB / 22.16MB 10.3s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 1.05MB / 20.43MB 8.6s
#8 sha256:37009c416213bafe8d6e7c68aa4bc36688e1d7b2974ecd8118fa29dab8c9b0e0 457B / 457B 0.4s done
#8 sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a 3.15MB / 8.23MB 1.5s
#8 sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a 4.19MB / 8.23MB 1.7s
#8 sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a 5.24MB / 8.23MB 1.8s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 2.10MB / 20.43MB 9.5s
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 10.49MB / 22.16MB 11.7s
#8 sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a 6.29MB / 8.23MB 3.5s
#8 sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a 7.34MB / 8.23MB 3.6s
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 12.58MB / 22.16MB 12.6s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 3.15MB / 20.43MB 11.0s
#8 sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a 8.23MB / 8.23MB 3.9s
#8 sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a 8.23MB / 8.23MB 4.6s done
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 14.68MB / 22.16MB 13.5s
#8 extracting sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a
#8 extracting sha256:58bd06dfc6cd22322de7d4fca516b20c2b75730c5bc85c35e92c3f2056dbfe9a 0.4s done
#8 extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0.1s done
#8 extracting sha256:37009c416213bafe8d6e7c68aa4bc36688e1d7b2974ecd8118fa29dab8c9b0e0 0.1s done
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 16.78MB / 22.16MB 14.1s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 4.19MB / 20.43MB 12.3s
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 18.87MB / 22.16MB 15.0s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 5.24MB / 20.43MB 13.5s
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 20.97MB / 22.16MB 15.4s
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 22.16MB / 22.16MB 15.7s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 6.29MB / 20.43MB 14.3s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 7.34MB / 20.43MB 14.9s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 8.39MB / 20.43MB 15.3s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 9.44MB / 20.43MB 15.6s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 10.49MB / 20.43MB 15.9s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 11.53MB / 20.43MB 16.1s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 12.58MB / 20.43MB 16.4s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 13.63MB / 20.43MB 16.5s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 14.68MB / 20.43MB 16.7s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 15.73MB / 20.43MB 16.8s
#8 sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 22.16MB / 22.16MB 18.8s done
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 17.83MB / 20.43MB 17.1s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 18.87MB / 20.43MB 17.3s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 19.92MB / 20.43MB 17.4s
#8 ...
#7 [api runtime 1/7] FROM docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26
#7 sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150 28.24MB / 28.24MB 22.5s
#7 sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150 28.24MB / 28.24MB 25.8s done
#7 DONE 26.4s
#8 [checks docker_cli 1/1] FROM docker.io/library/docker:28.5.1-cli@sha256:9190b0613792e658a7783cf14b2d5ace5941bb68ede7276922ea36ee457d76ad
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 20.43MB / 20.43MB 22.5s
#8 sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 20.43MB / 20.43MB 24.3s done
#8 extracting sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6
#8 extracting sha256:97fde3c329d3c913ca1405c177bffc2ecf02ddd298b32e4aa4489956fc2073e6 0.2s done
#8 DONE 28.8s
#7 [api runtime 1/7] FROM docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26
#7 extracting sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150
#7 ...
#8 [checks docker_cli 1/1] FROM docker.io/library/docker:28.5.1-cli@sha256:9190b0613792e658a7783cf14b2d5ace5941bb68ede7276922ea36ee457d76ad
#8 extracting sha256:cf58801a4522ea7409a3502e3295034b91bdc972cff8ab4bbd5290b68bc215f8 0.3s done
#8 DONE 29.1s
#8 [checks docker_cli 1/1] FROM docker.io/library/docker:28.5.1-cli@sha256:9190b0613792e658a7783cf14b2d5ace5941bb68ede7276922ea36ee457d76ad
#8 extracting sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38
#8 extracting sha256:e12f523aba6af55cd482ed9070373420fd68a86a955abec7599b5be4a6da2a38 0.4s done
#8 DONE 29.5s
#7 [api runtime 1/7] FROM docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26
#7 ...
#8 [checks docker_cli 1/1] FROM docker.io/library/docker:28.5.1-cli@sha256:9190b0613792e658a7783cf14b2d5ace5941bb68ede7276922ea36ee457d76ad
#8 extracting sha256:4ca5c7282971aeea368a994cfaadb48e02c3fd851ae30db56495310f53c981ee 0.0s done
#8 extracting sha256:0a7ca70c232bfd9c484ad87453366b208174bdb0e87e457ce31db9b4511d07e7 0.0s done
#8 extracting sha256:2d7ba3afdf47b5c4691580b6f2e2e6e3e2b3bc9bd6a7c60aeacb0d907f06c339 0.0s done
#8 DONE 29.6s
#7 [api runtime 1/7] FROM docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26
#7 extracting sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150 1.4s done
#7 DONE 27.8s
#7 [checks runtime 1/7] FROM docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26
#7 extracting sha256:774043ccc8ccd0d0833a9ee0792142ab7ad93df971e59dd248fbf82db16d0150 1.4s done
#7 extracting sha256:43dcef5051b0d747db428c3890841803fca6b253a38f1f66774f964ee0dbaa33
#7 extracting sha256:43dcef5051b0d747db428c3890841803fca6b253a38f1f66774f964ee0dbaa33 0.2s done
#7 DONE 28.0s
#7 [checks runtime 1/7] FROM docker.io/library/python:3.13.15-slim-bookworm@sha256:2325bb286ec344af3e5898cc224b5844e2707ac6e26b1632516fd3edc84a5e26
#7 extracting sha256:43dcef5051b0d747db428c3890841803fca6b253a38f1f66774f964ee0dbaa33 0.2s done
#7 extracting sha256:2e57034c052c6d059f36df62c9e6f4ee3630bb3a653a5f9e4a196159d53abb35
#7 extracting sha256:2e57034c052c6d059f36df62c9e6f4ee3630bb3a653a5f9e4a196159d53abb35 0.7s done
#7 extracting sha256:5483d2d2a330470f65f1ca5e1c6a9d4167051e32057624e31e13a9ec6392f42b
#7 extracting sha256:2e57034c052c6d059f36df62c9e6f4ee3630bb3a653a5f9e4a196159d53abb35 0.7s done
#7 extracting sha256:5483d2d2a330470f65f1ca5e1c6a9d4167051e32057624e31e13a9ec6392f42b 0.0s done
#7 DONE 28.8s
#9 [checks runtime 2/7] WORKDIR /app
#9 DONE 1.0s
#9 [checks runtime 2/7] WORKDIR /app
#9 DONE 1.0s
#10 [api runtime 3/7] COPY requirements.txt .
#10 DONE 0.0s
#11 [api runtime 4/7] RUN pip install --no-cache-dir -r requirements.txt && useradd --uid 10001 --create-home messenger
#11 2.091 Ignoring colorama: markers 'sys_platform == "win32"' don't match your environment
#11 2.326 Collecting annotated-doc==0.0.5 (from -r requirements.txt (line 1))
#11 2.926 Downloading annotated_doc-0.0.5-py3-none-any.whl.metadata (6.5 kB)
#11 3.093 Collecting annotated-types==0.8.0 (from -r requirements.txt (line 2))
#11 3.167 Downloading annotated_types-0.8.0-py3-none-any.whl.metadata (15 kB)
#11 3.383 Collecting anyio==4.15.1 (from -r requirements.txt (line 3))
#11 3.459 Downloading anyio-4.15.1-py3-none-any.whl.metadata (4.7 kB)
#11 3.601 Collecting click==8.5.0 (from -r requirements.txt (line 4))
#11 3.676 Downloading click-8.5.0-py3-none-any.whl.metadata (2.6 kB)
#11 3.871 Collecting fastapi==0.141.1 (from -r requirements.txt (line 6))
#11 3.943 Downloading fastapi-0.141.1-py3-none-any.whl.metadata (27 kB)
#11 4.112 Collecting h11==0.16.0 (from -r requirements.txt (line 7))
#11 4.184 Downloading h11-0.16.0-py3-none-any.whl.metadata (8.3 kB)
#11 4.269 Collecting idna==3.20 (from -r requirements.txt (line 8))
#11 4.346 Downloading idna-3.20-py3-none-any.whl.metadata (7.2 kB)
#11 4.650 Collecting pydantic==2.13.5 (from -r requirements.txt (line 9))
#11 4.722 Downloading pydantic-2.13.5-py3-none-any.whl.metadata (110 kB)
#11 5.936 Collecting pydantic-core==2.46.5 (from -r requirements.txt (line 10))
#11 6.010 Downloading pydantic_core-2.46.5-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
#11 6.154 Collecting starlette==1.6.0 (from -r requirements.txt (line 11))
#11 6.227 Downloading starlette-1.6.0-py3-none-any.whl.metadata (6.4 kB)
#11 6.352 Collecting typing-extensions==4.16.0 (from -r requirements.txt (line 12))
#11 6.424 Downloading typing_extensions-4.16.0-py3-none-any.whl.metadata (3.3 kB)
#11 6.607 Collecting typing-inspection==0.4.4 (from -r requirements.txt (line 13))
#11 6.708 Downloading typing_inspection-0.4.4-py3-none-any.whl.metadata (2.6 kB)
#11 6.849 Collecting uvicorn==0.53.0 (from -r requirements.txt (line 14))
#11 6.920 Downloading uvicorn-0.53.0-py3-none-any.whl.metadata (6.6 kB)
#11 7.063 Collecting psycopg==3.2.10 (from psycopg[binary]==3.2.10->-r requirements.txt (line 15))
#11 7.137 Downloading psycopg-3.2.10-py3-none-any.whl.metadata (4.6 kB)
#11 7.220 Collecting psycopg-pool==3.2.6 (from -r requirements.txt (line 16))
#11 7.293 Downloading psycopg_pool-3.2.6-py3-none-any.whl.metadata (2.6 kB)
#11 7.386 Collecting redis==6.4.0 (from -r requirements.txt (line 17))
#11 7.460 Downloading redis-6.4.0-py3-none-any.whl.metadata (10 kB)
#11 7.781 Collecting psycopg-binary==3.2.10 (from psycopg[binary]==3.2.10->-r requirements.txt (line 15))
#11 7.853 Downloading psycopg_binary-3.2.10-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (3.0 kB)
#11 7.984 Downloading annotated_doc-0.0.5-py3-none-any.whl (5.3 kB)
#11 8.056 Downloading annotated_types-0.8.0-py3-none-any.whl (13 kB)
#11 8.136 Downloading anyio-4.15.1-py3-none-any.whl (132 kB)
#11 8.296 Downloading click-8.5.0-py3-none-any.whl (125 kB)
#11 8.428 Downloading fastapi-0.141.1-py3-none-any.whl (131 kB)
#11 8.561 Downloading h11-0.16.0-py3-none-any.whl (37 kB)
#11 8.693 Downloading idna-3.20-py3-none-any.whl (69 kB)
#11 8.772 Downloading pydantic-2.13.5-py3-none-any.whl (472 kB)
#11 8.981 Downloading pydantic_core-2.46.5-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
#11 9.281 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 7.4 MB/s 0:00:00
#11 9.359 Downloading starlette-1.6.0-py3-none-any.whl (75 kB)
#11 9.493 Downloading typing_extensions-4.16.0-py3-none-any.whl (45 kB)
#11 9.576 Downloading typing_inspection-0.4.4-py3-none-any.whl (14 kB)
#11 9.698 Downloading uvicorn-0.53.0-py3-none-any.whl (87 kB)
#11 9.789 Downloading psycopg-3.2.10-py3-none-any.whl (206 kB)
#11 9.881 Downloading psycopg_pool-3.2.6-py3-none-any.whl (38 kB)
#11 10.01 Downloading redis-6.4.0-py3-none-any.whl (279 kB)
#11 10.13 Downloading psycopg_binary-3.2.10-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.4 MB)
#11 10.76 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 7.0 MB/s 0:00:00
#11 10.88 Installing collected packages: typing-extensions, redis, psycopg-binary, psycopg, idna, h11, click, annotated-types, annotated-doc, uvicorn, typing-inspection, pydantic-core, psycopg-pool, anyio, starlette, pydantic, fastapi
#11 12.46
#11 12.46 Successfully installed annotated-doc-0.0.5 annotated-types-0.8.0 anyio-4.15.1 click-8.5.0 fastapi-0.141.1 h11-0.16.0 idna-3.20 psycopg-3.2.10 psycopg-binary-3.2.10 psycopg-pool-3.2.6 pydantic-2.13.5 pydantic-core-2.46.5 redis-6.4.0 starlette-1.6.0 typing-extensions-4.16.0 typing-inspection-0.4.4 uvicorn-0.53.0
#11 12.46 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
#11 12.46
#11 12.46 [notice] A new release of pip is available: 26.2.1 -> 26.2.2
#11 12.46 [notice] To update, run: pip install --upgrade pip
#11 DONE 12.8s
#11 [api runtime 4/7] RUN pip install --no-cache-dir -r requirements.txt && useradd --uid 10001 --create-home messenger
#11 DONE 12.8s
#12 [api runtime 5/7] COPY app ./app
#12 DONE 0.1s
#13 [api runtime 6/7] COPY migrations ./migrations
#13 DONE 0.1s
#13 [checks runtime 6/7] COPY migrations ./migrations
#13 DONE 0.1s
#14 [checks runtime 7/7] COPY contracts/openapi.json ./contracts/openapi.json
#14 DONE 0.1s
#15 [api] exporting to image
#15 exporting layers
#15 ...
#16 [checks checks 1/4] COPY --from=docker_cli /usr/local/bin/docker /usr/local/bin/docker
#16 DONE 0.3s
#17 [checks checks 2/4] COPY --from=docker_cli /usr/local/libexec/docker/cli-plugins/docker-compose /usr/local/libexec/docker/cli-plugins/docker-compose
#17 DONE 0.4s
#15 [api] exporting to image
#15 exporting layers 1.5s done
#15 exporting manifest sha256:485f58e6ec80578fde7c7874080635b0af45b168452bd60283e9e73169a85232 0.0s done
#15 exporting config sha256:9bab07b6683f5e3e69635d80e5db8ff21ea57eb089a9c35bd826ef08817f1590
#15 exporting config sha256:9bab07b6683f5e3e69635d80e5db8ff21ea57eb089a9c35bd826ef08817f1590 0.0s done
#15 exporting attestation manifest sha256:189ff66057b7f64b45184dbbc370dd2fa38ea18ba9317966ffee5812c3b60087 0.0s done
#15 exporting manifest list sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8 0.0s done
#15 naming to docker.io/library/mtusi-messenger:lab3 done
#15 unpacking to docker.io/library/mtusi-messenger:lab3
#15 unpacking to docker.io/library/mtusi-messenger:lab3 0.7s done
#15 DONE 2.5s
#18 [checks checks 3/4] RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/*
#18 0.808 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
#18 0.952 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#18 1.086 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [34.8 kB]
#18 ...
#19 [api] resolving provenance for metadata file
#19 DONE 0.0s
#18 [checks checks 3/4] RUN apt-get update && apt-get install -y --no-install-recommends make && rm -rf /var/lib/apt/lists/*
#18 2.156 Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8790 kB]
#18 3.340 Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [6924 B]
#18 3.509 Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [360 kB]
#18 4.117 Fetched 9398 kB in 4s (2492 kB/s)
#18 4.117 Reading package lists...
#18 4.649 Reading package lists...
#18 5.153 Building dependency tree...
#18 5.292 Reading state information...
#18 5.511 Suggested packages:
#18 5.511 make-doc
#18 5.547 The following NEW packages will be installed:
#18 5.548 make
#18 6.203 0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded.
#18 6.203 Need to get 396 kB of archives.
#18 6.203 After this operation, 1630 kB of additional disk space will be used.
#18 6.203 Get:1 http://deb.debian.org/debian bookworm/main amd64 make amd64 4.3-4.1 [396 kB]
#18 6.348 debconf: delaying package configuration, since apt-utils is not installed
#18 6.387 Fetched 396 kB in 1s (604 kB/s)
#18 6.419 Selecting previously unselected package make.
#18 6.419 (Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 6647 files and directories currently installed.)
#18 6.429 Preparing to unpack .../make_4.3-4.1_amd64.deb ...
#18 6.434 Unpacking make (4.3-4.1) ...
#18 6.521 Setting up make (4.3-4.1) ...
#18 DONE 6.7s
#20 [checks checks 4/4] WORKDIR /checks
#20 DONE 0.0s
#21 [checks] exporting to image
#21 exporting layers
#21 exporting layers 2.5s done
#21 exporting manifest sha256:7d139777a4f9668222d1eaa10c2b725b012d44f7b9733dbe922692ca70c3f671 0.0s done
#21 exporting config sha256:ac219e35d292624b83e4e99111364e4b9490c14b2bcf6f84bf4660b58942c764 0.0s done
#21 exporting attestation manifest sha256:0f58b6134d663547706803962889ceffa18ee5a95e104418cb5d53cb0ad38632 0.0s done
#21 exporting manifest list sha256:352ce6fc20614470d7a12eca4cd30e65c997babd643284f5ac5d9da84e10902a 0.0s done
#21 naming to docker.io/library/mtusi-messenger:lab3-checks done
#21 unpacking to docker.io/library/mtusi-messenger:lab3-checks
#21 unpacking to docker.io/library/mtusi-messenger:lab3-checks 0.6s done
#21 DONE 3.3s
#22 [checks] resolving provenance for metadata file
#22 DONE 0.0s
mtusi-messenger:lab3 Built
mtusi-messenger:lab3-checks Built
+2
View File
@@ -0,0 +1,2 @@
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
+38
View File
@@ -0,0 +1,38 @@
[
{
"scenario": "failure-api-1",
"successful_rps": 105.235,
"failed_attempts": 51,
"first_write_after_action_seconds": 1.715,
"confirmed_writes_checked": 2350,
"successful_latency_ms": {
"p50": 32.652,
"p95": 75.027,
"p99": 96.705
}
},
{
"scenario": "failure-api-2",
"successful_rps": 108.762,
"failed_attempts": 57,
"first_write_after_action_seconds": 1.697,
"confirmed_writes_checked": 2429,
"successful_latency_ms": {
"p50": 32.088,
"p95": 65.561,
"p99": 97.489
}
},
{
"scenario": "rolling-update",
"successful_rps": 107.276,
"failed_attempts": 0,
"first_write_after_action_seconds": 23.527,
"confirmed_writes_checked": 4475,
"successful_latency_ms": {
"p50": 28.249,
"p95": 82.882,
"p99": 127.075
}
}
]
+3
View File
@@ -0,0 +1,3 @@
failure-api-1: 105.235 RPS; 51 errors
failure-api-2: 108.762 RPS; 57 errors
rolling-update: 107.276 RPS; 0 errors
+7
View File
@@ -0,0 +1,7 @@
{
"session_ttl_seconds": 2,
"wait_seconds": 2.3,
"before_expiry_status": 200,
"after_expiry_status": 401,
"result": "PASS"
}
+1
View File
@@ -0,0 +1 @@
PASS: TTL=2s, initial 200, after 2.3s -> 401; normal API TTL unchanged
+2
View File
@@ -0,0 +1,2 @@
python3 scripts/explain_history.py
PASS: 10000 messages, first/middle/late pages use history index; 51 SQL rows for 50 HTTP items
+235
View File
@@ -0,0 +1,235 @@
{
"messages_in_chat": 10000,
"page_limit": 50,
"index": "messages_chat_created_id_idx",
"cases": {
"first": {
"after_message": 0,
"sql": "SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE chat_id=%s ORDER BY created_at, id LIMIT %s",
"plan": {
"Plan": {
"Node Type": "Limit",
"Parallel Aware": false,
"Async Capable": false,
"Startup Cost": 0.29,
"Total Cost": 5.85,
"Plan Rows": 51,
"Plan Width": 71,
"Actual Startup Time": 0.024,
"Actual Total Time": 0.033,
"Actual Rows": 51,
"Actual Loops": 1,
"Shared Hit Blocks": 3,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0,
"Plans": [
{
"Node Type": "Index Scan",
"Parent Relationship": "Outer",
"Parallel Aware": false,
"Async Capable": false,
"Scan Direction": "Forward",
"Index Name": "messages_chat_created_id_idx",
"Relation Name": "messages",
"Alias": "messages",
"Startup Cost": 0.29,
"Total Cost": 1091.76,
"Plan Rows": 10000,
"Plan Width": 71,
"Actual Startup Time": 0.023,
"Actual Total Time": 0.029,
"Actual Rows": 51,
"Actual Loops": 1,
"Index Cond": "(chat_id = '77be5bc0-11e0-4330-a660-198fe31253f8'::uuid)",
"Rows Removed by Index Recheck": 0,
"Shared Hit Blocks": 3,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0
}
]
},
"Planning": {
"Shared Hit Blocks": 117,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0
},
"Planning Time": 0.485,
"Triggers": [],
"Execution Time": 0.068
}
},
"middle": {
"after_message": 5000,
"sql": "SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE chat_id=%s AND (created_at, id) > (%s, %s) ORDER BY created_at, id LIMIT %s",
"plan": {
"Plan": {
"Node Type": "Limit",
"Parallel Aware": false,
"Async Capable": false,
"Startup Cost": 0.29,
"Total Cost": 7.76,
"Plan Rows": 51,
"Plan Width": 71,
"Actual Startup Time": 0.018,
"Actual Total Time": 0.034,
"Actual Rows": 51,
"Actual Loops": 1,
"Shared Hit Blocks": 3,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0,
"Plans": [
{
"Node Type": "Index Scan",
"Parent Relationship": "Outer",
"Parallel Aware": false,
"Async Capable": false,
"Scan Direction": "Forward",
"Index Name": "messages_chat_created_id_idx",
"Relation Name": "messages",
"Alias": "messages",
"Startup Cost": 0.29,
"Total Cost": 707.54,
"Plan Rows": 4827,
"Plan Width": 71,
"Actual Startup Time": 0.017,
"Actual Total Time": 0.027,
"Actual Rows": 51,
"Actual Loops": 1,
"Index Cond": "((chat_id = '77be5bc0-11e0-4330-a660-198fe31253f8'::uuid) AND (ROW(created_at, id) > ROW('2026-11-01 13:55:23.37413+00'::timestamp with time zone, '8527233a-bd37-41c0-92ab-a83710dbaf42'::uuid)))",
"Rows Removed by Index Recheck": 0,
"Shared Hit Blocks": 3,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0
}
]
},
"Planning": {
"Shared Hit Blocks": 0,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0
},
"Planning Time": 0.121,
"Triggers": [],
"Execution Time": 0.082
}
},
"late": {
"after_message": 9900,
"sql": "SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE chat_id=%s AND (created_at, id) > (%s, %s) ORDER BY created_at, id LIMIT %s",
"plan": {
"Plan": {
"Node Type": "Limit",
"Parallel Aware": false,
"Async Capable": false,
"Startup Cost": 0.29,
"Total Cost": 87.97,
"Plan Rows": 51,
"Plan Width": 71,
"Actual Startup Time": 0.016,
"Actual Total Time": 0.024,
"Actual Rows": 51,
"Actual Loops": 1,
"Shared Hit Blocks": 4,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0,
"Plans": [
{
"Node Type": "Index Scan",
"Parent Relationship": "Outer",
"Parallel Aware": false,
"Async Capable": false,
"Scan Direction": "Forward",
"Index Name": "messages_chat_created_id_idx",
"Relation Name": "messages",
"Alias": "messages",
"Startup Cost": 0.29,
"Total Cost": 165.33,
"Plan Rows": 96,
"Plan Width": 71,
"Actual Startup Time": 0.016,
"Actual Total Time": 0.02,
"Actual Rows": 51,
"Actual Loops": 1,
"Index Cond": "((chat_id = '77be5bc0-11e0-4330-a660-198fe31253f8'::uuid) AND (ROW(created_at, id) > ROW('2026-11-01 13:55:23.37903+00'::timestamp with time zone, '2cb28284-13ea-47b3-9eda-45aeb4bda794'::uuid)))",
"Rows Removed by Index Recheck": 0,
"Shared Hit Blocks": 4,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0
}
]
},
"Planning": {
"Shared Hit Blocks": 0,
"Shared Read Blocks": 0,
"Shared Dirtied Blocks": 0,
"Shared Written Blocks": 0,
"Local Hit Blocks": 0,
"Local Read Blocks": 0,
"Local Dirtied Blocks": 0,
"Local Written Blocks": 0,
"Temp Read Blocks": 0,
"Temp Written Blocks": 0
},
"Planning Time": 0.073,
"Triggers": [],
"Execution Time": 0.035
}
}
}
}
+29
View File
@@ -0,0 +1,29 @@
FIRST
SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE chat_id=%s ORDER BY created_at, id LIMIT %s
Limit (cost=0.29..5.85 rows=51 width=71) (actual time=0.020..0.028 rows=51 loops=1)
Buffers: shared hit=3
-> Index Scan using messages_chat_created_id_idx on messages (cost=0.29..1091.76 rows=10000 width=71) (actual time=0.019..0.024 rows=51 loops=1)
Index Cond: (chat_id = '77be5bc0-11e0-4330-a660-198fe31253f8'::uuid)
Buffers: shared hit=3
Planning Time: 0.093 ms
Execution Time: 0.040 ms
MIDDLE
SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE chat_id=%s AND (created_at, id) > (%s, %s) ORDER BY created_at, id LIMIT %s
Limit (cost=0.29..7.76 rows=51 width=71) (actual time=0.029..0.046 rows=51 loops=1)
Buffers: shared hit=3
-> Index Scan using messages_chat_created_id_idx on messages (cost=0.29..707.54 rows=4827 width=71) (actual time=0.028..0.038 rows=51 loops=1)
Index Cond: ((chat_id = '77be5bc0-11e0-4330-a660-198fe31253f8'::uuid) AND (ROW(created_at, id) > ROW('2026-11-01 13:55:23.37413+00'::timestamp with time zone, '8527233a-bd37-41c0-92ab-a83710dbaf42'::uuid)))
Buffers: shared hit=3
Planning Time: 0.116 ms
Execution Time: 0.067 ms
LATE
SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE chat_id=%s AND (created_at, id) > (%s, %s) ORDER BY created_at, id LIMIT %s
Limit (cost=0.29..87.97 rows=51 width=71) (actual time=0.014..0.023 rows=51 loops=1)
Buffers: shared hit=4
-> Index Scan using messages_chat_created_id_idx on messages (cost=0.29..165.33 rows=96 width=71) (actual time=0.013..0.019 rows=51 loops=1)
Index Cond: ((chat_id = '77be5bc0-11e0-4330-a660-198fe31253f8'::uuid) AND (ROW(created_at, id) > ROW('2026-11-01 13:55:23.37903+00'::timestamp with time zone, '2cb28284-13ea-47b3-9eda-45aeb4bda794'::uuid)))
Buffers: shared hit=4
Planning Time: 0.079 ms
Execution Time: 0.035 ms
+4
View File
@@ -0,0 +1,4 @@
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://lb:8080"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 117, "failed_attempts": 2, "first_recovery_seconds": 2.064}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
+225
View File
@@ -0,0 +1,225 @@
{
"scenario": "failure-api-1",
"utc": "2026-11-01T13:55:54.273960+00:00",
"duration_seconds": 24.041,
"concurrency": 4,
"successful_requests": 2530,
"failed_attempts": 51,
"error_ratio": 0.01976,
"successful_rps": 105.235,
"successful_latency_ms": {
"p50": 32.652,
"p95": 75.027,
"p99": 96.705
},
"first_write_after_action_seconds": 1.715,
"action": {
"started_seconds": 4.1060343000572175,
"detail": {
"container": "56279f26a490e9c62b51ddd24d14d35e26c3069385f3d80d3360d17fa37a5523",
"hostname": "56279f26a490",
"stop_seconds": 1.299,
"exit_code": 0
},
"finished_seconds": 5.409909900045022,
"resources_during_load": [
{
"BlockIO": "0B / 0B",
"CPUPerc": "101.13%",
"Container": "eebe2ed461699e48f987d0f9f51e0331d2f5dcd0e54f6264efd45dab26d24040",
"ID": "eebe2ed46169",
"MemPerc": "27.80%",
"MemUsage": "71.18MiB / 256MiB",
"Name": "mtusi-lab3-api-2",
"NetIO": "4.12MB / 3.61MB",
"PIDs": "4"
},
{
"BlockIO": "0B / 422kB",
"CPUPerc": "27.80%",
"Container": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"ID": "62fe1bac69db",
"MemPerc": "7.14%",
"MemUsage": "36.57MiB / 512MiB",
"Name": "mtusi-lab3-postgres-1",
"NetIO": "3.27MB / 3.98MB",
"PIDs": "11"
},
{
"BlockIO": "0B / 24.6kB",
"CPUPerc": "4.90%",
"Container": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"ID": "8ce27a874af3",
"MemPerc": "1.92%",
"MemUsage": "3.695MiB / 192MiB",
"Name": "mtusi-lab3-redis-1",
"NetIO": "274kB / 236kB",
"PIDs": "6"
},
{
"BlockIO": "0B / 0B",
"CPUPerc": "14.96%",
"Container": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"ID": "c28b6968210f",
"MemPerc": "11.01%",
"MemUsage": "14.09MiB / 128MiB",
"Name": "mtusi-lab3-lb-1",
"NetIO": "2.36MB / 2.45MB",
"PIDs": "17"
}
]
},
"confirmed_writes_checked": 2350,
"history_messages": 2350,
"unique_history_messages": 2350,
"instances_before": [
"56279f26a490-6279063680e4",
"eebe2ed46169-28ea6dbaf5ae"
],
"series_2_seconds": [
{
"start_seconds": 0,
"rps": 147.5,
"errors": 0
},
{
"start_seconds": 2,
"rps": 154.0,
"errors": 0
},
{
"start_seconds": 4,
"rps": 43.5,
"errors": 51
},
{
"start_seconds": 6,
"rps": 114.0,
"errors": 0
},
{
"start_seconds": 8,
"rps": 105.0,
"errors": 0
},
{
"start_seconds": 10,
"rps": 94.0,
"errors": 0
},
{
"start_seconds": 12,
"rps": 121.5,
"errors": 0
},
{
"start_seconds": 14,
"rps": 96.0,
"errors": 0
},
{
"start_seconds": 16,
"rps": 78.0,
"errors": 0
},
{
"start_seconds": 18,
"rps": 93.0,
"errors": 0
},
{
"start_seconds": 20,
"rps": 115.0,
"errors": 0
},
{
"start_seconds": 22,
"rps": 101.5,
"errors": 0
},
{
"start_seconds": 24,
"rps": 2.0,
"errors": 0
}
],
"deployment_after": [
{
"service": "api",
"id": "56279f26a490e9c62b51ddd24d14d35e26c3069385f3d80d3360d17fa37a5523",
"name": "mtusi-lab3-api-1",
"started_at": "2026-11-01T13:55:11.890726782Z",
"status": "exited",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "api",
"id": "eebe2ed461699e48f987d0f9f51e0331d2f5dcd0e54f6264efd45dab26d24040",
"name": "mtusi-lab3-api-2",
"started_at": "2026-11-01T13:54:45.672968999Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "postgres",
"id": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"name": "mtusi-lab3-postgres-1",
"started_at": "2026-11-01T13:54:45.014430269Z",
"status": "running",
"image": "sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3",
"user": "",
"memory_bytes": 536870912,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_postgres_data"
]
},
{
"service": "redis",
"id": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"name": "mtusi-lab3-redis-1",
"started_at": "2026-11-01T13:54:44.838374892Z",
"status": "running",
"image": "sha256:90e7a336d044f1abc9e9dbc05d65566850896d11453bbd1dd0fb7e5059f0e8fb",
"user": "",
"memory_bytes": 201326592,
"nano_cpus": 500000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_redis_data"
]
},
{
"service": "lb",
"id": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"name": "mtusi-lab3-lb-1",
"started_at": "2026-11-01T13:54:45.212099737Z",
"status": "running",
"image": "sha256:e3229153dd6429fc71f8a9a1f1ccbf8dab6962422d7043c81e84dd6a075e90ee",
"user": "haproxy",
"memory_bytes": 134217728,
"nano_cpus": 500000000,
"published_ports": {
"8080/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "8080"
}
]
},
"volumes": []
}
],
"note": "Closed-loop mixed HTTP load, 4 clients; failures are attempts, retries retain keys; latency covers successes only."
}
+225
View File
@@ -0,0 +1,225 @@
{
"scenario": "failure-api-2",
"utc": "2026-11-01T13:56:24.941179+00:00",
"duration_seconds": 24.034,
"concurrency": 4,
"successful_requests": 2614,
"failed_attempts": 57,
"error_ratio": 0.02134,
"successful_rps": 108.762,
"successful_latency_ms": {
"p50": 32.088,
"p95": 65.561,
"p99": 97.489
},
"first_write_after_action_seconds": 1.697,
"action": {
"started_seconds": 4.083422699943185,
"detail": {
"container": "eebe2ed461699e48f987d0f9f51e0331d2f5dcd0e54f6264efd45dab26d24040",
"hostname": "eebe2ed46169",
"stop_seconds": 1.24,
"exit_code": 0
},
"finished_seconds": 5.323327000020072,
"resources_during_load": [
{
"BlockIO": "0B / 0B",
"CPUPerc": "98.17%",
"Container": "56279f26a490e9c62b51ddd24d14d35e26c3069385f3d80d3360d17fa37a5523",
"ID": "56279f26a490",
"MemPerc": "23.93%",
"MemUsage": "61.27MiB / 256MiB",
"Name": "mtusi-lab3-api-1",
"NetIO": "3.35MB / 2.87MB",
"PIDs": "2"
},
{
"BlockIO": "0B / 34.2MB",
"CPUPerc": "41.46%",
"Container": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"ID": "62fe1bac69db",
"MemPerc": "7.64%",
"MemUsage": "39.11MiB / 512MiB",
"Name": "mtusi-lab3-postgres-1",
"NetIO": "9.11MB / 11.6MB",
"PIDs": "11"
},
{
"BlockIO": "0B / 41kB",
"CPUPerc": "4.70%",
"Container": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"ID": "8ce27a874af3",
"MemPerc": "1.96%",
"MemUsage": "3.762MiB / 192MiB",
"Name": "mtusi-lab3-redis-1",
"NetIO": "692kB / 625kB",
"PIDs": "6"
},
{
"BlockIO": "0B / 0B",
"CPUPerc": "17.56%",
"Container": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"ID": "c28b6968210f",
"MemPerc": "11.73%",
"MemUsage": "15.02MiB / 128MiB",
"Name": "mtusi-lab3-lb-1",
"NetIO": "7.23MB / 7.58MB",
"PIDs": "17"
}
]
},
"confirmed_writes_checked": 2429,
"history_messages": 2429,
"unique_history_messages": 2429,
"instances_before": [
"eebe2ed46169-28ea6dbaf5ae",
"56279f26a490-b1a72fdd0fd8"
],
"series_2_seconds": [
{
"start_seconds": 0,
"rps": 145.5,
"errors": 0
},
{
"start_seconds": 2,
"rps": 135.5,
"errors": 0
},
{
"start_seconds": 4,
"rps": 56.5,
"errors": 57
},
{
"start_seconds": 6,
"rps": 113.5,
"errors": 0
},
{
"start_seconds": 8,
"rps": 87.5,
"errors": 0
},
{
"start_seconds": 10,
"rps": 118.0,
"errors": 0
},
{
"start_seconds": 12,
"rps": 118.5,
"errors": 0
},
{
"start_seconds": 14,
"rps": 95.5,
"errors": 0
},
{
"start_seconds": 16,
"rps": 116.5,
"errors": 0
},
{
"start_seconds": 18,
"rps": 100.5,
"errors": 0
},
{
"start_seconds": 20,
"rps": 106.0,
"errors": 0
},
{
"start_seconds": 22,
"rps": 111.5,
"errors": 0
},
{
"start_seconds": 24,
"rps": 2.0,
"errors": 0
}
],
"deployment_after": [
{
"service": "api",
"id": "56279f26a490e9c62b51ddd24d14d35e26c3069385f3d80d3360d17fa37a5523",
"name": "mtusi-lab3-api-1",
"started_at": "2026-11-01T13:55:57.397791954Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "api",
"id": "eebe2ed461699e48f987d0f9f51e0331d2f5dcd0e54f6264efd45dab26d24040",
"name": "mtusi-lab3-api-2",
"started_at": "2026-11-01T13:54:45.672968999Z",
"status": "exited",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "postgres",
"id": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"name": "mtusi-lab3-postgres-1",
"started_at": "2026-11-01T13:54:45.014430269Z",
"status": "running",
"image": "sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3",
"user": "",
"memory_bytes": 536870912,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_postgres_data"
]
},
{
"service": "redis",
"id": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"name": "mtusi-lab3-redis-1",
"started_at": "2026-11-01T13:54:44.838374892Z",
"status": "running",
"image": "sha256:90e7a336d044f1abc9e9dbc05d65566850896d11453bbd1dd0fb7e5059f0e8fb",
"user": "",
"memory_bytes": 201326592,
"nano_cpus": 500000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_redis_data"
]
},
{
"service": "lb",
"id": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"name": "mtusi-lab3-lb-1",
"started_at": "2026-11-01T13:54:45.212099737Z",
"status": "running",
"image": "sha256:e3229153dd6429fc71f8a9a1f1ccbf8dab6962422d7043c81e84dd6a075e90ee",
"user": "haproxy",
"memory_bytes": 134217728,
"nano_cpus": 500000000,
"published_ports": {
"8080/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "8080"
}
]
},
"volumes": []
}
],
"note": "Closed-loop mixed HTTP load, 4 clients; failures are attempts, retries retain keys; latency covers successes only."
}
+14
View File
@@ -0,0 +1,14 @@
{
"redis": {
"ready_status": 503,
"protected_status": 503,
"elapsed_seconds": 2.003,
"anonymous_status": 401
},
"postgres": {
"ready_status": 503,
"protected_status": 503,
"elapsed_seconds": 1.006,
"anonymous_status": 401
}
}
+27
View File
@@ -0,0 +1,27 @@
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
python3 tests/smoke.py --base-url "http://lb:8080"
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
python3 -m unittest discover -s student-tests -v
ROLLBACK: invalid second member left no chat or membership rows
test_composite_chat_rollback (test_api.APITests.test_composite_chat_rollback) ... ok
CONCURRENCY: 10 clients, 100 confirmations, 100 unique messages
test_concurrent_messages (test_api.APITests.test_concurrent_messages) ... ok
test_cursor_binding_and_chat_pages (test_api.APITests.test_cursor_binding_and_chat_pages) ... ok
test_dependency_failures (test_api.APITests.test_dependency_failures) ... ok
FAILURES: Redis/PostgreSQL -> bounded 503, live 200, anonymous 401, recovery with old session
test_passwords_and_authentication (test_api.APITests.test_passwords_and_authentication) ... ok
REDIS CRASH: old session survived SIGKILL/AOF replay, expiry deadline unchanged
test_redis_crash_durability (test_api.APITests.test_redis_crash_durability) ... ok
RACE: 10 requests, 1 created, 9 conflicts, 1 database row
test_registration_race (test_api.APITests.test_registration_race) ... ok
test_validation_and_access (test_api.APITests.test_validation_and_access) ... ok
test_expired_key_and_same_session_on_all_instances (test_replication.ReplicationTests.test_expired_key_and_same_session_on_all_instances) ... ok
test_failed_transaction_does_not_save_message_or_replay (test_replication.ReplicationTests.test_failed_transaction_does_not_save_message_or_replay) ... ok
test_key_validation_scope_conflict_and_logout (test_replication.ReplicationTests.test_key_validation_scope_conflict_and_logout) ... ok
test_parallel_replay_across_real_replicas (test_replication.ReplicationTests.test_parallel_replay_across_real_replicas) ... ok
----------------------------------------------------------------------
Ran 12 tests in 18.855s
OK
+9
View File
@@ -0,0 +1,9 @@
{
"requests": 100,
"clients": 10,
"real_containers": 2,
"observed_instances": 2,
"messages": 1,
"replay_records": 1,
"lb_replays": 12
}
+7
View File
@@ -0,0 +1,7 @@
{
"target_instance": "56279f26a490-cd4fe5da999e",
"container": "56279f26a490e9c62b51ddd24d14d35e26c3069385f3d80d3360d17fa37a5523",
"exit_code": 0,
"stop_seconds": 0.774,
"remains_stopped": true
}
+3
View File
@@ -0,0 +1,3 @@
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://lb:8080"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
+1
View File
@@ -0,0 +1 @@
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
+158
View File
@@ -0,0 +1,158 @@
{
"before": [
{
"service": "api",
"id": "56279f26a490e9c62b51ddd24d14d35e26c3069385f3d80d3360d17fa37a5523",
"name": "mtusi-lab3-api-1",
"started_at": "2026-11-01T13:52:02.178367306Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "api",
"id": "eebe2ed461699e48f987d0f9f51e0331d2f5dcd0e54f6264efd45dab26d24040",
"name": "mtusi-lab3-api-2",
"started_at": "2026-11-01T13:52:02.607123306Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "postgres",
"id": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"name": "mtusi-lab3-postgres-1",
"started_at": "2026-11-01T13:54:30.359656882Z",
"status": "running",
"image": "sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3",
"user": "",
"memory_bytes": 536870912,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_postgres_data"
]
},
{
"service": "redis",
"id": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"name": "mtusi-lab3-redis-1",
"started_at": "2026-11-01T13:54:33.555645547Z",
"status": "running",
"image": "sha256:90e7a336d044f1abc9e9dbc05d65566850896d11453bbd1dd0fb7e5059f0e8fb",
"user": "",
"memory_bytes": 201326592,
"nano_cpus": 500000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_redis_data"
]
},
{
"service": "lb",
"id": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"name": "mtusi-lab3-lb-1",
"started_at": "2026-11-01T13:54:10.199664374Z",
"status": "running",
"image": "sha256:e3229153dd6429fc71f8a9a1f1ccbf8dab6962422d7043c81e84dd6a075e90ee",
"user": "haproxy",
"memory_bytes": 134217728,
"nano_cpus": 500000000,
"published_ports": {
"8080/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "8080"
}
]
},
"volumes": []
}
],
"after": [
{
"service": "api",
"id": "56279f26a490e9c62b51ddd24d14d35e26c3069385f3d80d3360d17fa37a5523",
"name": "mtusi-lab3-api-1",
"started_at": "2026-11-01T13:54:45.563795974Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "api",
"id": "eebe2ed461699e48f987d0f9f51e0331d2f5dcd0e54f6264efd45dab26d24040",
"name": "mtusi-lab3-api-2",
"started_at": "2026-11-01T13:54:45.672968999Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "postgres",
"id": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"name": "mtusi-lab3-postgres-1",
"started_at": "2026-11-01T13:54:45.014430269Z",
"status": "running",
"image": "sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3",
"user": "",
"memory_bytes": 536870912,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_postgres_data"
]
},
{
"service": "redis",
"id": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"name": "mtusi-lab3-redis-1",
"started_at": "2026-11-01T13:54:44.838374892Z",
"status": "running",
"image": "sha256:90e7a336d044f1abc9e9dbc05d65566850896d11453bbd1dd0fb7e5059f0e8fb",
"user": "",
"memory_bytes": 201326592,
"nano_cpus": 500000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_redis_data"
]
},
{
"service": "lb",
"id": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"name": "mtusi-lab3-lb-1",
"started_at": "2026-11-01T13:54:45.212099737Z",
"status": "running",
"image": "sha256:e3229153dd6429fc71f8a9a1f1ccbf8dab6962422d7043c81e84dd6a075e90ee",
"user": "haproxy",
"memory_bytes": 134217728,
"nano_cpus": 500000000,
"published_ports": {
"8080/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "8080"
}
]
},
"volumes": []
}
]
}
+298
View File
@@ -0,0 +1,298 @@
{
"scenario": "rolling-update",
"utc": "2026-11-01T13:57:17.630617+00:00",
"duration_seconds": 45.015,
"concurrency": 4,
"successful_requests": 4829,
"failed_attempts": 0,
"error_ratio": 0.0,
"successful_rps": 107.276,
"successful_latency_ms": {
"p50": 28.249,
"p95": 82.882,
"p99": 127.075
},
"first_write_after_action_seconds": 23.527,
"action": {
"started_seconds": 4.1119035999290645,
"detail": {
"updates": [
{
"old_container": "56279f26a490e9c62b51ddd24d14d35e26c3069385f3d80d3360d17fa37a5523",
"new_container": "918d3851dffce737d07c642d85b7f3332167a48c66314296605ccc0164ec97c5",
"backend_slot": "messenger/api1",
"new_image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8"
},
{
"old_container": "eebe2ed461699e48f987d0f9f51e0331d2f5dcd0e54f6264efd45dab26d24040",
"new_container": "113f897e60a031968c8eb7db3ca995df858af91ea8e49a23a9e74bbe896e3705",
"backend_slot": "messenger/api2",
"new_image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8"
}
],
"migration": "Unchanged: 001_initial.sql\nUnchanged: 002_message_requests.sql\nPASS: migrations committed",
"method": "drain, stop, remove and recreate each container; same application image, new APP_VERSION configuration"
},
"finished_seconds": 27.62092129990924,
"resources_during_load": [
{
"BlockIO": "0B / 0B",
"CPUPerc": "75.87%",
"Container": "113f897e60a031968c8eb7db3ca995df858af91ea8e49a23a9e74bbe896e3705",
"ID": "113f897e60a0",
"MemPerc": "17.73%",
"MemUsage": "45.39MiB / 256MiB",
"Name": "mtusi-lab3-api-4",
"NetIO": "1.82MB / 1.57MB",
"PIDs": "2"
},
{
"BlockIO": "0B / 0B",
"CPUPerc": "79.24%",
"Container": "918d3851dffce737d07c642d85b7f3332167a48c66314296605ccc0164ec97c5",
"ID": "918d3851dffc",
"MemPerc": "17.90%",
"MemUsage": "45.82MiB / 256MiB",
"Name": "mtusi-lab3-api-3",
"NetIO": "4.12MB / 3.52MB",
"PIDs": "2"
},
{
"BlockIO": "0B / 76.6MB",
"CPUPerc": "61.03%",
"Container": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"ID": "62fe1bac69db",
"MemPerc": "9.98%",
"MemUsage": "51.12MiB / 512MiB",
"Name": "mtusi-lab3-postgres-1",
"NetIO": "19.9MB / 25.3MB",
"PIDs": "15"
},
{
"BlockIO": "0B / 61.4kB",
"CPUPerc": "3.11%",
"Container": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"ID": "8ce27a874af3",
"MemPerc": "1.97%",
"MemUsage": "3.785MiB / 192MiB",
"Name": "mtusi-lab3-redis-1",
"NetIO": "1.5MB / 1.37MB",
"PIDs": "6"
},
{
"BlockIO": "0B / 0B",
"CPUPerc": "23.63%",
"Container": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"ID": "c28b6968210f",
"MemPerc": "13.55%",
"MemUsage": "17.34MiB / 128MiB",
"Name": "mtusi-lab3-lb-1",
"NetIO": "15.4MB / 16.3MB",
"PIDs": "17"
}
]
},
"confirmed_writes_checked": 4475,
"history_messages": 4475,
"unique_history_messages": 4475,
"instances_before": [
"eebe2ed46169-7f2687009ab0",
"56279f26a490-b1a72fdd0fd8"
],
"series_2_seconds": [
{
"start_seconds": 0,
"rps": 138.5,
"errors": 0
},
{
"start_seconds": 2,
"rps": 129.5,
"errors": 0
},
{
"start_seconds": 4,
"rps": 116.0,
"errors": 0
},
{
"start_seconds": 6,
"rps": 108.5,
"errors": 0
},
{
"start_seconds": 8,
"rps": 77.5,
"errors": 0
},
{
"start_seconds": 10,
"rps": 93.5,
"errors": 0
},
{
"start_seconds": 12,
"rps": 62.5,
"errors": 0
},
{
"start_seconds": 14,
"rps": 52.5,
"errors": 0
},
{
"start_seconds": 16,
"rps": 86.0,
"errors": 0
},
{
"start_seconds": 18,
"rps": 50.5,
"errors": 0
},
{
"start_seconds": 20,
"rps": 80.0,
"errors": 0
},
{
"start_seconds": 22,
"rps": 79.0,
"errors": 0
},
{
"start_seconds": 24,
"rps": 52.5,
"errors": 0
},
{
"start_seconds": 26,
"rps": 102.0,
"errors": 0
},
{
"start_seconds": 28,
"rps": 119.5,
"errors": 0
},
{
"start_seconds": 30,
"rps": 121.5,
"errors": 0
},
{
"start_seconds": 32,
"rps": 162.5,
"errors": 0
},
{
"start_seconds": 34,
"rps": 139.5,
"errors": 0
},
{
"start_seconds": 36,
"rps": 122.5,
"errors": 0
},
{
"start_seconds": 38,
"rps": 160.0,
"errors": 0
},
{
"start_seconds": 40,
"rps": 139.5,
"errors": 0
},
{
"start_seconds": 42,
"rps": 138.0,
"errors": 0
},
{
"start_seconds": 44,
"rps": 83.0,
"errors": 0
}
],
"deployment_after": [
{
"service": "api",
"id": "113f897e60a031968c8eb7db3ca995df858af91ea8e49a23a9e74bbe896e3705",
"name": "mtusi-lab3-api-4",
"started_at": "2026-11-01T13:56:54.417531752Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "api",
"id": "918d3851dffce737d07c642d85b7f3332167a48c66314296605ccc0164ec97c5",
"name": "mtusi-lab3-api-3",
"started_at": "2026-11-01T13:56:44.055659428Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "postgres",
"id": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"name": "mtusi-lab3-postgres-1",
"started_at": "2026-11-01T13:54:45.014430269Z",
"status": "running",
"image": "sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3",
"user": "",
"memory_bytes": 536870912,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_postgres_data"
]
},
{
"service": "redis",
"id": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"name": "mtusi-lab3-redis-1",
"started_at": "2026-11-01T13:54:44.838374892Z",
"status": "running",
"image": "sha256:90e7a336d044f1abc9e9dbc05d65566850896d11453bbd1dd0fb7e5059f0e8fb",
"user": "",
"memory_bytes": 201326592,
"nano_cpus": 500000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_redis_data"
]
},
{
"service": "lb",
"id": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"name": "mtusi-lab3-lb-1",
"started_at": "2026-11-01T13:54:45.212099737Z",
"status": "running",
"image": "sha256:e3229153dd6429fc71f8a9a1f1ccbf8dab6962422d7043c81e84dd6a075e90ee",
"user": "haproxy",
"memory_bytes": 134217728,
"nano_cpus": 500000000,
"published_ports": {
"8080/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "8080"
}
]
},
"volumes": []
}
],
"note": "Closed-loop mixed HTTP load, 4 clients; failures are attempts, retries retain keys; latency covers successes only."
}
+5
View File
@@ -0,0 +1,5 @@
Checking contract, HTTP API and student tests
Running public failover scenario
Checking short session expiry and indexed history
Measuring failure of each replica and sequential replacement under load
PASS: lab3 checks completed; evidence/verification.json
+2
View File
@@ -0,0 +1,2 @@
python3 tests/smoke.py --base-url "http://lb:8080"
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
+2
View File
@@ -0,0 +1,2 @@
python3 tests/smoke.py --base-url "http://lb:8080"
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
+21
View File
@@ -0,0 +1,21 @@
{
"baseline_commit": "0ab3a341aa7ba32363ce43061d4ff02b1f2964af",
"normalized_eol": "LF",
"unchanged_files": {
"CONTRIBUTING.md": "2c01dd698cc9653dece49a94c9fd3f58d36e84764aaa643b46e26d7e599dc12e",
"COURSE.md": "c34d53cd082d194aa4b58f504c4e0348d25b6afec3d6a9c0963dd01c9c81e971",
"README.md": "7622c956ad4508d10d1807b4190461e3a7911a22759108ecf812d5bc477ff7a2",
"contracts/README.md": "72df27a559f88ffa3fd8c3c79d3c32f6db2c5986a0602677c51994c434a3fc4f",
"contracts/examples.json": "0a71cec72903af556df85f939c7e347fc73a86a1a4ffc39fb9c8ded169b7df45",
"contracts/infrastructure.json": "99320becab20381c3d31983336dd4e0aef4bca992d672538f049b6684714a577",
"contracts/openapi.json": "9c3787e5d08a60c59ae67eeb429e9d75fbf134992a4e1f0954e926db49e111be",
"lab.json": "acf873f5132bbd0af78dc878852baeb49dbfa026c072932f8bd70b958114e938",
"scripts/check_contract.py": "7c4402c2a6ab80d3b4676e1866d6ecc87b6daea14c080bcb9be9a7e8b959d671",
"scripts/validate_openapi.py": "4e981acbfc10dac58817bf922d5896e996798551fedc1b2385130e1cddb13d54",
"tests/README.md": "e0bfa7e196797f6a5bf6e44d8ad2ffeb82651927ff5d8484eb1c2dda1ec7dec5",
"tests/client.py": "8d50976fc7c7bbb81e88a152f613c61ab25eaf53dba33c6a9dfab19f868d546d",
"tests/load.py": "6d3511ea719c7a7e841083e73d435c86dcbfd9f4e5c2a07d62eb9614b01d9933",
"tests/resilience.py": "1f96663a174b3978d24eff66f2741f5f8c9b788d3242261d37a972720c52788b",
"tests/smoke.py": "52ac38eb343ff7d43e7543588e1da58ca06c9fce74b2b0d0bf0e4982bc23c1db"
}
}
+32
View File
@@ -0,0 +1,32 @@
Container mtusi-lab3-redis-1 Running
Container mtusi-lab3-postgres-1 Running
Container mtusi-lab3-api-1 Running
Container mtusi-lab3-api-2 Running
Container mtusi-lab3-lb-1 Recreate
Container mtusi-lab3-lb-1 Recreated
Container mtusi-lab3-postgres-1 Waiting
Container mtusi-lab3-postgres-1 Healthy
Container mtusi-lab3-migrate-1 Starting
Container mtusi-lab3-migrate-1 Started
Container mtusi-lab3-migrate-1 Waiting
Container mtusi-lab3-redis-1 Waiting
Container mtusi-lab3-redis-1 Healthy
Container mtusi-lab3-migrate-1 Exited
Container mtusi-lab3-api-1 Waiting
Container mtusi-lab3-api-2 Waiting
Container mtusi-lab3-api-1 Healthy
Container mtusi-lab3-api-2 Healthy
Container mtusi-lab3-lb-1 Starting
Container mtusi-lab3-lb-1 Started
Container mtusi-lab3-api-1 Waiting
Container mtusi-lab3-api-2 Waiting
Container mtusi-lab3-lb-1 Waiting
Container mtusi-lab3-postgres-1 Waiting
Container mtusi-lab3-redis-1 Waiting
Container mtusi-lab3-migrate-1 Waiting
Container mtusi-lab3-redis-1 Healthy
Container mtusi-lab3-postgres-1 Healthy
Container mtusi-lab3-api-1 Healthy
Container mtusi-lab3-api-2 Healthy
Container mtusi-lab3-migrate-1 Exited
Container mtusi-lab3-lb-1 Healthy
+27
View File
@@ -0,0 +1,27 @@
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
python3 tests/smoke.py --base-url "http://lb:8080"
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
python3 -m unittest discover -s student-tests -v
ROLLBACK: invalid second member left no chat or membership rows
test_composite_chat_rollback (test_api.APITests.test_composite_chat_rollback) ... ok
CONCURRENCY: 10 clients, 100 confirmations, 100 unique messages
test_concurrent_messages (test_api.APITests.test_concurrent_messages) ... ok
test_cursor_binding_and_chat_pages (test_api.APITests.test_cursor_binding_and_chat_pages) ... ok
FAILURES: Redis/PostgreSQL -> bounded 503, live 200, anonymous 401, recovery with old session
test_dependency_failures (test_api.APITests.test_dependency_failures) ... ok
test_passwords_and_authentication (test_api.APITests.test_passwords_and_authentication) ... ok
REDIS CRASH: old session survived SIGKILL/AOF replay, expiry deadline unchanged
test_redis_crash_durability (test_api.APITests.test_redis_crash_durability) ... ok
RACE: 10 requests, 1 created, 9 conflicts, 1 database row
test_registration_race (test_api.APITests.test_registration_race) ... ok
test_validation_and_access (test_api.APITests.test_validation_and_access) ... ok
test_expired_key_and_same_session_on_all_instances (test_replication.ReplicationTests.test_expired_key_and_same_session_on_all_instances) ... ok
test_failed_transaction_does_not_save_message_or_replay (test_replication.ReplicationTests.test_failed_transaction_does_not_save_message_or_replay) ... ok
test_key_validation_scope_conflict_and_logout (test_replication.ReplicationTests.test_key_validation_scope_conflict_and_logout) ... ok
test_parallel_replay_across_real_replicas (test_replication.ReplicationTests.test_parallel_replay_across_real_replicas) ... ok
----------------------------------------------------------------------
Ran 12 tests in 19.357s
OK
+23
View File
@@ -0,0 +1,23 @@
python3 -m unittest discover -s student-tests -v
ROLLBACK: invalid second member left no chat or membership rows
CONCURRENCY: 10 clients, 100 confirmations, 100 unique messages
FAILURES: Redis/PostgreSQL -> bounded 503, live 200, anonymous 401, recovery with old session
REDIS CRASH: old session survived SIGKILL/AOF replay, expiry deadline unchanged
RACE: 10 requests, 1 created, 9 conflicts, 1 database row
test_composite_chat_rollback (test_api.APITests.test_composite_chat_rollback) ... ok
test_concurrent_messages (test_api.APITests.test_concurrent_messages) ... ok
test_cursor_binding_and_chat_pages (test_api.APITests.test_cursor_binding_and_chat_pages) ... ok
test_dependency_failures (test_api.APITests.test_dependency_failures) ... ok
test_passwords_and_authentication (test_api.APITests.test_passwords_and_authentication) ... ok
test_redis_crash_durability (test_api.APITests.test_redis_crash_durability) ... ok
test_registration_race (test_api.APITests.test_registration_race) ... ok
test_validation_and_access (test_api.APITests.test_validation_and_access) ... ok
test_expired_key_and_same_session_on_all_instances (test_replication.ReplicationTests.test_expired_key_and_same_session_on_all_instances) ... ok
test_failed_transaction_does_not_save_message_or_replay (test_replication.ReplicationTests.test_failed_transaction_does_not_save_message_or_replay) ... ok
test_key_validation_scope_conflict_and_logout (test_replication.ReplicationTests.test_key_validation_scope_conflict_and_logout) ... ok
test_parallel_replay_across_real_replicas (test_replication.ReplicationTests.test_parallel_replay_across_real_replicas) ... ok
----------------------------------------------------------------------
Ran 12 tests in 17.871s
OK
+101
View File
@@ -0,0 +1,101 @@
{
"result": "PASS",
"utc": "2026-11-01T13:57:24.463073+00:00",
"base_url": "http://127.0.0.1:8080",
"checks": [
"public_smoke",
"student_tests",
"full_restart",
"public_failover",
"expiry",
"index_10000",
"published_smoke",
"each_replica_failure",
"sequential_replacement_under_load"
],
"docker": {
"ServerVersion": "28.5.1",
"OperatingSystem": "Docker Desktop",
"Architecture": "x86_64",
"NCPU": 16,
"MemTotal": 8017608704
},
"deployment": [
{
"service": "api",
"id": "113f897e60a031968c8eb7db3ca995df858af91ea8e49a23a9e74bbe896e3705",
"name": "mtusi-lab3-api-4",
"started_at": "2026-11-01T13:56:54.417531752Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "api",
"id": "918d3851dffce737d07c642d85b7f3332167a48c66314296605ccc0164ec97c5",
"name": "mtusi-lab3-api-3",
"started_at": "2026-11-01T13:56:44.055659428Z",
"status": "running",
"image": "sha256:d37648c74d3fad854c3620bdfe611a43dd76c7aef672b0045759d82876a27dd8",
"user": "messenger",
"memory_bytes": 268435456,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": []
},
{
"service": "postgres",
"id": "62fe1bac69db36ae9f836e897aab943cb4abdc0cb891b542a6cd2c50f9320a17",
"name": "mtusi-lab3-postgres-1",
"started_at": "2026-11-01T13:54:45.014430269Z",
"status": "running",
"image": "sha256:f3bd19c606e442c3d7bdfa8002e03fe260a1023351e0ea4598032022b68dd6e3",
"user": "",
"memory_bytes": 536870912,
"nano_cpus": 1000000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_postgres_data"
]
},
{
"service": "redis",
"id": "8ce27a874af3533ce9fc95ca050970c901ec67bd9ed8b0956e2367ff1e0b9dd7",
"name": "mtusi-lab3-redis-1",
"started_at": "2026-11-01T13:54:44.838374892Z",
"status": "running",
"image": "sha256:90e7a336d044f1abc9e9dbc05d65566850896d11453bbd1dd0fb7e5059f0e8fb",
"user": "",
"memory_bytes": 201326592,
"nano_cpus": 500000000,
"published_ports": {},
"volumes": [
"mtusi-lab3_redis_data"
]
},
{
"service": "lb",
"id": "c28b6968210fe5872afd276cb25e31c1356518975896402ab8b49ea984cec4dc",
"name": "mtusi-lab3-lb-1",
"started_at": "2026-11-01T13:54:45.212099737Z",
"status": "running",
"image": "sha256:e3229153dd6429fc71f8a9a1f1ccbf8dab6962422d7043c81e84dd6a075e90ee",
"user": "haproxy",
"memory_bytes": 134217728,
"nano_cpus": 500000000,
"published_ports": {
"8080/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "8080"
}
]
},
"volumes": []
}
]
}
+39
View File
@@ -0,0 +1,39 @@
global
log stdout format raw local0
maxconn 512
stats socket ipv4@0.0.0.0:9999 level admin
defaults
mode http
log global
option httplog
option http-server-close
timeout connect 1s
timeout client 10s
timeout server 6s
timeout http-request 5s
timeout check 1s
retries 0
resolvers docker
nameserver embedded 127.0.0.11:53
resolve_retries 2
timeout resolve 1s
timeout retry 1s
hold valid 1s
hold obsolete 1s
hold nx 1s
hold timeout 1s
hold refused 1s
frontend public
bind :8080
default_backend messenger
backend messenger
balance roundrobin
option httpchk
http-check send meth GET uri /health/ready ver HTTP/1.1 hdr Host api
http-check expect status 200
default-server inter 1s fastinter 500ms downinter 1s fall 2 rise 1
server-template api 8 api:8080 check resolvers docker init-addr none resolve-prefer ipv4
+9
View File
@@ -0,0 +1,9 @@
bind 0.0.0.0
protected-mode yes
port 6379
dir /data
appendonly yes
appendfsync always
save ""
maxmemory 128mb
maxmemory-policy noeviction
+32
View File
@@ -0,0 +1,32 @@
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
display_name text NOT NULL CHECK (char_length(display_name) BETWEEN 1 AND 100),
password_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL CHECK (char_length(title) BETWEEN 1 AND 100),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
last_message_at timestamptz NOT NULL DEFAULT '-infinity'
);
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats(id),
user_id uuid NOT NULL REFERENCES users(id),
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX chat_members_user_chat_idx ON chat_members(user_id, chat_id);
CREATE INDEX chats_created_id_idx ON chats(created_at, id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats(id),
sender_id uuid NOT NULL,
text text NOT NULL CHECK (char_length(text) BETWEEN 1 AND 4000),
created_at timestamptz NOT NULL,
FOREIGN KEY (chat_id, sender_id) REFERENCES chat_members(chat_id, user_id)
);
CREATE INDEX messages_chat_created_id_idx ON messages(chat_id, created_at, id);
+11
View File
@@ -0,0 +1,11 @@
CREATE TABLE message_requests (
user_id uuid NOT NULL REFERENCES users(id),
chat_id uuid NOT NULL REFERENCES chats(id),
request_key varchar(128) NOT NULL,
request_hash char(64) NOT NULL,
message_id uuid NOT NULL REFERENCES messages(id),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
expires_at timestamptz NOT NULL DEFAULT (clock_timestamp() + interval '24 hours'),
PRIMARY KEY (user_id, chat_id, request_key)
);
CREATE INDEX message_requests_expiry_idx ON message_requests(expires_at);
+17
View File
@@ -0,0 +1,17 @@
annotated-doc==0.0.5
annotated-types==0.8.0
anyio==4.15.1
click==8.5.0
colorama==0.4.6; sys_platform == 'win32'
fastapi==0.141.1
h11==0.16.0
idna==3.20
pydantic==2.13.5
pydantic-core==2.46.5
starlette==1.6.0
typing-extensions==4.16.0
typing-inspection==0.4.4
uvicorn==0.53.0
psycopg[binary]==3.2.10
psycopg-pool==3.2.6
redis==6.4.0
+35
View File
@@ -0,0 +1,35 @@
"""Run against a separate API configured with SESSION_TTL_SECONDS=2."""
from pathlib import Path
import json
import os
import sys
import time
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, check
def main():
c = Client(os.environ["EXPIRY_BASE_URL"])
deadline = time.monotonic() + 30
while True:
try:
c.call("GET", "/health/ready", schema="Health")
break
except AssertionError:
check(time.monotonic() < deadline, "TTL API did not become ready")
time.sleep(0.25)
actor = c.user("expiry")
c.call("GET", "/api/v1/users/me", actor=actor, schema="User")
time.sleep(2.3)
response = c.call("GET", "/api/v1/users/me", 401, actor=actor)
report = {"session_ttl_seconds": 2, "wait_seconds": 2.3,
"before_expiry_status": 200, "after_expiry_status": response["status"],
"result": "PASS"}
(ROOT / "evidence" / "expiry.json").write_text(json.dumps(report, indent=2) + "\n")
print("PASS: TTL=2s, initial 200, after 2.3s -> 401; normal API TTL unchanged")
if __name__ == "__main__":
main()
+208
View File
@@ -0,0 +1,208 @@
"""Repeatable API failures and sequential replacement under HTTP load."""
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime, timezone
import json
import math
import os
from pathlib import Path
import sys
import threading
import time
from uuid import uuid4
from ops import ROOT, command, compose, containers, summary
sys.path.insert(0, str(ROOT / "tests"))
from client import Client, Failure
def wait_instances(client, actor, expected, excluded=(), version=None):
deadline = time.monotonic() + 60
while time.monotonic() < deadline:
seen = {}
try:
for _ in range(20):
r = client.call("GET", "/api/v1/users/me", actor=actor, schema="User")
seen[r["headers"]["X-Instance-Id"]] = r["headers"].get("X-App-Version")
if len(seen) >= expected and not any(any(s.startswith(x + "-") for x in excluded) for s in seen):
if version is None or all(v == version for v in seen.values()):
return seen
except Failure:
pass
time.sleep(.2)
raise AssertionError("Expected API replicas were not observed through the load balancer")
def history(client, actor, chat):
from urllib.parse import quote
result, cursor = [], None
while True:
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=100"
if cursor:
path += "&cursor=" + quote(cursor, safe="")
page = client.call("GET", path, actor=actor, schema="MessagePage")["json"]
result.extend(page["items"])
cursor = page["next_cursor"]
if cursor is None:
return result
def resource_sample():
ids = [r["id"] for r in summary() if r["status"] == "running"]
return [json.loads(line) for line in command(["docker", "stats", "--no-stream", "--format", "{{json .}}", *ids]).splitlines()]
def workload(base_url, label, action, duration=24, concurrency=4):
client = Client(base_url, timeout=3)
alice, bob, _, chat = client.bootstrap()
before_message = client.message(alice, chat, "confirmed before " + label)["json"]
observed_before = wait_instances(client, alice, 2)
started = time.monotonic()
deadline = started + duration
events, confirmations = [], {}
action_result = {}
lock = threading.Lock()
def worker(index):
local = Client(base_url, timeout=3)
key = uuid4().hex
attempt = 0
while time.monotonic() < deadline:
begin = time.monotonic()
try:
if index == 0 and attempt % 4 == 0:
response = local.call("GET", "/api/v1/users/me", actor=alice, schema="User")
kind = "read"
else:
response = local.message(alice, chat, "load-" + key, key=key)
kind = "write"
with lock:
confirmations[response["json"]["id"]] = response["json"]
key = uuid4().hex
event = {"at": time.monotonic() - started, "ok": True, "kind": kind,
"ms": (time.monotonic() - begin) * 1000,
"instance": response["headers"]["X-Instance-Id"]}
except Failure:
event = {"at": time.monotonic() - started, "ok": False,
"ms": (time.monotonic() - begin) * 1000}
# Preserve the key after uncertain POST responses.
with lock:
events.append(event)
attempt += 1
def disrupt():
time.sleep(4)
action_result["started_seconds"] = time.monotonic() - started
action_result["detail"] = action(client, alice)
action_result["finished_seconds"] = time.monotonic() - started
action_result["resources_during_load"] = resource_sample()
with ThreadPoolExecutor(max_workers=concurrency + 1) as pool:
workers = [pool.submit(worker, i) for i in range(concurrency)]
change = pool.submit(disrupt)
for future in workers:
future.result()
change.result()
elapsed = time.monotonic() - started
rows = history(client, bob, chat)
by_id = {r["id"]: r for r in rows}
assert by_id[before_message["id"]] == before_message
assert all(by_id.get(mid) == msg for mid, msg in confirmations.items()), "Confirmed write lost"
assert len(rows) == len(by_id) == len({r["text"] for r in rows}), "Retry created duplicate messages"
success = [e for e in events if e["ok"]]
failure = [e for e in events if not e["ok"]]
finish = action_result["finished_seconds"]
writes_after = [e for e in success if e["at"] >= finish and e["kind"] == "write"]
reads_after = [e for e in success if e["at"] >= finish and e["kind"] == "read"]
assert writes_after and reads_after, "Read/write did not resume"
recovery = min(e["at"] for e in writes_after) - action_result["started_seconds"]
if label.startswith("failure"):
assert recovery <= 10, "Failover exceeded 10 seconds"
hostname = action_result["detail"]["hostname"]
assert all(not e["instance"].startswith(hostname + "-") for e in writes_after[-20:])
latencies = sorted(e["ms"] for e in success)
def percentile(q):
return round(latencies[max(0, math.ceil(len(latencies) * q) - 1)], 3)
buckets = []
for start in range(0, math.ceil(elapsed), 2):
group = [e for e in events if start <= e["at"] < start + 2]
buckets.append({"start_seconds": start, "rps": sum(e["ok"] for e in group) / 2,
"errors": sum(not e["ok"] for e in group)})
report = {"scenario": label, "utc": datetime.now(timezone.utc).isoformat(),
"duration_seconds": round(elapsed, 3), "concurrency": concurrency,
"successful_requests": len(success), "failed_attempts": len(failure),
"error_ratio": round(len(failure) / len(events), 6),
"successful_rps": round(len(success) / elapsed, 3),
"successful_latency_ms": {"p50": percentile(.5), "p95": percentile(.95), "p99": percentile(.99)},
"first_write_after_action_seconds": round(recovery, 3), "action": action_result,
"confirmed_writes_checked": len(confirmations) + 1, "history_messages": len(rows),
"unique_history_messages": len(by_id), "instances_before": list(observed_before),
"series_2_seconds": buckets, "deployment_after": summary(),
"note": "Closed-loop mixed HTTP load, 4 clients; failures are attempts, retries retain keys; latency covers successes only."}
(ROOT / "evidence" / (label + ".json")).write_text(json.dumps(report, indent=2) + "\n")
print(label + ": " + str(report["successful_rps"]) + " RPS; " + str(len(failure)) + " errors", flush=True)
return report
def failure_action(target):
def stop(client, actor):
begin = time.monotonic()
command(["docker", "stop", "--time", "6", target["Id"]], timeout=10)
row = json.loads(command(["docker", "inspect", target["Id"]]))[0]
assert not row["State"]["Running"] and row["State"]["ExitCode"] == 0
elapsed = time.monotonic() - begin
assert elapsed <= 10
return {"container": row["Id"], "hostname": row["Config"]["Hostname"],
"stop_seconds": round(elapsed, 3), "exit_code": row["State"]["ExitCode"]}
return stop
def rolling_action(client, actor):
original = containers()
assert len(original) == 2
updates = []
env = dict(os.environ, APP_VERSION="lab3-v2")
# Both migrations are additive. Check/replay once before replacing processes.
migration = compose("run", "--rm", "--no-deps", "migrate")
for old in original:
address = next(n["IPAddress"] for name, n in old["NetworkSettings"]["Networks"].items() if name.endswith("_edge"))
slot = compose("run", "--rm", "--no-deps", "checks", "python", "scripts/ops.py", "drain", address)
command(["docker", "stop", "--time", "6", old["Id"]])
command(["docker", "rm", old["Id"]])
compose("up", "-d", "--no-deps", "--no-recreate", "--scale", "api=2", "api", env=env)
compose("run", "--rm", "--no-deps", "checks", "python", "scripts/ops.py", "ready", slot)
current = containers()
fresh = [r for r in current if r["Id"] not in {x["Id"] for x in original} and r["Id"] not in {x["new_container"] for x in updates}]
assert len(fresh) == 1
wait_instances(client, actor, 2, excluded=[old["Config"]["Hostname"]])
updates.append({"old_container": old["Id"], "new_container": fresh[0]["Id"], "backend_slot": slot,
"new_image": fresh[0]["Image"]})
wait_instances(client, actor, 2, version="lab3-v2")
return {"updates": updates, "migration": migration,
"method": "drain, stop, remove and recreate each container; same application image, new APP_VERSION configuration"}
def main():
import argparse
p = argparse.ArgumentParser()
p.add_argument("--base-url", default="http://127.0.0.1:8080")
args = p.parse_args()
originals = containers()
assert len(originals) == 2
reports = []
for index, target in enumerate(originals, 1):
try:
reports.append(workload(args.base_url, "failure-api-" + str(index), failure_action(target)))
finally:
command(["docker", "start", target["Id"]])
c = Client(args.base_url)
alice = c.user("restore")
wait_instances(c, alice, 2)
reports.append(workload(args.base_url, "rolling-update", rolling_action, duration=45))
(ROOT / "evidence" / "experiments.json").write_text(json.dumps([
{k: r[k] for k in ("scenario", "successful_rps", "failed_attempts", "first_write_after_action_seconds",
"confirmed_writes_checked", "successful_latency_ms")} for r in reports], indent=2) + "\n")
if __name__ == "__main__":
main()
+76
View File
@@ -0,0 +1,76 @@
"""Seed an isolated synthetic chat, measure the actual keyset SQL, keep raw plans."""
from datetime import datetime, timedelta, timezone
import json
from pathlib import Path
import sys
from urllib.parse import quote
from uuid import UUID
import psycopg
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
sys.path.insert(0, str(ROOT / "tests"))
from app.config import secret
from app.security import Cursors, stamp
from client import Client, check
def walk(node):
yield node
for child in node.get("Plans", []):
yield from walk(child)
def main():
c = Client()
actor = c.user("explain")
chat = c.call("POST", "/api/v1/chats", 201, actor=actor,
body={"title": "EXPLAIN 10000", "member_ids": [actor["id"]]}, schema="Chat")["json"]
base = datetime.now(timezone.utc)
with psycopg.connect(secret("DATABASE_URL"), connect_timeout=2) as conn:
conn.execute("""INSERT INTO messages(id, chat_id, sender_id, text, created_at)
SELECT gen_random_uuid(), %s, %s, 'synthetic-' || i,
%s + i * interval '1 microsecond'
FROM generate_series(1, 10000) AS i""", (UUID(chat["id"]), UUID(actor["id"]), base))
conn.execute("UPDATE chats SET last_message_at=%s WHERE id=%s",
(base + timedelta(microseconds=10000), UUID(chat["id"])))
conn.execute("ANALYZE messages")
report = {"messages_in_chat": 10000, "page_limit": 50,
"index": "messages_chat_created_id_idx", "cases": {}}
text_plans = []
cursors = Cursors(secret("CURSOR_KEY").encode())
with psycopg.connect(secret("DATABASE_URL"), connect_timeout=2) as conn:
for label, offset in (("first", 0), ("middle", 5000), ("late", 9900)):
sql = "SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE chat_id=%s"
params = [UUID(chat["id"])]
path = "/api/v1/chats/" + chat["id"] + "/messages?limit=50"
if offset:
boundary = conn.execute("SELECT created_at, id FROM messages WHERE chat_id=%s AND text=%s",
(UUID(chat["id"]), "synthetic-" + str(offset))).fetchone()
sql += " AND (created_at, id) > (%s, %s)"
params.extend(boundary)
token = cursors.encode("messages:" + chat["id"],
{"created_at": stamp(boundary[0]), "id": str(boundary[1])})
path += "&cursor=" + quote(token, safe="")
sql += " ORDER BY created_at, id LIMIT %s"
params.append(51)
data = conn.execute("EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) " + sql, params).fetchone()[0][0]
nodes = list(walk(data["Plan"]))
check(not any(x["Node Type"] == "Seq Scan" for x in nodes), "Unexpected full scan in keyset query")
check(any(x.get("Index Name") == report["index"] for x in nodes), "History index not used")
check(data["Plan"]["Actual Rows"] == 51, "LIMIT was not respected")
page = c.call("GET", path, actor=actor, schema="MessagePage")["json"]
check(len(page["items"]) == 50 and page["next_cursor"], "HTTP page differs from measured SQL")
expected = ["synthetic-" + str(i) for i in range(offset + 1, offset + 51)]
check([x["text"] for x in page["items"]] == expected, "HTTP keyset boundary differs")
report["cases"][label] = {"after_message": offset, "sql": sql, "plan": data}
text_plans.append(label.upper() + "\n" + sql + "\n" + "\n".join(
row[0] for row in conn.execute("EXPLAIN (ANALYZE, BUFFERS) " + sql, params).fetchall()))
(ROOT / "evidence" / "explain.json").write_text(json.dumps(report, indent=2) + "\n")
(ROOT / "evidence" / "explain.txt").write_text("\n\n".join(text_plans) + "\n")
print("PASS: 10000 messages, first/middle/late pages use history index; 51 SQL rows for 50 HTTP items")
if __name__ == "__main__":
main()
+35
View File
@@ -0,0 +1,35 @@
"""Create local synthetic credentials once without printing secret values."""
from pathlib import Path
import secrets
ROOT = Path(__file__).resolve().parents[1]
def main():
directory = ROOT / "secrets"
directory.mkdir(exist_ok=True)
for name in ("postgres_password", "redis_password", "cursor_key"):
path = directory / name
if not path.exists():
path.write_text(secrets.token_hex(32) + "\n", encoding="utf-8", newline="\n")
else:
# Shell command substitution on Linux must not retain a Windows CR.
value = path.read_text(encoding="utf-8").strip()
path.write_text(value + "\n", encoding="utf-8", newline="\n")
pg_password = (directory / "postgres_password").read_text().strip()
redis_password = (directory / "redis_password").read_text().strip()
for name, value in {
"database_url": f"postgresql://messenger:{pg_password}@postgres:5432/messenger",
"cache_url": f"redis://:{redis_password}@redis:6379/0",
}.items():
path = directory / name
if not path.exists():
path.write_text(value + "\n", encoding="utf-8", newline="\n")
env = ROOT / ".env"
if not env.exists():
env.write_bytes((ROOT / ".env.example").read_bytes())
print("PASS: local .env and secret files exist; existing values preserved")
if __name__ == "__main__":
main()
+94
View File
@@ -0,0 +1,94 @@
"""Docker operations scoped to the configured local Compose project."""
import csv
import io
import json
import os
from pathlib import Path
import socket
import subprocess
import time
ROOT = Path(__file__).resolve().parents[1]
def command(args, timeout=120, env=None):
result = subprocess.run(args, cwd=ROOT, capture_output=True, text=True,
encoding="utf-8", errors="replace", timeout=timeout, env=env)
if result.returncode:
# Commands and stderr can contain secrets. Keep detailed errors local.
raise RuntimeError("Local command failed: " + " ".join(args[:3]) + "\n" + result.stderr[-1500:])
return result.stdout.strip()
def compose(*args, timeout=120, env=None):
return command(["docker", "compose", "--env-file", ".env", *args], timeout, env)
def project():
return os.getenv("COMPOSE_PROJECT_NAME") or json.loads(compose("config", "--format", "json"))["name"]
def containers(service="api", running=True):
args = ["docker", "ps"] + ([] if running else ["-a"])
ids = command(args + ["--filter", "label=com.docker.compose.project=" + project(),
"--filter", "label=com.docker.compose.service=" + service,
"--filter", "label=com.docker.compose.oneoff=False", "--format", "{{.ID}}"])
return json.loads(command(["docker", "inspect", *ids.split()])) if ids else []
def summary():
result = []
for service in ("api", "postgres", "redis", "lb"):
for row in containers(service, running=False):
result.append({"service": service, "id": row["Id"], "name": row["Name"].lstrip("/"),
"started_at": row["State"]["StartedAt"], "status": row["State"]["Status"],
"image": row["Image"], "user": row["Config"]["User"],
"memory_bytes": row["HostConfig"]["Memory"], "nano_cpus": row["HostConfig"]["NanoCpus"],
"published_ports": row["HostConfig"]["PortBindings"],
"volumes": [m["Name"] for m in row["Mounts"] if m["Type"] == "volume"]})
return result
def runtime(line):
with socket.create_connection(("lb", 9999), timeout=3) as sock:
sock.sendall((line + "\n").encode())
sock.shutdown(socket.SHUT_WR)
chunks = []
while True:
data = sock.recv(65536)
if not data:
return b"".join(chunks).decode()
chunks.append(data)
def drain(address):
lines = runtime("show servers state messenger").splitlines()
fields = next(line[2:].split() for line in lines if line.startswith("# "))
for line in lines:
values = line.split()
if not values or line.startswith("#") or len(values) != len(fields):
continue
row = dict(zip(fields, values))
if row["srv_addr"] == address:
slot = "messenger/" + row["srv_name"]
runtime("set server " + slot + " state drain")
# Let requests already assigned to this replica finish (HTTP budget 5 s).
deadline = time.monotonic() + 7
while time.monotonic() < deadline:
stats = list(csv.DictReader(io.StringIO(runtime("show stat").lstrip("# "))))
active = [s for s in stats if s["pxname"] == "messenger" and s["svname"] == row["srv_name"]]
if active and int(active[0]["scur"]) == 0:
return slot
time.sleep(.1)
raise RuntimeError("Replica did not drain within 7 seconds")
raise RuntimeError("Backend address was not found in HAProxy")
if __name__ == "__main__":
import sys
if sys.argv[1] == "drain":
print(drain(sys.argv[2]))
elif sys.argv[1] == "ready":
runtime("set server " + sys.argv[2] + " state ready")
elif sys.argv[1] == "summary":
print(json.dumps(summary(), indent=2))
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
docker compose --env-file .env restart --timeout 6 api postgres redis lb
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
exec python scripts/stop_one.py
+24
View File
@@ -0,0 +1,24 @@
"""Stop exactly the replica selected by the public failover scenario."""
import json
import os
import time
from ops import ROOT, command, containers
def main():
target = os.environ["TARGET_INSTANCE_ID"]
matches = [row for row in containers() if target.startswith(row["Config"]["Hostname"] + "-")]
if len(matches) != 1:
raise RuntimeError("Target does not identify exactly one running API replica")
row = matches[0]
started = time.monotonic()
command(["docker", "stop", "--time", "6", row["Id"]], timeout=9)
state = json.loads(command(["docker", "inspect", row["Id"]]))[0]["State"]
assert not state["Running"] and state["ExitCode"] == 0, "Graceful shutdown failed"
report = {"target_instance": target, "container": row["Id"], "exit_code": state["ExitCode"],
"stop_seconds": round(time.monotonic() - started, 3), "remains_stopped": True}
(ROOT / "evidence" / "last-stop.json").write_text(json.dumps(report, indent=2) + "\n")
if __name__ == "__main__":
main()
+97
View File
@@ -0,0 +1,97 @@
"""Build and verify the complete local lab3 stack without deleting volumes."""
import argparse
from datetime import datetime, timezone
import json
import os
from pathlib import Path
import subprocess
import sys
from ops import ROOT, command, compose, containers, summary
def saved(name, args, timeout=300):
result = subprocess.run(args, cwd=ROOT, capture_output=True, text=True,
encoding="utf-8", errors="replace", timeout=timeout)
(ROOT / "evidence" / name).write_text(result.stdout + result.stderr, encoding="utf-8")
if result.returncode:
raise RuntimeError("Check failed; see evidence/" + name)
return result.stdout
def checks(name, *args):
return saved(name, ["docker", "compose", "--env-file", ".env", "run", "--rm", "--no-deps", "checks", *args])
def main():
p = argparse.ArgumentParser()
p.add_argument("--skip-build", action="store_true")
p.add_argument("--base-url", default=None)
p.add_argument("--skip-experiments", action="store_true")
args = p.parse_args()
(ROOT / "evidence").mkdir(exist_ok=True)
command([sys.executable, "-X", "utf8", "scripts/init_local.py"])
compose("config", "--quiet")
if not args.skip_build:
print("Building API and check images", flush=True)
saved("build.txt", ["docker", "compose", "--env-file", ".env", "build", "api", "checks"], 900)
saved("startup.txt", ["docker", "compose", "--env-file", ".env", "up", "-d", "--wait", "--wait-timeout", "120"], 240)
config = json.loads(compose("config", "--format", "json"))
base_url = args.base_url or "http://127.0.0.1:" + str(config["services"]["lb"]["ports"][0]["published"])
print("Checking contract, HTTP API and student tests", flush=True)
checks("contract-check.txt", "make", "check")
checks("smoke.txt", "make", "test")
checks("student-tests.txt", "make", "test-student")
before = summary()
checks("persistence.txt", "sh", "-ec", "chmod +x scripts/restart.sh; make test-persistence ACTION_SCRIPT=scripts/restart.sh")
after = summary()
by_id = {r["id"]: r for r in after}
for old in before:
new = by_id[old["id"]]
assert old["started_at"] != new["started_at"]
assert old["volumes"] == new["volumes"]
(ROOT / "evidence" / "restart.json").write_text(json.dumps({"before": before, "after": after}, indent=2) + "\n")
print("Running public failover scenario", flush=True)
try:
checks("failover.txt", "sh", "-ec", "chmod +x scripts/stop-one.sh; make test-failover ACTION_SCRIPT=scripts/stop-one.sh")
finally:
compose("start", "api")
# Wait for both replicas through LB before starting subsequent checks.
sys.path.insert(0, str(ROOT / "tests"))
from client import Client
from experiments import wait_instances
client = Client(base_url)
actor = client.user("verify")
wait_instances(client, actor, 2)
print("Checking short session expiry and indexed history", flush=True)
ttl_name = config["name"] + "-ttl-check"
assert not command(["docker", "ps", "-a", "--filter", "name=^/" + ttl_name + "$", "--format", "{{.ID}}"])
try:
compose("run", "-d", "--no-deps", "--name", ttl_name, "-e", "SESSION_TTL_SECONDS=2", "api")
checks("expiry.txt", "env", "EXPIRY_BASE_URL=http://" + ttl_name + ":8080", "python", "scripts/check_expiry.py")
finally:
command(["docker", "stop", "--time", "6", ttl_name])
command(["docker", "rm", ttl_name])
checks("explain-run.txt", "make", "explain")
checks("smoke-final.txt", "make", "test")
saved("published-smoke.txt", [sys.executable, "-X", "utf8", "tests/smoke.py", "--base-url", base_url])
if not args.skip_experiments:
print("Measuring failure of each replica and sequential replacement under load", flush=True)
saved("experiments.txt", [sys.executable, "-X", "utf8", "scripts/experiments.py", "--base-url", base_url], 360)
deployed = summary()
assert len([r for r in deployed if r["service"] == "api" and r["status"] == "running"]) == 2
assert all(not r["published_ports"] for r in deployed if r["service"] != "lb")
assert all(r["memory_bytes"] and r["nano_cpus"] for r in deployed)
versions = command(["docker", "info", "--format", "{{json .}}"])
info = json.loads(versions)
report = {"result": "PASS", "utc": datetime.now(timezone.utc).isoformat(), "base_url": base_url,
"checks": ["public_smoke", "student_tests", "full_restart", "public_failover", "expiry", "index_10000", "published_smoke"]
+ ([] if args.skip_experiments else ["each_replica_failure", "sequential_replacement_under_load"]),
"docker": {k: info[k] for k in ("ServerVersion", "OperatingSystem", "Architecture", "NCPU", "MemTotal")},
"deployment": deployed}
(ROOT / "evidence" / "verification.json").write_text(json.dumps(report, indent=2) + "\n")
print("PASS: lab3 checks completed; evidence/verification.json", flush=True)
if __name__ == "__main__":
main()
+11
View File
@@ -0,0 +1,11 @@
$ErrorActionPreference = 'Stop'
Set-Location -LiteralPath $PSScriptRoot
docker info --format '{{.ServerVersion}}'
if ($LASTEXITCODE -ne 0) { throw 'Start Docker Desktop first.' }
docker run --rm --mount "type=bind,source=$PSScriptRoot,target=/work" --workdir /work python:3.13.15-slim-bookworm python scripts/init_local.py
if ($LASTEXITCODE -ne 0) { throw 'Local configuration initialization failed.' }
docker compose --env-file .env up -d --build --wait --wait-timeout 120
if ($LASTEXITCODE -ne 0) { throw 'Stack startup failed; inspect docker compose logs.' }
$taskConfig = docker compose --env-file .env config --format json | ConvertFrom-Json
$taskPort = $taskConfig.services.lb.ports[0].published
Write-Host "Ready: http://127.0.0.1:$taskPort/health/ready"
+222
View File
@@ -0,0 +1,222 @@
"""Integration tests against real HTTP, PostgreSQL and Redis on the local stack."""
from concurrent.futures import ThreadPoolExecutor
import base64
import hashlib
import json
from pathlib import Path
import subprocess
import sys
import time
import unittest
from urllib.parse import quote
from uuid import uuid4
import psycopg
import redis
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
sys.path.insert(0, str(ROOT / "scripts"))
from client import Client
from ops import containers
from app.config import secret
class APITests(unittest.TestCase):
def setUp(self):
self.c = Client(timeout=8)
self.alice = self.c.user("student")
self.bob = self.c.user("member")
self.chat = self.c.call("POST", "/api/v1/chats", 201, actor=self.alice,
body={"title": "Учебный чат", "member_ids": [self.alice["id"], self.bob["id"]]},
schema="Chat")["json"]
self.path = "/api/v1/chats/" + self.chat["id"]
def scalar(self, query, params=()):
with psycopg.connect(secret("DATABASE_URL"), connect_timeout=2) as conn:
return conn.execute(query, params).fetchone()[0]
def history(self, limit=17):
result, cursor = [], None
for _ in range(100):
path = self.path + "/messages?limit=" + str(limit)
if cursor:
path += "&cursor=" + quote(cursor, safe="")
page = self.c.call("GET", path, actor=self.alice, schema="MessagePage")["json"]
result.extend(page["items"])
cursor = page["next_cursor"]
if cursor is None:
return result
self.fail("Pagination did not terminate")
def test_registration_race(self):
registration = {"username": "race_" + uuid4().hex[:16], "display_name": "Гонка",
"password": "Synthetic-password-12345"}
def register(_):
return self.c.call("POST", "/api/v1/users", [201, 409], body=registration,
schema="User")["status"]
with ThreadPoolExecutor(max_workers=10) as pool:
statuses = list(pool.map(register, range(10)))
self.assertEqual(statuses.count(201), 1)
self.assertEqual(statuses.count(409), 9)
self.assertEqual(self.scalar("SELECT count(*) FROM users WHERE username=%s",
(registration["username"],)), 1)
print("RACE: 10 requests, 1 created, 9 conflicts, 1 database row", flush=True)
def test_composite_chat_rollback(self):
before = self.scalar("SELECT count(*) FROM chats")
memberships = self.scalar("SELECT count(*) FROM chat_members")
self.c.call("POST", "/api/v1/chats", 400, actor=self.alice,
body={"title": "Rollback", "member_ids": [self.alice["id"], str(uuid4())]})
self.assertEqual(self.scalar("SELECT count(*) FROM chats"), before)
self.assertEqual(self.scalar("SELECT count(*) FROM chat_members"), memberships)
self.assertEqual(self.scalar("SELECT count(*) FROM chats c WHERE NOT EXISTS "
"(SELECT 1 FROM chat_members m WHERE m.chat_id=c.id)"), 0)
print("ROLLBACK: invalid second member left no chat or membership rows", flush=True)
def test_concurrent_messages(self):
def send(index):
return self.c.message(self.alice, self.chat, "parallel-" + str(index))["json"]["id"]
with ThreadPoolExecutor(max_workers=10) as pool:
ids = list(pool.map(send, range(100)))
messages = self.history()
self.assertEqual(len(messages), 100)
self.assertEqual(len(set(ids)), 100)
self.assertEqual({row["id"] for row in messages}, set(ids))
self.assertEqual({row["text"] for row in messages}, {"parallel-" + str(i) for i in range(100)})
self.assertEqual(messages, sorted(messages, key=lambda row: (row["created_at"], row["id"])))
print("CONCURRENCY: 10 clients, 100 confirmations, 100 unique messages", flush=True)
def test_validation_and_access(self):
for payload in [b"{", b"null", b"[]", b"\xff", b'{"text":NaN}']:
self.c.call("POST", self.path + "/messages", 400, actor=self.alice, raw=payload,
headers={"Content-Type": "application/json", "Idempotency-Key": uuid4().hex})
for body in [{"text": ""}, {"text": " \n\t"}, {"text": 123}, {"text": "x" * 4001},
{"text": "hello", "sender_id": self.bob["id"]}]:
self.c.call("POST", self.path + "/messages", 400, actor=self.alice, body=body,
headers={"Idempotency-Key": uuid4().hex})
outsider = self.c.user("outsider")
for suffix in ("", "/messages"):
self.c.call("GET", self.path + suffix, 404, actor=outsider)
self.c.call("POST", self.path + "/messages", 404, actor=outsider, body={"text": "hidden"})
for members in [[self.bob["id"]], [self.alice["id"], self.alice["id"]], ["bad"], []]:
self.c.call("POST", "/api/v1/chats", 400, actor=self.alice,
body={"title": "Invalid", "member_ids": members})
self.c.call("GET", "/api/v1/chats/not-uuid", 400, actor=self.alice)
for limit in ("0", "101", "1.1", "-1", "", "true"):
self.c.call("GET", self.path + "/messages?limit=" + limit, 400, actor=self.alice)
self.c.call("POST", "/api/v1/users", 409, body=self.alice["registration"])
for raw in (b"{}", b'{"username":123}', b'{"username":"ab","display_name":"x","password":"123456789012"}'):
self.c.call("POST", "/api/v1/users", 400, raw=raw, headers={"Content-Type": "application/json"})
def test_passwords_and_authentication(self):
registration = {"username": "unicode_" + uuid4().hex[:12], "display_name": "Имя 🧪",
"password": "я:🧪" + "密" * 125}
user = self.c.call("POST", "/api/v1/users", 201, body=registration, schema="User")["json"]
credential = base64.b64encode((registration["username"] + ":" + registration["password"]).encode()).decode()
session = self.c.call("POST", "/api/v1/auth/sessions", 201,
headers={"Authorization": "Basic " + credential}, schema="Session")["json"]
self.assertEqual(session["user"]["id"], user["id"])
encoded = self.scalar("SELECT password_hash FROM users WHERE id=%s", (user["id"],))
algorithm, n, r, p, salt, digest = encoded.split("$")
self.assertEqual(algorithm, "scrypt")
actual = hashlib.scrypt(registration["password"].encode(), salt=bytes.fromhex(salt),
n=int(n), r=int(r), p=int(p), dklen=32, maxmem=64 * 1024 * 1024).hex()
self.assertTrue(actual == digest)
self.assertNotIn("password", user)
for username in (registration["username"], "missing_" + uuid4().hex[:12]):
bad = base64.b64encode((username + ":wrong-password-123").encode()).decode()
response = self.c.call("POST", "/api/v1/auth/sessions", 401,
headers={"Authorization": "Basic " + bad})
self.assertEqual(response["headers"]["WWW-Authenticate"], 'Basic realm="messenger", charset="UTF-8"')
self.assertEqual(response["json"]["error"]["message"], "Valid credentials required")
self.c.call("GET", "/api/v1/users/me", 401, headers={"Authorization": "Basic " + credential})
self.c.call("GET", "/api/v1/users/me", 401, headers={"X-User-Id": user["id"]})
self.c.call("GET", "/api/v1/users/me", 401, headers={"Authorization": "Bearer " + "x" * 43})
actor = {"id": user["id"], "token": session["token"]}
self.c.call("DELETE", "/api/v1/auth/sessions/current", 204, actor=actor)
self.c.call("GET", "/api/v1/users/me", 401, actor=actor)
self.c.call("DELETE", "/api/v1/auth/sessions/current", 401, actor=actor)
def test_cursor_binding_and_chat_pages(self):
sent = [self.c.message(self.alice, self.chat, "page-" + str(i))["json"] for i in range(7)]
self.assertEqual(self.history(limit=2), sent)
page = self.c.call("GET", self.path + "/messages?limit=2", actor=self.alice, schema="MessagePage")["json"]
cursor = page["next_cursor"]
other = self.c.call("POST", "/api/v1/chats", 201, actor=self.alice,
body={"title": "Other", "member_ids": [self.alice["id"]]}, schema="Chat")["json"]
self.c.call("GET", "/api/v1/chats/" + other["id"] + "/messages?cursor=" + quote(cursor), 400, actor=self.alice)
for token in ("bad", "", cursor[:-3] + "xxx"):
self.c.call("GET", self.path + "/messages?cursor=" + quote(token), 400, actor=self.alice)
later = self.c.message(self.alice, self.chat, "later")["json"]
continued = self.c.call("GET", self.path + "/messages?limit=100&cursor=" + quote(cursor),
actor=self.alice, schema="MessagePage")["json"]
self.assertEqual(continued["items"], sent[2:] + [later])
first = self.c.call("GET", "/api/v1/chats?limit=1", actor=self.alice, schema="ChatPage")["json"]
second = self.c.call("GET", "/api/v1/chats?limit=100&cursor=" + quote(first["next_cursor"]),
actor=self.alice, schema="ChatPage")["json"]
self.assertEqual([row["id"] for row in first["items"] + second["items"]],
[self.chat["id"], other["id"]])
def compose(self, *args):
result = subprocess.run(["docker", "compose", "--env-file", ".env", *args], cwd=ROOT,
capture_output=True, text=True, timeout=45)
if result.returncode:
self.fail("Compose action failed: " + " ".join(args[:2]))
def wait_ready(self):
deadline = time.monotonic() + 40
while time.monotonic() < deadline:
try:
self.c.call("GET", "/health/ready", schema="Health")
return
except AssertionError:
time.sleep(0.25)
self.fail("Stack did not recover")
def test_dependency_failures(self):
# With all backends unready, LB may generate its own 503. Diagnose
# the application error contract on an internal replica explicitly.
row = containers()[0]
self.c = Client("http://" + row["Name"].lstrip("/") + ":8080", timeout=8)
report = {}
for service in ("redis", "postgres"):
self.compose("stop", "--timeout", "2", service)
try:
self.c.call("GET", "/health/live", schema="Health")
ready = self.c.call("GET", "/health/ready", 503)
started = time.monotonic()
response = self.c.call("GET", "/api/v1/users/me", 503, actor=self.alice)
elapsed = time.monotonic() - started
self.assertEqual(response["json"]["error"]["code"], "unavailable")
self.assertLess(elapsed, 6)
self.c.call("GET", "/api/v1/users/me", 401, headers={"X-User-Id": self.alice["id"]})
report[service] = {"ready_status": ready["status"], "protected_status": response["status"],
"elapsed_seconds": round(elapsed, 3), "anonymous_status": 401}
finally:
self.compose("start", service)
self.wait_ready()
self.c.call("GET", "/api/v1/users/me", actor=self.alice, schema="User")
self.c.message(self.alice, self.chat, "after-" + service)
(ROOT / "evidence" / "failures.json").write_text(json.dumps(report, indent=2) + "\n")
print("FAILURES: Redis/PostgreSQL -> bounded 503, live 200, anonymous 401, recovery with old session", flush=True)
def test_redis_crash_durability(self):
client = redis.Redis.from_url(secret("CACHE_URL"), decode_responses=True)
try:
self.assertEqual(client.config_get("appendonly")["appendonly"], "yes")
self.assertEqual(client.config_get("appendfsync")["appendfsync"], "always")
key = "session:" + hashlib.sha256(self.alice["token"].encode()).hexdigest()
deadline_before = client.pexpiretime(key)
self.compose("kill", "--signal", "SIGKILL", "redis")
self.compose("start", "redis")
self.wait_ready()
self.c.call("GET", "/api/v1/users/me", actor=self.alice, schema="User")
self.assertEqual(client.pexpiretime(key), deadline_before)
finally:
client.close()
print("REDIS CRASH: old session survived SIGKILL/AOF replay, expiry deadline unchanged", flush=True)
if __name__ == "__main__":
unittest.main()
+110
View File
@@ -0,0 +1,110 @@
"""Cross-process idempotency, access and transaction boundaries."""
from concurrent.futures import ThreadPoolExecutor
import json
from pathlib import Path
import sys
import threading
import unittest
from uuid import uuid4
import psycopg
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "tests"))
sys.path.insert(0, str(ROOT / "scripts"))
from client import Client
from ops import containers
from app.config import secret
class ReplicationTests(unittest.TestCase):
def setUp(self):
self.c = Client()
self.alice, self.bob, self.eve, self.chat = self.c.bootstrap()
self.path = "/api/v1/chats/" + self.chat["id"] + "/messages"
def sql(self, query, values=()):
with psycopg.connect(secret("DATABASE_URL"), connect_timeout=2) as conn:
cur = conn.execute(query, values)
return cur.fetchall() if cur.description else None
def test_parallel_replay_across_real_replicas(self):
replicas = containers()
self.assertGreaterEqual(len(replicas), 2)
clients = [Client("http://" + row["Name"].lstrip("/") + ":8080") for row in replicas]
barrier = threading.Barrier(10)
key = uuid4().hex
def send(index):
if index < 10:
barrier.wait()
return clients[index % len(clients)].message(self.alice, self.chat, "Один результат 🧪", key=key)
with ThreadPoolExecutor(max_workers=10) as pool:
responses = list(pool.map(send, range(100)))
self.assertEqual(len({r["json"]["id"] for r in responses}), 1)
self.assertGreaterEqual(len({r["headers"]["X-Instance-Id"] for r in responses}), 2)
self.assertTrue(all(r["json"] == responses[0]["json"] for r in responses))
self.assertEqual(self.sql("SELECT count(*) FROM messages WHERE chat_id=%s", (self.chat["id"],))[0][0], 1)
self.assertEqual(self.sql("SELECT count(*) FROM message_requests WHERE chat_id=%s", (self.chat["id"],))[0][0], 1)
for _ in range(12):
self.assertEqual(self.c.message(self.alice, self.chat, "Один результат 🧪", key=key)["json"], responses[0]["json"])
report = {"requests": 100, "clients": 10, "real_containers": len(replicas),
"observed_instances": len({r["headers"]["X-Instance-Id"] for r in responses}),
"messages": 1, "replay_records": 1, "lb_replays": 12}
(ROOT / "evidence" / "idempotency.json").write_text(json.dumps(report, indent=2) + "\n")
def test_key_validation_scope_conflict_and_logout(self):
for key in (None, "", "bad key", "bad/key", "a" * 129):
headers = {} if key is None else {"Idempotency-Key": key}
self.c.call("POST", self.path, 400, actor=self.alice, body={"text": "test"}, headers=headers)
key = uuid4().hex
first = self.c.message(self.alice, self.chat, "first", key=key)["json"]
replay = self.c.call("POST", self.path, 201, actor=self.alice,
raw=b'{\n "text" : "first"\n}', headers={"Content-Type": "application/json", "Idempotency-Key": key},
schema="Message")["json"]
self.assertEqual(first, replay)
self.c.call("POST", self.path, 409, actor=self.alice, body={"text": "changed"}, headers={"Idempotency-Key": key})
second = self.c.message(self.bob, self.chat, "first", key=key)["json"]
self.assertNotEqual(first["id"], second["id"])
other = self.c.call("POST", "/api/v1/chats", 201, actor=self.alice,
body={"title": "scope", "member_ids": [self.alice["id"]]}, schema="Chat")["json"]
self.assertNotEqual(self.c.message(self.alice, other, "first", key=key)["json"]["id"], first["id"])
self.c.call("POST", self.path, 404, actor=self.eve, body={"text": "first"}, headers={"Idempotency-Key": key})
self.c.call("DELETE", "/api/v1/auth/sessions/current", 204, actor=self.alice)
self.c.call("POST", self.path, 401, actor=self.alice, body={"text": "first"}, headers={"Idempotency-Key": key})
def test_failed_transaction_does_not_save_message_or_replay(self):
key = uuid4().hex
# Fail after INSERT messages, at INSERT replay, to check the whole
# transaction rather than merely validation before the first write.
self.sql("""CREATE FUNCTION lab3_test_reject() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN RAISE EXCEPTION 'synthetic transaction failure'; END $$""")
self.sql("CREATE TRIGGER lab3_test_reject BEFORE INSERT ON message_requests "
"FOR EACH ROW EXECUTE FUNCTION lab3_test_reject()")
try:
self.c.call("POST", self.path, 503, actor=self.alice, body={"text": "atomic"}, headers={"Idempotency-Key": key})
self.assertEqual(self.sql("SELECT count(*) FROM messages WHERE chat_id=%s", (self.chat["id"],))[0][0], 0)
self.assertEqual(self.sql("SELECT count(*) FROM message_requests WHERE chat_id=%s", (self.chat["id"],))[0][0], 0)
finally:
self.sql("DROP TRIGGER lab3_test_reject ON message_requests")
self.sql("DROP FUNCTION lab3_test_reject()")
first = self.c.message(self.alice, self.chat, "atomic", key=key)["json"]
self.assertEqual(self.c.message(self.alice, self.chat, "atomic", key=key)["json"], first)
def test_expired_key_and_same_session_on_all_instances(self):
observed = set()
for _ in range(30):
response = self.c.call("GET", "/api/v1/users/me", actor=self.alice, schema="User")
self.assertEqual(response["json"]["id"], self.alice["id"])
observed.add(response["headers"]["X-Instance-Id"])
self.assertGreaterEqual(len(observed), 2)
key = uuid4().hex
original = self.c.message(self.alice, self.chat, "before expiry", key=key)["json"]
self.sql("UPDATE message_requests SET expires_at=clock_timestamp()-interval '1 second' "
"WHERE user_id=%s AND chat_id=%s AND request_key=%s", (self.alice["id"], self.chat["id"], key))
fresh = self.c.message(self.alice, self.chat, "after expiry", key=key)["json"]
self.assertNotEqual(original["id"], fresh["id"])
self.assertEqual(self.c.message(self.alice, self.chat, "after expiry", key=key)["json"], fresh)
if __name__ == "__main__":
unittest.main()