Compare commits

4 Commits
Author SHA1 Message Date
Getlles 444227de60 report 2026-11-09 15:19:41 +03:00
Getlles 21218e0392 grade4 2026-11-09 14:52:10 +03:00
Getlles 234e2a2fd7 grade3 2026-11-09 12:40:25 +03:00
Getlles b750ef2722 lab2 2026-11-09 11:11:33 +03:00
15 changed files with 1116 additions and 24 deletions
+22
View File
@@ -0,0 +1,22 @@
FROM python:3.11.7-slim-bookworm AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
FROM python:3.11.7-slim-bookworm AS runner
RUN useradd -u 10001 -m appuser
WORKDIR /home/appuser
COPY --from=builder /install /usr/local
COPY app.py .
USER appuser
EXPOSE 8080
HEALTHCHECK --interval=5s --timeout=3s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8080/health/ready')"
ENV PYTHONUNBUFFERED=1
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080", "--timeout-graceful-shutdown", "10"]
+57 -22
View File
@@ -1,37 +1,72 @@
# Отчёт — лабораторная 3 # Отчёт — лабораторная 3
- Автор, группа: TODO - Автор, группа: Бондаренко Мария Максимовна, БПИ2302
- Целевая оценка; выбранный трек 5, если нужен: TODO - Целевая оценка: 4
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
## Запуск из чистого clone ## 1. Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных. ### Зависимости и предусловия
- Установленный Docker и Docker Compose.
- Python 3.9+ для запуска локальных тестов.
## Архитектура и решения ### Пошаговые команды запуска и проверки
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab2, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации. 1. Создать .env из шаблона и заполнить локальные секреты. Задать в .env: POSTGRES_PASSWORD, SECRET_HMAC_KEY
2. Собрать и запустить стенд:
docker compose --env-file .env up -d --build
3. Дождаться готовности всех сервисов:
docker compose ps
curl -s http://localhost:8080/health/ready
4. Публичные проверки:
make test
make test-persistence ACTION_SCRIPT=scripts/restart.sh
make test-failover ACTION_SCRIPT=scripts/stop-one.sh
При повторном запуске последнего теста (test-failover) необходимо перезапустить контейнеры api-1 и api-2 с помощью команды docker compose start api-1 api-2 или docker compose restart
5. Остановка и очистка данных:
docker compose stop
docker compose down -v
## Доказательства критериев ## 2. Архитектура и решения
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства | - Точка входа и балансировка: Публичный доступ осуществляется строго через единую точку входа Nginx. Балансировщик распределяет HTTP-запросы между двумя API-репликами (api-1:8080 и api-2:8080) по алгоритму Round-Robin
- Пассивный Health Check и отказоустойчивость LB: В nginx настроены параметры max_fails=1 fail_timeout=5s и директива proxy_next_upstream error timeout http_502 http_503 http_504. В случае временной недоступности или сбоя одной из реплик Nginx автоматически перенаправляет запрос на соседний экземпляр без выдачи ошибки клиенту
- Идентификация реплик: Каждый ответ API содержит заголовок X-Instance-Id, жестко заданный через переменные окружения, что позволяет отслеживать конкретную реплику, обслужившую запрос
- Изолированный запуск миграций: Миграция структуры БД выделена в самостоятельный одноразовый сервис migrate, который ожидает готовности PostgreSQL и выполняет migrations/init.sql до старта API-реплик, предотвращая состояние гонки при создании таблиц
- Атомарная идемпотентность: В PostgreSQL создана таблица idempotency_keys с первичным ключом (user_id, method, path, key). Обработка Idempotency-Key выполняется в рамках одной транзакции СУБД с использованием INSERT ... ON CONFLICT DO NOTHING, что исключает появление дубликатов сообщений при параллельных повторных запросах через разные реплики API
- Безопасность и лимиты: Порты бэкендов api-1 и api-2 не публикуются наружу в обход Nginx. В compose.yaml для всех сервисов заданы аппаратные ограничения ресурсов. В Dockerfile и compose.yaml настроен Graceful Shutdown
## 3. Доказательства выполнения критериев
| Критерий README | Команда / Сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO | | Проверка публичного API | make test | PASS: lab3 public API scenario; infrastructure and grading evidence remain separate. | tests/smoke.py |
| Остальные пункты уровня 3 | TODO | TODO | TODO | | Сохранение данных при перезапуске | make test-persistence ACTION_SCRIPT=scripts/restart.sh | PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence. | scripts/restart.sh, tests/resilience.py |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO | | Выдерживание отказа реплики API | make test-failover ACTION_SCRIPT=scripts/stop-one.sh | PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence. | scripts/stop-one.sh, tests/resilience.py |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO | | Поочерёдное обновление | ./scripts/rolling-update.sh | Поочерёдная перезаборка api-1 и api-2 с ожиданием /health/ready и перезагрузкой Nginx | scripts/rolling-update.sh |
| Нагрузочное тестирование при Rolling Update | python3 tests/load.py --duration 60 --concurrency 4 --output evidence/load-rolling.json | | evidence/load-rolling.json |
| Ресурсные лимиты и изоляция | Описание в compose.yaml | Все сервисы имеют лимиты CPU/RAM; API доступен только через Nginx | compose.yaml |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены. ## 4. Эксперименты и отказы
## Эксперименты и отказы ### 4.1. Тест отказоустойчивости
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения. В ходе автоматизированного теста resilience.py производилась параллельная запись сообщений с проверкой балансировки:
- Наблюдаемые реплики до отказа: 2 (api-replica-1, api-replica-2)
- Воздействие: Выполнение scripts/stop-one.sh и принудительная остановка реплики TARGET_INSTANCE_ID с таймаутом docker compose stop -t 3
- Результат: Зафиксировано 77 успешных записей при 2 неудачных попытках в момент переключения; время восстановления составило 1.125 сек. Все подтвержденные сообщения сохранились без дублирования
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных | ### 4.2. Нагрузка во время поочерёдной замены
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
## Ограничения и следующие шаги Во время проведения поочерёдного обновления контейнеров (scripts/rolling-update.sh) выполнялся нагрузочный сценарий tests/load.py:
- Параметры: Длительность 60 с, параллелизм (concurrency) = 4.
- Метрики:
- - Успешные запросы: 1272
- - Ошибки: 0 (error_ratio: 0.0)
- - Средняя пропускная способность: 21.112 RPS
- - Латентность успешных запросов: p50 = 188.0 ms, p95 = 297.0 ms, p99 = 406.0 ms
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза. ## 5. Ограничения и следующие шаги
- Пассивный Health Check в Nginx OSS: Балансировщик Nginx OSS использует пассивное обнаружение отказов и повторный запрос через proxy_next_upstream. Активный поллинг ендпоинта /health/ready со стороны балансировщика без использования сторонних модулей или Nginx Plus не поддерживается
- Ограничения хоста: Тестирование проводилось на локальном стенде (Windows/Docker Desktop). При дальнейшем масштабировании узлов потребуется переход на оркестрацию уровня Docker Swarm
+644
View File
@@ -0,0 +1,644 @@
import os
import base64
import secrets
import uuid
import json
import re
from datetime import datetime, timezone, timedelta
from typing import List, Optional
from uuid import UUID
import psycopg2
import psycopg2.extras
import bcrypt
from fastapi import FastAPI, Header, HTTPException, Query, Response, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field, field_validator, ConfigDict
import redis
import hashlib
import hmac
app = FastAPI(title="MTUSI Messenger — lab3", version="1.3.0")
DATABASE_URL = os.getenv("DATABASE_URL", "postgresql://postgres:postgres@postgres:5432/messenger")
SESSION_TTL_SECONDS = int(os.getenv("SESSION_TTL_SECONDS", "3600"))
INSTANCE_ID = os.getenv("INSTANCE_ID", "instance-1")
REDIS_URL = os.getenv("REDIS_URL", "redis://redis:6379/0")
redis_client = redis.Redis.from_url(REDIS_URL, decode_responses=True)
SECRET_HMAC_KEY = os.environ["SECRET_HMAC_KEY"]
def get_db_connection():
conn = psycopg2.connect(DATABASE_URL, cursor_factory=psycopg2.extras.RealDictCursor)
return conn
def generate_request_id() -> str:
return uuid.uuid4().hex
def make_error_response(
status_code: int,
code: str,
message: str,
req_id: str,
extra_headers: Optional[dict] = None,
) -> JSONResponse:
headers = {"X-Request-Id": req_id, "X-Instance-Id": INSTANCE_ID}
if extra_headers:
headers.update(extra_headers)
return JSONResponse(
status_code=status_code,
headers=headers,
content={
"error": {
"code": code,
"message": message,
"request_id": req_id,
}
},
)
def encode_cursor(chat_id: str, created_at: str, msg_id: str) -> str:
payload = f"{chat_id}|{created_at}|{msg_id}"
signature = hmac.new(SECRET_HMAC_KEY.encode(), payload.encode(), hashlib.sha256).hexdigest()[:16]
raw = f"{payload}|{signature}"
return base64.urlsafe_b64encode(raw.encode()).decode()
def decode_cursor(cursor: str, expected_chat_id: str) -> tuple:
try:
decoded = base64.urlsafe_b64decode(cursor.encode()).decode()
parts = decoded.split("|")
if len(parts) != 4:
raise ValueError()
chat_id, created_at, msg_id, signature = parts
if chat_id != expected_chat_id:
raise HTTPException(status_code=400, detail="Cursor belongs to another chat")
expected_sig = hmac.new(SECRET_HMAC_KEY.encode(), f"{chat_id}|{created_at}|{msg_id}".encode(), hashlib.sha256).hexdigest()[:16]
if not hmac.compare_digest(signature, expected_sig):
raise ValueError()
return created_at, msg_id
except HTTPException:
raise
except Exception:
raise HTTPException(status_code=400, detail="Invalid cursor format")
@app.exception_handler(RequestValidationError)
async def validation_exception_handler(request: Request, exc: RequestValidationError):
req_id = request.headers.get("X-Request-Id") or generate_request_id()
return make_error_response(
status_code=400,
code="invalid_request",
message="Validation error in request body or parameters",
req_id=req_id
)
@app.exception_handler(HTTPException)
async def http_exception_handler(request: Request, exc: HTTPException):
req_id = request.headers.get("X-Request-Id") or generate_request_id()
headers = {"X-Request-Id": req_id, "X-Instance-Id": INSTANCE_ID}
if exc.headers:
headers.update(exc.headers)
detail = exc.detail
if isinstance(detail, dict) and "error" in detail:
detail["error"]["request_id"] = req_id
return JSONResponse(status_code=exc.status_code, headers=headers, content=detail)
code_map = {
400: "invalid_request",
401: "unauthorized",
404: "not_found",
409: "conflict",
413: "payload_too_large",
415: "unsupported_media_type",
429: "rate_limited",
503: "unavailable",
500: "internal_error",
}
return JSONResponse(
status_code=exc.status_code,
headers=headers,
content={
"error": {
"code": code_map.get(exc.status_code, "internal_error"),
"message": str(detail),
"request_id": req_id,
}
},
)
@app.exception_handler(Exception)
async def unhandled_exception_handler(request: Request, exc: Exception):
req_id = request.headers.get("X-Request-Id") or generate_request_id()
return make_error_response(
status_code=500,
code="internal_error",
message="Internal server error",
req_id=req_id,
)
class CreateUser(BaseModel):
model_config = ConfigDict(extra='forbid')
username: str = Field(..., pattern=r"^[a-z0-9_]{3,32}$")
display_name: str = Field(..., min_length=1, max_length=100)
password: str = Field(..., min_length=12, max_length=128)
@field_validator("display_name", "password")
def not_blank(cls, v):
if not v.strip():
raise ValueError("Must not be blank")
return v
class CreateChat(BaseModel):
model_config = ConfigDict(extra='forbid')
title: str = Field(..., min_length=1, max_length=100)
member_ids: List[uuid.UUID] = Field(..., min_items=1, max_items=100)
@field_validator("title")
def not_blank(cls, v):
if not v.strip():
raise ValueError("Must not be blank")
return v
class CreateMessage(BaseModel):
model_config = ConfigDict(extra='forbid')
text: str = Field(..., min_length=1, max_length=4000)
@field_validator("text")
def not_blank(cls, v):
if not v.strip():
raise ValueError("Must not be blank")
return v
@app.get("/health/live")
def health_live(response: Response):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
return {"status": "ok"}
@app.get("/health/ready")
def health_ready(response: Response):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
try:
conn = get_db_connection()
conn.close()
redis_client.ping()
except Exception:
raise HTTPException(status_code=503, detail="Database or Redis unavailable")
return {"status": "ok"}
@app.post("/api/v1/users", status_code=201)
def register_user(body: CreateUser, response: Response):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
password_hash = bcrypt.hashpw(body.password.encode('utf-8'), bcrypt.gensalt()).decode('utf-8')
user_id = str(uuid.uuid4())
created_at = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
conn = get_db_connection()
try:
with conn.cursor() as cur:
cur.execute(
"INSERT INTO users (id, username, display_name, password_hash, created_at) VALUES (%s, %s, %s, %s, %s)",
(user_id, body.username, body.display_name, password_hash, created_at)
)
conn.commit()
except psycopg2.errors.UniqueViolation:
conn.rollback()
raise HTTPException(status_code=409, detail={"error": {"code": "conflict", "message": "Username already taken", "request_id": req_id}})
finally:
conn.close()
return {
"id": user_id,
"username": body.username,
"display_name": body.display_name,
"created_at": created_at
}
@app.post("/api/v1/auth/sessions", status_code=201)
def create_session(authorization: Optional[str] = Header(None), response: Response = None):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
if not authorization or not authorization.startswith("Basic "):
raise HTTPException(
status_code=401,
detail="Missing or invalid basic auth",
headers={"WWW-Authenticate": 'Basic realm="messenger", charset="UTF-8"'}
)
try:
encoded = authorization.split(" ")[1]
decoded = base64.b64decode(encoded).decode("utf-8")
username, password = decoded.split(":", 1)
except Exception:
raise HTTPException(
status_code=401,
detail="Invalid credentials format",
headers={"WWW-Authenticate": 'Basic realm="messenger", charset="UTF-8"'}
)
conn = get_db_connection()
with conn.cursor() as cur:
cur.execute("SELECT * FROM users WHERE username = %s", (username,))
user = cur.fetchone()
if not user or not bcrypt.checkpw(password.encode('utf-8'), user['password_hash'].encode('utf-8')):
conn.close()
raise HTTPException(
status_code=401,
detail="Invalid credentials",
headers={"WWW-Authenticate": 'Basic realm="messenger", charset="UTF-8"'}
)
token = secrets.token_urlsafe(48)
redis_client.setex(f"session:{token}", SESSION_TTL_SECONDS, user['id'])
expires_at = datetime.now(timezone.utc) + timedelta(seconds=SESSION_TTL_SECONDS)
expires_at_str = expires_at.isoformat().replace("+00:00", "Z")
return {
"token": token,
"token_type": "Bearer",
"expires_at": expires_at_str,
"user": {
"id": user['id'],
"username": user['username'],
"display_name": user['display_name'],
"created_at": user['created_at'].isoformat().replace("+00:00", "Z") if isinstance(user['created_at'], datetime) else user['created_at']
}
}
def get_current_user_from_token(authorization: Optional[str] = Header(None)) -> dict:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(
status_code=401,
detail="Missing or invalid bearer token",
headers={"WWW-Authenticate": 'Bearer realm="messenger"'}
)
token = authorization.split(" ")[1]
try:
user_id = redis_client.get(f"session:{token}")
except Exception:
raise HTTPException(status_code=503, detail="Session store unavailable")
if not user_id:
raise HTTPException(
status_code=401,
detail="Invalid or expired session token",
headers={"WWW-Authenticate": 'Bearer realm="messenger"'}
)
conn = get_db_connection()
with conn.cursor() as cur:
cur.execute("SELECT * FROM users WHERE id = %s", (user_id,))
row = cur.fetchone()
conn.close()
if not row:
raise HTTPException(
status_code=401,
detail="User not found",
headers={"WWW-Authenticate": 'Bearer realm="messenger"'}
)
return {
"id": row['id'],
"username": row['username'],
"display_name": row['display_name'],
"created_at": row['created_at'].isoformat().replace("+00:00", "Z") if isinstance(row['created_at'], datetime) else row['created_at']
}
@app.delete("/api/v1/auth/sessions/current", status_code=204)
def revoke_session(authorization: Optional[str] = Header(None)):
req_id = generate_request_id()
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(
status_code=401,
detail="Unauthorized",
headers={"WWW-Authenticate": 'Bearer realm="messenger"'}
)
token = authorization.split(" ")[1]
deleted = redis_client.delete(f"session:{token}")
if not deleted:
raise HTTPException(
status_code=401,
detail="Invalid session token",
headers={"WWW-Authenticate": 'Bearer realm="messenger"'}
)
return Response(
status_code=204,
headers={
"X-Request-Id": req_id,
"X-Instance-Id": INSTANCE_ID
},
)
@app.get("/api/v1/users/me")
def current_user(response: Response, authorization: Optional[str] = Header(None)):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
return get_current_user_from_token(authorization)
@app.post("/api/v1/chats", status_code=201)
def create_chat(body: CreateChat, response: Response, authorization: Optional[str] = Header(None)):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
user = get_current_user_from_token(authorization)
member_ids_str = [str(mid) for mid in body.member_ids]
if user["id"] not in member_ids_str:
raise HTTPException(status_code=400, detail="Caller must be in member_ids")
if len(member_ids_str) != len(set(member_ids_str)):
raise HTTPException(status_code=400, detail="Duplicate member IDs")
conn = get_db_connection()
with conn.cursor() as cur:
for mid_str in member_ids_str:
cur.execute("SELECT id FROM users WHERE id = %s", (mid_str,))
if not cur.fetchone():
conn.close()
raise HTTPException(status_code=400, detail=f"User {mid_str} not found")
chat_id = str(uuid.uuid4())
created_at = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
try:
with conn.cursor() as cur:
cur.execute(
"INSERT INTO chats (id, title, created_at) VALUES (%s, %s, %s)",
(chat_id, body.title, created_at)
)
for mid_str in member_ids_str:
cur.execute(
"INSERT INTO chat_members (chat_id, user_id) VALUES (%s, %s)",
(chat_id, mid_str)
)
conn.commit()
except Exception:
conn.rollback()
conn.close()
raise HTTPException(status_code=500, detail="Internal error creating chat")
conn.close()
return {
"id": chat_id,
"title": body.title,
"member_ids": member_ids_str,
"created_at": created_at
}
@app.get("/api/v1/chats")
def list_chats(limit: int = 50, cursor: Optional[str] = None, response: Response = None, authorization: Optional[str] = Header(None)):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
user = get_current_user_from_token(authorization)
if limit < 1 or limit > 100:
raise HTTPException(status_code=400, detail="Invalid limit")
conn = get_db_connection()
with conn.cursor() as cur:
cur.execute("""
SELECT c.id, c.title, c.created_at FROM chats c
JOIN chat_members cm ON c.id = cm.chat_id
WHERE cm.user_id = %s
ORDER BY c.created_at ASC, c.id ASC
""", (user["id"],))
rows = cur.fetchall()
conn.close()
user_chats = []
for r in rows:
conn = get_db_connection()
with conn.cursor() as cur:
cur.execute("SELECT user_id FROM chat_members WHERE chat_id = %s", (r['id'],))
m_rows = cur.fetchall()
conn.close()
user_chats.append({
"id": r['id'],
"title": r['title'],
"member_ids": [m['user_id'] for m in m_rows],
"created_at": r['created_at'].isoformat().replace("+00:00", "Z") if isinstance(r['created_at'], datetime) else r['created_at']
})
start_idx = 0
if cursor:
try:
parts = cursor.split("_")
c_time, c_id = parts[0], parts[1]
found = False
for idx, c in enumerate(user_chats):
if (c["created_at"], c["id"]) >= (c_time, c_id):
start_idx = idx if (c["created_at"], c["id"]) > (c_time, c_id) else idx + 1
found = True
break
if not found:
start_idx = len(user_chats)
except Exception:
raise HTTPException(status_code=400, detail="Invalid cursor format")
page_items = user_chats[start_idx:start_idx + limit]
next_cursor = None
if start_idx + limit < len(user_chats):
last = page_items[-1]
next_cursor = f"{last['created_at']}_{last['id']}"
return {"items": page_items, "next_cursor": next_cursor}
@app.get("/api/v1/chats/{chat_id}")
def get_chat(chat_id: UUID, response: Response, authorization: Optional[str] = Header(None)):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
user = get_current_user_from_token(authorization)
chat_id_str = str(chat_id)
conn = get_db_connection()
with conn.cursor() as cur:
cur.execute("""
SELECT c.* FROM chats c
JOIN chat_members cm ON c.id = cm.chat_id
WHERE c.id = %s AND cm.user_id = %s
""", (chat_id_str, user["id"]))
chat = cur.fetchone()
if chat:
cur.execute("SELECT user_id FROM chat_members WHERE chat_id = %s", (chat_id_str,))
members = cur.fetchall()
conn.close()
if not chat:
raise HTTPException(status_code=404, detail="Chat not found")
return {
"id": chat['id'],
"title": chat['title'],
"member_ids": [m['user_id'] for m in members],
"created_at": chat['created_at'].isoformat().replace("+00:00", "Z") if isinstance(chat['created_at'], datetime) else chat['created_at']
}
@app.post("/api/v1/chats/{chat_id}/messages", status_code=201)
def create_message(
chat_id: UUID,
body: CreateMessage,
response: Response,
authorization: Optional[str] = Header(None),
idempotency_key: Optional[str] = Header(None, alias="Idempotency-Key")
):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
user = get_current_user_from_token(authorization)
if not idempotency_key or not re.match(r"^[A-Za-z0-9._:-]{1,128}$", idempotency_key):
raise HTTPException(status_code=400, detail="Missing or invalid Idempotency-Key")
chat_id_str = str(chat_id)
method = "POST"
path = f"/api/v1/chats/{chat_id_str}/messages"
body_json_str = json.dumps(body.model_dump(), sort_keys=True)
request_hash = hashlib.sha256(body_json_str.encode("utf-8")).hexdigest()
conn = get_db_connection()
try:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id_str, user["id"])
)
if not cur.fetchone():
conn.close()
raise HTTPException(status_code=404, detail="Chat not found")
msg_id = str(uuid.uuid4())
created_at = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
msg_response = {
"id": msg_id,
"chat_id": chat_id_str,
"sender_id": user["id"],
"text": body.text,
"created_at": created_at
}
cur.execute("""
INSERT INTO idempotency_keys (user_id, method, path, key, request_hash, response_status, response_body)
VALUES (%s, %s, %s, %s, %s, %s, %s)
ON CONFLICT (user_id, method, path, key) DO NOTHING
""", (user["id"], method, path, idempotency_key, request_hash, 201, json.dumps(msg_response)))
if cur.rowcount == 0:
cur.execute("""
SELECT request_hash, response_status, response_body
FROM idempotency_keys
WHERE user_id = %s AND method = %s AND path = %s AND key = %s
""", (user["id"], method, path, idempotency_key))
row = cur.fetchone()
conn.commit()
conn.close()
if row["request_hash"] != request_hash:
raise HTTPException(status_code=409, detail="Idempotency key conflict with different payload")
return row["response_body"]
cur.execute(
"INSERT INTO messages (id, chat_id, sender_id, text, created_at) VALUES (%s, %s, %s, %s, %s)",
(msg_id, chat_id_str, user["id"], body.text, created_at)
)
conn.commit()
except HTTPException:
raise
except Exception as exc:
conn.rollback()
conn.close()
raise HTTPException(status_code=500, detail="Internal error processing message")
finally:
if not conn.closed:
conn.close()
return msg_response
@app.get("/api/v1/chats/{chat_id}/messages")
def list_messages(chat_id: UUID, limit: int = 50, cursor: Optional[str] = None, response: Response = None, authorization: Optional[str] = Header(None)):
req_id = generate_request_id()
response.headers["X-Request-Id"] = req_id
response.headers["X-Instance-Id"] = INSTANCE_ID
user = get_current_user_from_token(authorization)
chat_id_str = str(chat_id)
conn = get_db_connection()
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id_str, user["id"])
)
if not cur.fetchone():
conn.close()
raise HTTPException(status_code=404, detail="Chat not found")
if limit < 1 or limit > 100:
conn.close()
raise HTTPException(status_code=400, detail="Invalid limit")
query = """
SELECT id, chat_id, sender_id, text, created_at FROM messages
WHERE chat_id = %s
"""
params = [chat_id_str]
if cursor:
c_time, c_id = decode_cursor(cursor, chat_id_str)
query += " AND (created_at, id) > (%s, %s)"
params.extend([c_time, c_id])
query += " ORDER BY created_at ASC, id ASC LIMIT %s"
params.append(limit + 1)
with conn.cursor() as cur:
cur.execute(query, tuple(params))
rows = cur.fetchall()
conn.close()
has_next = len(rows) > limit
page_items = rows[:limit]
formatted_msgs = []
for m in page_items:
formatted_msgs.append({
"id": m['id'],
"chat_id": m['chat_id'],
"sender_id": m['sender_id'],
"text": m['text'],
"created_at": m['created_at'].isoformat().replace("+00:00", "Z") if isinstance(m['created_at'], datetime) else m['created_at']
})
next_cursor = None
if has_next and formatted_msgs:
last = formatted_msgs[-1]
next_cursor = encode_cursor(chat_id_str, last['created_at'], last['id'])
return {"items": formatted_msgs, "next_cursor": next_cursor}
+130
View File
@@ -0,0 +1,130 @@
services:
postgres:
image: postgres:15.5-alpine3.19
environment:
POSTGRES_DB: messenger
POSTGRES_USER: postgres
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
volumes:
- pgdata:/var/lib/postgresql/data
deploy:
resources:
limits:
cpus: '0.50'
memory: 512M
reservations:
cpus: '0.10'
memory: 128M
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d messenger"]
interval: 3s
timeout: 3s
retries: 5
redis:
image: redis:7.2.4-alpine
command: redis-server --save 60 1 --loglevel warning
volumes:
- redisdata:/data
deploy:
resources:
limits:
cpus: '0.25'
memory: 256M
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 3s
timeout: 3s
retries: 5
migrate:
image: postgres:15.5-alpine3.19
environment:
PGPASSWORD: ${POSTGRES_PASSWORD}
volumes:
- ./migrations:/migrations
command: >
sh -c "until pg_isready -h postgres -U postgres; do sleep 1; done;
psql -h postgres -U postgres -d messenger -f /migrations/init.sql"
depends_on:
postgres:
condition: service_healthy
restart: "no"
api-1:
build: .
stop_grace_period: 15s
environment:
- DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432/messenger
- REDIS_URL=redis://redis:6379/0
- SESSION_TTL_SECONDS=3600
- INSTANCE_ID=api-replica-1
- SECRET_HMAC_KEY=${SECRET_HMAC_KEY}
deploy:
resources:
limits:
cpus: '0.50'
memory: 256M
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
migrate:
condition: service_completed_successfully
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8080/health/ready')"]
interval: 5s
timeout: 3s
retries: 3
api-2:
build: .
stop_grace_period: 15s
environment:
- DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432/messenger
- REDIS_URL=redis://redis:6379/0
- SESSION_TTL_SECONDS=3600
- INSTANCE_ID=api-replica-2
- SECRET_HMAC_KEY=${SECRET_HMAC_KEY}
deploy:
resources:
limits:
cpus: '0.50'
memory: 256M
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
migrate:
condition: service_completed_successfully
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8080/health/ready')"]
interval: 5s
timeout: 3s
retries: 3
nginx:
image: nginx:1.25.4-alpine
ports:
- "8080:8080"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
deploy:
resources:
limits:
cpus: '0.25'
memory: 128M
depends_on:
- api-1
- api-2
healthcheck:
test: ["CMD", "nginx", "-t"]
interval: 5s
timeout: 3s
retries: 3
volumes:
pgdata:
redisdata:
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 60.0,
"elapsed_seconds": 60.25,
"concurrency": 4,
"successful_requests": 1272,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 21.112,
"successful_latency_ms": {
"p50": 188.0,
"p95": 297.0,
"p99": 406.0
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+41
View File
@@ -0,0 +1,41 @@
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY,
username VARCHAR(32) UNIQUE NOT NULL,
display_name VARCHAR(100) NOT NULL,
password_hash VARCHAR(255) NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE IF NOT EXISTS chats (
id UUID PRIMARY KEY,
title VARCHAR(100) NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE IF NOT EXISTS chat_members (
chat_id UUID REFERENCES chats(id) ON DELETE CASCADE,
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
PRIMARY KEY (chat_id, user_id)
);
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY,
chat_id UUID REFERENCES chats(id) ON DELETE CASCADE,
sender_id UUID REFERENCES users(id) ON DELETE CASCADE,
text VARCHAR(4000) NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_messages_chat_pagination ON messages(chat_id, created_at, id);
CREATE TABLE IF NOT EXISTS idempotency_keys (
user_id UUID NOT NULL,
method VARCHAR(16) NOT NULL,
path TEXT NOT NULL,
key VARCHAR(128) NOT NULL,
request_hash VARCHAR(64) NOT NULL,
response_status INT NOT NULL,
response_body JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (user_id, method, path, key)
);
+28
View File
@@ -0,0 +1,28 @@
events { worker_connections 1024; }
http {
upstream messenger_api {
server api-1:8080 max_fails=1 fail_timeout=5s;
server api-2:8080 max_fails=1 fail_timeout=5s;
}
server {
listen 8080;
location / {
proxy_pass http://messenger_api;
proxy_connect_timeout 2s;
proxy_send_timeout 5s;
proxy_read_timeout 5s;
proxy_next_upstream error timeout http_502 http_503 http_504;
proxy_next_upstream_tries 2;
proxy_next_upstream_timeout 5s;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
+13
View File
@@ -0,0 +1,13 @@
fastapi==0.110.0
uvicorn==0.28.0
pydantic==2.6.2
starlette==0.36.3
idna==3.6
anyio==4.3.0
sniffio==1.3.1
click==8.1.7
h11==0.14.0
annotated-types==0.6.0
psycopg2-binary==2.9.9
bcrypt==4.1.2
redis==5.0.1
+3
View File
@@ -0,0 +1,3 @@
@echo off
docker compose restart
exit /b %ERRORLEVEL%
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
set -eu
docker compose restart
+35
View File
@@ -0,0 +1,35 @@
#!/bin/sh
set -eu
wait_ready() {
svc=$1
timeout=60
echo "Waiting for $svc readiness (up to ${timeout}s)..."
while [ "$timeout" -gt 0 ]; do
if docker compose exec -T "$svc" python -c \
"import urllib.request; urllib.request.urlopen('http://localhost:8080/health/ready')" 2>/dev/null; then
echo "$svc is ready!"
return 0
fi
sleep 1
timeout=$((timeout - 1))
done
echo "$svc failed to become ready" >&2
return 1
}
echo "Starting Rolling Update"
echo "1. Rebuilding and replacing api-1..."
docker compose up -d --no-deps --build --force-recreate api-1
wait_ready api-1
echo "Reloading nginx to pick up new api-1 IP..."
docker compose exec -T nginx nginx -s reload
echo "2. Rebuilding and replacing api-2..."
docker compose up -d --no-deps --build --force-recreate api-2
wait_ready api-2
echo "Reloading nginx to pick up new api-2 IP..."
docker compose exec -T nginx nginx -s reload
echo "Rolling Update Completed Successfully"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
set -eu
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
if [ "$TARGET_INSTANCE_ID" = "api-replica-1" ]; then
docker compose stop -t 3 api-1
elif [ "$TARGET_INSTANCE_ID" = "api-replica-2" ]; then
docker compose stop -t 3 api-2
else
echo "Unknown target instance: $TARGET_INSTANCE_ID" >&2
exit 1
fi
+1 -1
View File
@@ -14,7 +14,7 @@ import uuid
ROOT = Path(__file__).resolve().parents[1] ROOT = Path(__file__).resolve().parents[1]
LAB = json.loads((ROOT / 'lab.json').read_text())['number'] LAB = json.loads((ROOT / 'lab.json').read_text())['number']
SPEC = json.loads((ROOT / 'contracts/openapi.json').read_text()) SPEC = json.loads((ROOT / 'contracts/openapi.json').read_text(encoding='utf-8'))
class Failure(AssertionError): class Failure(AssertionError):
pass pass
+109
View File
@@ -0,0 +1,109 @@
import concurrent.futures
import urllib.request
import urllib.error
import json
import unittest
from client import Client
class TestMessengerLab4(unittest.TestCase):
def setUp(self):
self.client = Client()
self.alice = self.client.user("alice")
self.bob = self.client.user("bob")
self.eve = self.client.user("eve")
res = self.client.call(
'POST', '/api/v1/chats', 201,
actor=self.alice,
body={"title": "Test Chat", "member_ids": [self.alice["id"], self.bob["id"]]},
schema='Chat'
)
self.chat = res['json']
def test_unauthorized_access_to_chat(self):
self.client.call('GET', f"/api/v1/chats/{self.chat['id']}", 404, actor=self.eve)
def test_empty_message(self):
self.client.call('POST', f"/api/v1/chats/{self.chat['id']}/messages", 400, actor=self.alice, body={"text": ""})
def test_malformed_json(self):
url = self.client.base + f"/api/v1/chats/{self.chat['id']}/messages"
req = urllib.request.Request(
url,
data=b"{broken json content without quotes",
headers={
"Content-Type": "application/json",
"X-User-Id": self.alice["id"]
},
method="POST"
)
try:
with urllib.request.urlopen(req) as resp:
self.fail("Expected HTTP 400 for malformed JSON")
except urllib.error.HTTPError as e:
self.assertEqual(e.code, 400)
body = json.loads(e.read().decode())
self.assertEqual(body["error"]["code"], "invalid_request")
def test_duplicate_username(self):
self.client.call('POST', '/api/v1/users', 409, body=self.alice["registration"])
def test_pagination_multiple_pages(self):
import time
msg_texts = ["msg-page-1", "msg-page-2", "msg-page-3"]
for txt in msg_texts:
self.client.message(self.alice, self.chat, txt)
time.sleep(0.01)
path = f"/api/v1/chats/{self.chat['id']}/messages"
collected_items = []
cursor = None
for _ in range(5):
url_suffix = "?limit=1"
if cursor:
url_suffix += f"&cursor={cursor}"
page_res = self.client.call('GET', path + url_suffix, actor=self.alice, schema='MessagePage')
page = page_res['json']
for item in page['items']:
self.assertNotIn(item['id'], [x['id'] for x in collected_items])
collected_items.append(item)
cursor = page.get('next_keyword') or page.get('next_cursor')
if not cursor:
break
self.assertEqual(len(collected_items), 3)
self.assertEqual(sorted([x['text'] for x in collected_items]), sorted(msg_texts))
def test_concurrent_messages_strict(self):
def worker_send(worker_idx):
local_client = Client()
ids = []
for i in range(10):
res = local_client.message(self.alice, self.chat, f"w-{worker_idx}-msg-{i}")
ids.append(res['json']['id'])
return ids
all_ids = []
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
futures = [executor.submit(worker_send, w) for w in range(10)]
for f in concurrent.futures.as_completed(futures):
all_ids.extend(f.result())
self.assertEqual(len(all_ids), 100)
self.assertEqual(len(set(all_ids)), 100)
page = self.client.call(
'GET', f"/api/v1/chats/{self.chat['id']}/messages?limit=100",
actor=self.alice,
schema='MessagePage'
)
server_ids = [m['id'] for m in page['json']['items']]
self.assertEqual(len(server_ids), 100)
self.assertEqual(len(set(server_ids)), 100)
if __name__ == '__main__':
unittest.main()
+1 -1
View File
@@ -55,7 +55,7 @@ def main():
check(len(observed)>=2,'less than two API instances observed before failure; verify LB/session affinity') check(len(observed)>=2,'less than two API instances observed before failure; verify LB/session affinity')
print('Executing explicit '+args.mode+' hook; hook output suppressed to avoid leaking secrets.') print('Executing explicit '+args.mode+' hook; hook output suppressed to avoid leaking secrets.')
# No shell interpolation and no Docker commands in this test. # No shell interpolation and no Docker commands in this test.
proc = subprocess.Popen([str(script)],env=env,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL) proc = subprocess.Popen(['sh', str(script)], env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
started = time.monotonic() started = time.monotonic()
successes,failures,after_instances = [],0,[] successes,failures,after_instances = [],0,[]
try: try: