1 Commits
Author SHA1 Message Date
Xenia 0a2eeda753 перенесенная вторая и лабораторная 3 2026-11-20 17:32:05 +03:00
18 changed files with 975 additions and 13 deletions
+6 -2
View File
@@ -1,4 +1,9 @@
.git
.venv
__pycache__
*.pyc
.env
evidence.git
.github
.env
.env.*
@@ -16,5 +21,4 @@ backups
.venv
__pycache__
node_modules
evidence
.DS_Store
evidence
+15
View File
@@ -0,0 +1,15 @@
FROM python:3.12.7-slim-bookworm
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app
COPY app.py server.py .
COPY messenger ./messenger
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN useradd --create-home --uid 10001 appuser
USER appuser
EXPOSE 8080
HEALTHCHECK --interval=10s --timeout=3s --start-period=2s --retries=3 CMD python -c "from urllib.request import urlopen; urlopen('http://127.0.0.1:8080/health/ready', timeout=2)"
CMD ["python", "server.py"]
+19 -11
View File
@@ -1,24 +1,32 @@
# Отчёт — лабораторная 3
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: Барановская Ксения Игоревна, БПИ2304
- Целевая оценка: 3
- Commit/PR: локальная работа
- Использованные источники, библиотеки и помощь ИИ: Python stdlib, psycopg 3.2.3, PostgreSQL 16.4, Nginx 1.27.2; помощь ИИ использована для реализации каркаса.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Требуется Docker Compose. Запуск из корня:
```sh
docker compose up -d --build
make test
```
Схема PostgreSQL создаётся приложением один раз при старте первой реплики. Готовность проверяется healthcheck. Остановка без удаления данных: `docker compose down`; намеренное удаление учебных данных: `docker compose down -v`.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab2, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
Клиент обращается к Nginx (`localhost:8080`), который балансирует `api1` и `api2`. Реплики не публикуют порты наружу. PostgreSQL хранит пользователей, чаты, сообщения, сессии и ключи идемпотентности; volume `postgres_data` переживает перезапуск. При каждом запросе API перечитывает общий state из PostgreSQL, поэтому сессия и история не зависят от конкретной реплики. Запись сообщения и ключ идемпотентности фиксируются одной транзакцией с уникальным ключом `(user, chat, Idempotency-Key)`.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Публичный smoke | `make test` | PASS после запуска Compose | `tests/smoke.py` |
| Две реплики и LB | `docker compose ps`, повторные запросы | `X-Instance-Id: api1/api2` | `compose.yaml`, `infra/nginx.conf` |
| Отказ одной API-реплики | `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` | LB продолжает обслуживать запросы, PostgreSQL сохраняет данные | `scripts/stop-one.sh`, `tests/resilience.py` |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
@@ -26,12 +34,12 @@ TODO: добавить отдельную строку для каждого з
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Для уровня 3 измеряется только корректность отказа. Nginx исключает остановленный backend после неудачных проверок; точное время зависит от `max_fails=2` и `fail_timeout=5s`. Сценарий повторяет неопределённые POST с тем же ключом и проверяет отсутствие дублей.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| остановка `api1` или `api2` | новые запросы продолжаются через вторую реплику | проверяется `tests/resilience.py` | не более 10 секунд по контракту |
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
Стенд выдерживает отказ одной API-реплики. PostgreSQL остаётся единой точкой отказа и не заявляется как отказоустойчивый. Сессии хранятся в PostgreSQL, а не в памяти процесса. Параметры CPU/RAM и rolling update уровня 4 не заявлены.
+21
View File
@@ -0,0 +1,21 @@
"""Compatibility entry point and test imports for the modular messenger."""
from http.server import ThreadingHTTPServer
from messenger.http import make_handler
from messenger.service import MessengerService
from messenger.state import State
state = State()
service = MessengerService(state)
Handler = make_handler(service)
def main():
from server import main as run_server
run_server()
if __name__ == "__main__":
main()
+59
View File
@@ -0,0 +1,59 @@
services:
postgres:
image: postgres:16.4
environment:
POSTGRES_DB: messenger
POSTGRES_USER: messenger
POSTGRES_PASSWORD: messenger-local-only
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 3s
timeout: 3s
retries: 20
api1:
build: .
environment:
DATABASE_URL: postgresql://messenger:messenger-local-only@postgres:5432/messenger
INSTANCE_ID: api1
SESSION_TTL_SECONDS: 3600
depends_on:
postgres:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c", "from urllib.request import urlopen; urlopen('http://127.0.0.1:8080/health/ready', timeout=2)"]
interval: 3s
timeout: 3s
retries: 20
api2:
build: .
environment:
DATABASE_URL: postgresql://messenger:messenger-local-only@postgres:5432/messenger
INSTANCE_ID: api2
SESSION_TTL_SECONDS: 3600
depends_on:
postgres:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c", "from urllib.request import urlopen; urlopen('http://127.0.0.1:8080/health/ready', timeout=2)"]
interval: 3s
timeout: 3s
retries: 20
lb:
image: nginx:1.27.2-alpine
ports:
- "8080:80"
volumes:
- ./infra/nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
api1:
condition: service_healthy
api2:
condition: service_healthy
volumes:
postgres_data:
+20
View File
@@ -0,0 +1,20 @@
events {}
http {
upstream messenger_api {
server api1:8080 max_fails=2 fail_timeout=5s;
server api2:8080 max_fails=2 fail_timeout=5s;
}
server {
listen 80;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
proxy_next_upstream error timeout http_502 http_503 http_504;
location / {
proxy_pass http://messenger_api;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
+1
View File
@@ -0,0 +1 @@
"""In-memory messenger application."""
+29
View File
@@ -0,0 +1,29 @@
import json
import re
from datetime import datetime, timezone
UUID_RE = re.compile(r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-5][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$")
USERNAME_RE = re.compile(r"^[a-z0-9_]{3,32}$")
def now():
return datetime.now(timezone.utc).isoformat(timespec="milliseconds").replace("+00:00", "Z")
def compact_json(value):
return json.dumps(value, ensure_ascii=False, separators=(",", ":"))
def require_uuid(value):
return isinstance(value, str) and UUID_RE.fullmatch(value) is not None
def validate_object(body, allowed, required):
if not isinstance(body, dict) or set(body) - set(allowed) or not set(required).issubset(body):
from .errors import ApiError
raise ApiError(400, "invalid_request", "Неверный формат запроса")
def validate_text(value, maximum):
return isinstance(value, str) and value.strip() and len(value) <= maximum
+5
View File
@@ -0,0 +1,5 @@
class ApiError(Exception):
def __init__(self, status, code, message):
self.status = status
self.code = code
self.message = message
+106
View File
@@ -0,0 +1,106 @@
import json
import logging
import os
import re
import uuid
from http.server import BaseHTTPRequestHandler
from urllib.parse import parse_qs, urlparse
from .domain import compact_json
from .errors import ApiError
from .service import MessengerService
def make_handler(service: MessengerService):
class Handler(BaseHTTPRequestHandler):
server_version = "Messenger/1.0"
def log_message(self, fmt, *args):
logging.info("%s %s", self.command, self.path.split("?", 1)[0])
def send_json(self, status, payload, headers=None):
request_id = str(uuid.uuid4())
if status >= 400 and isinstance(payload, dict) and "error" in payload:
payload["error"]["request_id"] = request_id
data = compact_json(payload).encode("utf-8")
if status == 204:
data = b""
self.send_response(status)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(data)))
self.send_header("X-Request-Id", request_id)
self.send_header("X-Instance-Id", os.environ.get("INSTANCE_ID", "local"))
for key, value in (headers or {}).items():
self.send_header(key, value)
self.end_headers()
if status != 204:
self.wfile.write(data)
def send_error_json(self, error):
headers = {}
if error.status == 401:
if self.path.split("?", 1)[0] == "/api/v1/auth/sessions":
headers["WWW-Authenticate"] = 'Basic realm="messenger", charset="UTF-8"'
else:
headers["WWW-Authenticate"] = 'Bearer realm="messenger"'
self.send_json(error.status, {"error": {"code": error.code, "message": error.message, "request_id": ""}}, headers)
def read_json(self):
if self.headers.get_content_type() != "application/json":
raise ApiError(400, "invalid_request", "Ожидается application/json")
try:
length = int(self.headers.get("Content-Length", "-1"))
if length < 0 or length > 1024 * 1024:
raise ValueError
return json.loads(self.rfile.read(length).decode("utf-8"))
except (ValueError, UnicodeDecodeError, json.JSONDecodeError):
raise ApiError(400, "invalid_request", "Некорректный JSON")
def route(self):
parsed = urlparse(self.path)
path = parsed.path
query = parse_qs(parsed.query, keep_blank_values=True)
if self.command == "GET" and path in ("/health/live", "/health/ready"):
return self.send_json(200, {"status": "ok"})
if self.command == "POST" and path == "/api/v1/users":
return self.send_json(201, service.create_user(self.read_json()))
if self.command == "POST" and path == "/api/v1/auth/sessions":
return self.send_json(201, service.create_session(self.headers.get("Authorization", "")))
if self.command == "DELETE" and path == "/api/v1/auth/sessions/current":
service.revoke_session(self.headers)
return self.send_json(204, None)
if path == "/api/v1/users/me" and self.command == "GET":
return self.send_json(200, service.current_user(self.headers))
if path == "/api/v1/chats" and self.command == "POST":
return self.send_json(201, service.create_chat(self.headers, self.read_json()))
if path == "/api/v1/chats" and self.command == "GET":
return self.send_json(200, service.list_chats(self.headers, query))
match = re.fullmatch(r"/api/v1/chats/([^/]+)", path)
if match and self.command == "GET":
return self.send_json(200, service.get_chat(self.headers, match.group(1)))
match = re.fullmatch(r"/api/v1/chats/([^/]+)/messages", path)
if match and self.command == "POST":
return self.send_json(201, service.create_message(self.headers, match.group(1), self.read_json()))
if match and self.command == "GET":
return self.send_json(200, service.list_messages(self.headers, match.group(1), query))
raise ApiError(404, "not_found", "Объект не найден")
def do_GET(self):
self.handle_request()
def do_POST(self):
self.handle_request()
def do_DELETE(self):
self.handle_request()
def handle_request(self):
try:
self.route()
except ApiError as error:
self.send_error_json(error)
except Exception:
logging.exception("request failed")
self.send_error_json(ApiError(500, "internal_error", "Внутренняя ошибка"))
return Handler
+292
View File
@@ -0,0 +1,292 @@
import base64
import binascii
import hashlib
import hmac
import os
import secrets
import time
import uuid
from datetime import datetime, timezone
from .domain import USERNAME_RE, now, require_uuid, validate_object, validate_text
from .errors import ApiError
from .state import State, stored_time
import re
class MessengerService:
def __init__(self, state: State):
self.state = state
def actor(self, headers):
self.state.refresh()
authorization = headers.get("Authorization", "")
if not authorization.startswith("Bearer "):
raise ApiError(401, "unauthorized", "Требуется Bearer-сессия")
token = authorization[7:].strip()
with self.state.lock:
session = self.state.sessions.get(token)
user = self.state.users.get(session["user_id"]) if session else None
if session and (session["revoked"] or session["expires_at"] <= time.time()):
user = None
if not user:
raise ApiError(401, "unauthorized", "Требуется действительная сессия")
return user
@staticmethod
def public_user(user):
return {key: value for key, value in user.items() if not key.startswith("_")}
@staticmethod
def _password_hash(password):
salt = secrets.token_bytes(16)
digest = hashlib.scrypt(password.encode("utf-8"), salt=salt, n=16384, r=8, p=1)
return "scrypt$16384$8$1$%s$%s" % (salt.hex(), digest.hex())
@staticmethod
def _password_matches(password, encoded):
try:
_, n, r, p, salt, digest = encoded.split("$")
actual = hashlib.scrypt(password.encode("utf-8"), salt=bytes.fromhex(salt),
n=int(n), r=int(r), p=int(p))
return hmac.compare_digest(actual.hex(), digest)
except (ValueError, TypeError):
return False
def create_user(self, body):
self.state.refresh()
validate_object(body, ("username", "display_name", "password"), ("username", "display_name", "password"))
username = body["username"]
display_name = body["display_name"]
password = body["password"]
if (
not isinstance(username, str)
or not isinstance(display_name, str)
or not USERNAME_RE.fullmatch(username)
or not validate_text(display_name, 100)
or not isinstance(password, str)
or not 12 <= len(password) <= 128
):
raise ApiError(400, "invalid_request", "Некорректные данные пользователя")
with self.state.lock:
if username in self.state.usernames:
raise ApiError(409, "conflict", "Username уже занят")
user = {"id": str(uuid.uuid4()), "username": username, "display_name": display_name,
"created_at": now(), "_password_hash": self._password_hash(password)}
self.state.users[user["id"]] = user
self.state.usernames[username] = user["id"]
self.state.execute(
"INSERT INTO users (id, username, display_name, password_hash, created_at) VALUES (%s,%s,%s,%s,%s)",
(user["id"], username, display_name, user["_password_hash"], user["created_at"]))
return self.public_user(user)
def create_session(self, authorization):
self.state.refresh()
if not authorization.startswith("Basic "):
raise ApiError(401, "unauthorized", "Неверные credentials")
try:
decoded = base64.b64decode(authorization[6:], validate=True).decode("utf-8")
username, password = decoded.split(":", 1)
except (ValueError, UnicodeDecodeError, binascii.Error):
raise ApiError(401, "unauthorized", "Неверные credentials")
with self.state.lock:
user = self.state.users.get(self.state.usernames.get(username, ""))
if not user or not self._password_matches(password, user["_password_hash"]):
raise ApiError(401, "unauthorized", "Неверные credentials")
expires = time.time() + int(os.environ.get("SESSION_TTL_SECONDS", "3600"))
token = secrets.token_urlsafe(32)
with self.state.lock:
self.state.sessions[token] = {"user_id": user["id"], "expires_at": expires, "revoked": False}
self.state.execute("INSERT INTO sessions (token,user_id,expires_at) VALUES (%s,%s,to_timestamp(%s))",
(token, user["id"], expires))
expires_at = datetime.fromtimestamp(expires, timezone.utc).isoformat(timespec="milliseconds").replace("+00:00", "Z")
return {"token": token, "token_type": "Bearer", "expires_at": expires_at,
"user": self.public_user(user)}
def revoke_session(self, headers):
authorization = headers.get("Authorization", "")
if not authorization.startswith("Bearer "):
raise ApiError(401, "unauthorized", "Требуется Bearer-сессия")
token = authorization[7:].strip()
self.actor(headers)
with self.state.lock:
self.state.sessions[token]["revoked"] = True
self.state.execute("UPDATE sessions SET revoked=TRUE WHERE token=%s", (token,))
def current_user(self, headers):
return self.public_user(self.actor(headers))
def create_chat(self, headers, body):
self.state.refresh()
actor = self.actor(headers)
validate_object(body, ("title", "member_ids"), ("title", "member_ids"))
members = body["member_ids"]
if not validate_text(body["title"], 100) or not isinstance(members, list) or not 1 <= len(members) <= 100:
raise ApiError(400, "invalid_request", "Некорректные данные чата")
if not all(isinstance(item, str) and require_uuid(item) for item in members):
raise ApiError(400, "invalid_request", "Некорректный список участников")
if len(set(members)) != len(members):
raise ApiError(400, "invalid_request", "Некорректный список участников")
with self.state.lock:
if actor["id"] not in members:
raise ApiError(400, "invalid_request", "Вызывающий должен быть участником")
if any(member not in self.state.users for member in members):
raise ApiError(400, "invalid_request", "Неизвестный участник")
chat = {"id": str(uuid.uuid4()), "title": body["title"], "member_ids": members, "created_at": now()}
self.state.chats[chat["id"]] = chat
self.state.messages[chat["id"]] = []
self.state.transaction([
("INSERT INTO chats (id,title,created_at) VALUES (%s,%s,%s)",
(chat["id"], chat["title"], chat["created_at"])),
("INSERT INTO chat_members (chat_id,user_id) VALUES " + ",".join(["(%s,%s)"] * len(members)),
tuple(value for member in members for value in (chat["id"], member))),
])
return chat
def accessible_chat(self, chat_id, actor):
if not require_uuid(chat_id):
raise ApiError(400, "invalid_request", "Некорректный UUID чата")
with self.state.lock:
chat = self.state.chats.get(chat_id)
if not chat or actor["id"] not in chat["member_ids"]:
raise ApiError(404, "not_found", "Объект не найден")
return chat
def list_chats(self, headers, query):
actor = self.actor(headers)
limit = self.limit(query)
cursor = query.get("cursor", [None])[0]
with self.state.lock:
chats = [chat for chat in self.state.chats.values() if actor["id"] in chat["member_ids"]]
chats.sort(key=lambda item: (item["created_at"], item["id"]))
if cursor:
try:
padded_cursor = cursor + "=" * (-len(cursor) % 4)
cursor_time, cursor_id = base64.urlsafe_b64decode(padded_cursor.encode()).decode().split("|", 1)
except (ValueError, UnicodeDecodeError, binascii.Error):
raise ApiError(400, "invalid_request", "Некорректный cursor")
if not require_uuid(cursor_id) or not any(
item["id"] == cursor_id and item["created_at"] == cursor_time for item in chats
):
raise ApiError(400, "invalid_request", "Cursor не принадлежит этому списку")
chats = [item for item in chats if (item["created_at"], item["id"]) > (cursor_time, cursor_id)]
page = chats[:limit]
next_cursor = None
if len(chats) > limit:
last = page[-1]
next_cursor = base64.urlsafe_b64encode(
(last["created_at"] + "|" + last["id"]).encode()
).decode().rstrip("=")
return {"items": page, "next_cursor": next_cursor}
def get_chat(self, headers, chat_id):
return self.accessible_chat(chat_id, self.actor(headers))
def create_message(self, headers, chat_id, body):
actor = self.actor(headers)
chat = self.accessible_chat(chat_id, actor)
key = headers.get("Idempotency-Key", "")
if not re.fullmatch(r"[A-Za-z0-9._:-]{1,128}", key):
raise ApiError(400, "invalid_request", "Требуется корректный Idempotency-Key")
validate_object(body, ("text",), ("text",))
if not validate_text(body["text"], 4000):
raise ApiError(400, "invalid_request", "Сообщение не может быть пустым")
request_hash = hashlib.sha256(body["text"].encode("utf-8")).hexdigest()
if self.state.connection_factory:
message_id = str(uuid.uuid4())
created_at = now()
with self.state.connection_factory(self.state.database_url) as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT request_hash, message_id FROM message_idempotency "
"WHERE user_id=%s AND chat_id=%s AND idempotency_key=%s FOR UPDATE",
(actor["id"], chat_id, key))
existing = cur.fetchone()
if existing:
if existing[0] != request_hash:
raise ApiError(409, "conflict", "Ключ уже использован для другого сообщения")
cur.execute("SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE id=%s",
(existing[1],))
row = cur.fetchone()
if row:
return self._message_from_row(row)
cur.execute(
"INSERT INTO messages (id,chat_id,sender_id,text,created_at) VALUES (%s,%s,%s,%s,%s)",
(message_id, chat_id, actor["id"], body["text"], created_at))
cur.execute(
"INSERT INTO message_idempotency "
"(user_id,chat_id,idempotency_key,request_hash,message_id,created_at) "
"VALUES (%s,%s,%s,%s,%s,%s) "
"ON CONFLICT (user_id,chat_id,idempotency_key) DO NOTHING "
"RETURNING message_id",
(actor["id"], chat_id, key, request_hash, message_id, created_at))
owner = cur.fetchone()
if not owner or str(owner[0]) != message_id:
cur.execute("DELETE FROM messages WHERE id=%s", (message_id,))
cur.execute("SELECT request_hash, message_id FROM message_idempotency "
"WHERE user_id=%s AND chat_id=%s AND idempotency_key=%s",
(actor["id"], chat_id, key))
winner_hash, winner_id = cur.fetchone()
if winner_hash != request_hash:
raise ApiError(409, "conflict", "Ключ уже использован для другого сообщения")
cur.execute("SELECT id, chat_id, sender_id, text, created_at FROM messages WHERE id=%s",
(winner_id,))
return self._message_from_row(cur.fetchone())
conn.commit()
self.state.refresh()
return {"id": message_id, "chat_id": chat_id, "sender_id": actor["id"],
"text": body["text"], "created_at": created_at}
with self.state.lock:
identity = (actor["id"], chat_id, key)
existing = self.state.idempotency.get(identity)
if existing:
if existing[0] != request_hash:
raise ApiError(409, "conflict", "Ключ уже использован для другого сообщения")
return existing[1]
message = {"id": str(uuid.uuid4()), "chat_id": chat["id"], "sender_id": actor["id"], "text": body["text"], "created_at": now()}
self.state.messages[chat_id].append(message)
self.state.idempotency[identity] = (request_hash, message)
self.state.execute(
"INSERT INTO messages (id,chat_id,sender_id,text,created_at) VALUES (%s,%s,%s,%s,%s)",
(message["id"], chat_id, actor["id"], message["text"], message["created_at"]))
return message
@staticmethod
def _message_from_row(row):
message_id, chat_id, sender_id, text, created_at = row
return {"id": str(message_id), "chat_id": str(chat_id), "sender_id": str(sender_id),
"text": text, "created_at": stored_time(created_at)}
def limit(self, query):
raw = query.get("limit", ["50"])[0]
try:
value = int(raw)
except ValueError:
raise ApiError(400, "invalid_request", "Некорректный limit")
if not 1 <= value <= 100:
raise ApiError(400, "invalid_request", "limit должен быть от 1 до 100")
return value
def list_messages(self, headers, chat_id, query):
actor = self.actor(headers)
self.accessible_chat(chat_id, actor)
limit = self.limit(query)
cursor = query.get("cursor", [None])[0]
with self.state.lock:
messages = list(self.state.messages[chat_id])
messages.sort(key=lambda item: (item["created_at"], item["id"]))
if cursor:
try:
padded_cursor = cursor + "=" * (-len(cursor) % 4)
cursor_time, cursor_id = base64.urlsafe_b64decode(padded_cursor.encode()).decode().split("|", 1)
except (ValueError, UnicodeDecodeError, binascii.Error):
raise ApiError(400, "invalid_request", "Некорректный cursor")
if not require_uuid(cursor_id) or not any(item["id"] == cursor_id and item["created_at"] == cursor_time for item in messages):
raise ApiError(400, "invalid_request", "Cursor не принадлежит этому чату")
messages = [item for item in messages if (item["created_at"], item["id"]) > (cursor_time, cursor_id)]
page = messages[:limit]
next_cursor = None
if len(messages) > limit:
last = page[-1]
next_cursor = base64.urlsafe_b64encode((last["created_at"] + "|" + last["id"]).encode()).decode().rstrip("=")
return {"items": page, "next_cursor": next_cursor}
+130
View File
@@ -0,0 +1,130 @@
from dataclasses import dataclass, field
import os
import threading
SCHEMA = """
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY, username VARCHAR(32) UNIQUE NOT NULL,
display_name VARCHAR(100) NOT NULL, password_hash TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE IF NOT EXISTS chats (
id UUID PRIMARY KEY, title VARCHAR(100) NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE IF NOT EXISTS chat_members (
chat_id UUID REFERENCES chats(id) ON DELETE CASCADE,
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
PRIMARY KEY (chat_id, user_id)
);
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY, chat_id UUID REFERENCES chats(id) ON DELETE CASCADE,
sender_id UUID REFERENCES users(id), text TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
CREATE INDEX IF NOT EXISTS messages_chat_created_idx ON messages(chat_id, created_at, id);
CREATE TABLE IF NOT EXISTS sessions (
token TEXT PRIMARY KEY, user_id UUID REFERENCES users(id) ON DELETE CASCADE,
expires_at TIMESTAMPTZ NOT NULL, revoked BOOLEAN NOT NULL DEFAULT FALSE
);
CREATE TABLE IF NOT EXISTS message_idempotency (
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
idempotency_key VARCHAR(128) NOT NULL,
request_hash CHAR(64) NOT NULL,
message_id UUID NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
created_at TIMESTAMPTZ NOT NULL,
PRIMARY KEY (user_id, chat_id, idempotency_key)
);
"""
def stored_time(value):
return value.isoformat(timespec="milliseconds").replace("+00:00", "Z")
@dataclass
class State:
users: dict[str, dict] = field(default_factory=dict)
usernames: dict[str, str] = field(default_factory=dict)
chats: dict[str, dict] = field(default_factory=dict)
messages: dict[str, list[dict]] = field(default_factory=dict)
sessions: dict[str, dict] = field(default_factory=dict)
idempotency: dict[tuple[str, str, str], tuple[str, dict]] = field(default_factory=dict)
lock: threading.RLock = field(default_factory=threading.RLock)
database_url: str = field(default_factory=lambda: os.environ.get("DATABASE_URL", ""))
connection_factory: object = field(init=False, default=None)
def __post_init__(self):
if not self.database_url:
return
try:
import psycopg
self.connection_factory = psycopg.connect
with self.connection_factory(self.database_url, autocommit=True) as conn:
with conn.cursor() as cur:
cur.execute(SCHEMA)
self.refresh()
except ImportError as exc:
raise RuntimeError("DATABASE_URL задан, но пакет psycopg не установлен") from exc
def refresh(self):
if not self.connection_factory:
return
with self.connection_factory(self.database_url, autocommit=True) as conn:
with conn.cursor() as cur:
cur.execute("SELECT id, username, display_name, created_at, password_hash FROM users")
users = cur.fetchall()
cur.execute("SELECT id, title, created_at FROM chats")
chats = cur.fetchall()
cur.execute("SELECT chat_id, user_id FROM chat_members ORDER BY chat_id, user_id")
members = cur.fetchall()
cur.execute("SELECT id, chat_id, sender_id, text, created_at FROM messages ORDER BY created_at, id")
messages = cur.fetchall()
cur.execute("SELECT token, user_id, expires_at, revoked FROM sessions")
sessions = cur.fetchall()
with self.lock:
self.users.clear()
self.usernames.clear()
for user_id, username, display_name, created_at, password_hash in users:
user = {"id": str(user_id), "username": username, "display_name": display_name,
"created_at": stored_time(created_at),
"_password_hash": password_hash}
self.users[user["id"]] = user
self.usernames[username] = user["id"]
self.chats.clear()
self.messages.clear()
member_map = {}
for chat_id, user_id in members:
member_map.setdefault(str(chat_id), []).append(str(user_id))
for chat_id, title, created_at in chats:
self.chats[str(chat_id)] = {"id": str(chat_id), "title": title,
"member_ids": member_map.get(str(chat_id), []),
"created_at": stored_time(created_at)}
self.messages[str(chat_id)] = []
for message_id, chat_id, sender_id, text, created_at in messages:
self.messages.setdefault(str(chat_id), []).append({
"id": str(message_id), "chat_id": str(chat_id), "sender_id": str(sender_id),
"text": text, "created_at": stored_time(created_at)})
self.sessions.clear()
for token, user_id, expires_at, revoked in sessions:
self.sessions[token] = {"user_id": str(user_id), "expires_at": expires_at.timestamp(),
"revoked": revoked}
def execute(self, query, params=()):
if not self.connection_factory:
return
with self.connection_factory(self.database_url) as conn:
with conn.cursor() as cur:
cur.execute(query, params)
conn.commit()
def transaction(self, statements):
if not self.connection_factory:
return
with self.connection_factory(self.database_url) as conn:
with conn.cursor() as cur:
for query, params in statements:
cur.execute(query, params)
conn.commit()
+1
View File
@@ -0,0 +1 @@
psycopg[binary]==3.2.3
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
set -eu
docker compose up -d --build
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
set -eu
: "${TARGET_INSTANCE_ID:?TARGET_INSTANCE_ID is required}"
case "$TARGET_INSTANCE_ID" in
api1|api2) docker compose stop "$TARGET_INSTANCE_ID" ;;
*) echo "unknown API instance" >&2; exit 2 ;;
esac
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
import logging
import os
import signal
import threading
from http.server import ThreadingHTTPServer
from messenger.http import make_handler
from messenger.service import MessengerService
from messenger.state import State
def main():
logging.basicConfig(
level=os.environ.get("LOG_LEVEL", "INFO"),
format="%(asctime)s %(levelname)s %(message)s",
)
host = os.environ.get("APP_HOST", "0.0.0.0")
port = int(os.environ.get("APP_PORT", "8080"))
state = State()
service = MessengerService(state)
server = ThreadingHTTPServer((host, port), make_handler(service))
def stop(signum, _frame):
logging.info("received signal %s, stopping", signum)
threading.Thread(target=server.shutdown, daemon=True).start()
signal.signal(signal.SIGTERM, stop)
signal.signal(signal.SIGINT, stop)
logging.info("messenger listening on %s:%s", host, port)
try:
server.serve_forever(poll_interval=0.2)
finally:
server.server_close()
logging.info("messenger stopped")
if __name__ == "__main__":
main()
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env python3
"""Demonstrate that restarting lab1 clears the in-memory state."""
import argparse
import json
import os
import subprocess
import time
import uuid
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
def request(base_url, method, path, body=None, actor=None):
headers = {"Accept": "application/json"}
if actor:
headers["X-User-Id"] = actor
data = None
if body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
headers["Content-Type"] = "application/json"
request = Request(base_url + path, data=data, headers=headers, method=method)
try:
response = urlopen(request, timeout=3)
except HTTPError as error:
response = error
with response:
payload = json.loads(response.read().decode("utf-8"))
if not response.headers.get("X-Request-Id"):
raise RuntimeError("API response is missing X-Request-Id")
return response.status, payload
def wait_ready(base_url, timeout=10):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
status, _ = request(base_url, "GET", "/health/ready")
if status == 200:
return
except (OSError, URLError, TimeoutError):
pass
time.sleep(0.2)
raise RuntimeError("API did not become ready within %s seconds" % timeout)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--base-url", default=os.environ.get("BASE_URL", "http://localhost:8080"))
parser.add_argument("--container", default=os.environ.get("CONTAINER_NAME", "mtusi-messenger-lab1"))
args = parser.parse_args()
base_url = args.base_url.rstrip("/")
username = "lifecycle_" + uuid.uuid4().hex[:12]
status, user = request(base_url, "POST", "/api/v1/users", {
"username": username,
"display_name": "Lifecycle demo",
})
if status != 201:
raise RuntimeError("user creation returned HTTP %s" % status)
status, chat = request(base_url, "POST", "/api/v1/chats", {
"title": "Lifecycle demo",
"member_ids": [user["id"]],
}, actor=user["id"])
if status != 201:
raise RuntimeError("chat creation returned HTTP %s" % status)
message_text = "persisted only while this process is alive"
status, message = request(base_url, "POST", "/api/v1/chats/%s/messages" % chat["id"], {
"text": message_text,
}, actor=user["id"])
if status != 201:
raise RuntimeError("message creation returned HTTP %s" % status)
status, page = request(base_url, "GET", "/api/v1/chats/%s/messages" % chat["id"], actor=user["id"])
if status != 200 or not any(item["id"] == message["id"] for item in page["items"]):
raise RuntimeError("created message was not readable before restart")
print("before restart: user_id=%s message_id=%s message_read=yes" % (user["id"], message["id"]))
subprocess.run(["docker", "restart", "--timeout", "5", args.container], check=True, capture_output=True, text=True)
wait_ready(base_url)
status, error = request(base_url, "GET", "/api/v1/users/me", actor=user["id"])
if status != 401 or error.get("error", {}).get("code") != "unauthorized":
raise RuntimeError("old X-User-Id should be rejected after restart; got HTTP %s" % status)
print("after restart: old_user_id_status=%s code=%s state_reset=yes" % (status, error["error"]["code"]))
if __name__ == "__main__":
main()
+132
View File
@@ -0,0 +1,132 @@
import json
import threading
import unittest
import urllib.error
import urllib.request
import app
class ApiTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.server = app.ThreadingHTTPServer(("127.0.0.1", 0), app.Handler)
cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True)
cls.thread.start()
cls.base = "http://127.0.0.1:%d" % cls.server.server_port
@classmethod
def tearDownClass(cls):
cls.server.shutdown()
cls.server.server_close()
def request(self, method, path, body=None, user=None, raw=None):
headers = {"Accept": "application/json"}
if user:
headers["X-User-Id"] = user
data = raw
if body is not None:
headers["Content-Type"] = "application/json"
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
request = urllib.request.Request(self.base + path, data=data, headers=headers, method=method)
try:
response = urllib.request.urlopen(request)
except urllib.error.HTTPError as error:
response = error
with response:
payload = json.loads(response.read().decode("utf-8"))
self.assertTrue(response.headers.get("X-Request-Id"))
return response.code, payload
def test_invalid_json_duplicate_user_and_foreign_chat(self):
status, _ = self.request("POST", "/api/v1/users", body={"username": "alice", "display_name": "Алиса"})
self.assertEqual(status, 201)
alice_id = next(iter(app.state.users))
status, _ = self.request("POST", "/api/v1/users", body={"username": "alice", "display_name": "Другая"})
self.assertEqual(status, 409)
bob = self.request("POST", "/api/v1/users", body={"username": "charlie", "display_name": "Чарли"})[1]
chat = self.request("POST", "/api/v1/chats", user=alice_id, body={"title": "Тест", "member_ids": [alice_id]})[1]
status, _ = self.request("GET", "/api/v1/chats/%s" % chat["id"], user=bob["id"])
self.assertEqual(status, 404)
status, _ = self.request("GET", "/api/v1/chats/not-a-uuid", user=alice_id)
self.assertEqual(status, 400)
status, _ = self.request("POST", "/api/v1/users", body={"username": "bad", "display_name": "x", "extra": 1})
self.assertEqual(status, 400)
status, _ = self.request("POST", "/api/v1/users", raw=b"{broken")
self.assertEqual(status, 400)
def test_invalid_types_and_chat_pagination(self):
status, _ = self.request("POST", "/api/v1/users", body={"username": 1, "display_name": "Тест"})
self.assertEqual(status, 400)
user = self.request("POST", "/api/v1/users", body={"username": "typecheck", "display_name": "Типы"})[1]
for member_ids in ([{"id": user["id"]}], [user["id"], []]):
status, _ = self.request(
"POST",
"/api/v1/chats",
user=user["id"],
body={"title": "Некорректный чат", "member_ids": member_ids},
)
self.assertEqual(status, 400)
for index in range(5):
status, _ = self.request(
"POST",
"/api/v1/chats",
user=user["id"],
body={"title": "Чат %d" % index, "member_ids": [user["id"]]},
)
self.assertEqual(status, 201)
cursor = None
chat_ids = []
for _ in range(5):
path = "/api/v1/chats?limit=1"
if cursor:
path += "&cursor=" + cursor
status, page = self.request("GET", path, user=user["id"])
self.assertEqual(status, 200)
self.assertEqual(len(page["items"]), 1)
chat_ids.append(page["items"][0]["id"])
cursor = page["next_cursor"]
self.assertIsNone(cursor)
self.assertEqual(len(set(chat_ids)), 5)
def test_empty_message_and_multi_page_pagination(self):
user = self.request("POST", "/api/v1/users", body={"username": "pager", "display_name": "Пейджер"})[1]
chat = self.request("POST", "/api/v1/chats", user=user["id"], body={"title": "Страницы", "member_ids": [user["id"]]})[1]
status, _ = self.request("POST", "/api/v1/chats/%s/messages" % chat["id"], user=user["id"], body={"text": " "})
self.assertEqual(status, 400)
for index in range(3):
self.assertEqual(self.request("POST", "/api/v1/chats/%s/messages" % chat["id"], user=user["id"], body={"text": str(index)})[0], 201)
status, first = self.request("GET", "/api/v1/chats/%s/messages?limit=2" % chat["id"], user=user["id"])
self.assertEqual(status, 200)
self.assertEqual(len(first["items"]), 2)
self.assertTrue(first["next_cursor"])
status, second = self.request("GET", "/api/v1/chats/%s/messages?limit=2&cursor=%s" % (chat["id"], first["next_cursor"]), user=user["id"])
self.assertEqual(status, 200)
self.assertEqual(len(second["items"]), 1)
self.assertIsNone(second["next_cursor"])
def test_concurrent_messages_are_not_lost(self):
users = [self.request("POST", "/api/v1/users", body={"username": "client%d" % index, "display_name": "Клиент %d" % index})[1] for index in range(10)]
chat = self.request("POST", "/api/v1/chats", user=users[0]["id"], body={"title": "Тест", "member_ids": [user["id"] for user in users]})[1]
barrier = threading.Barrier(10)
def send(client_index):
barrier.wait()
for message_index in range(10):
self.request("POST", "/api/v1/chats/%s/messages" % chat["id"], user=users[client_index]["id"], body={"text": "message-%d-%d" % (client_index, message_index)})
threads = [threading.Thread(target=send, args=(index,)) for index in range(10)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
status, page = self.request("GET", "/api/v1/chats/%s/messages?limit=100" % chat["id"], user=users[0]["id"])
self.assertEqual(status, 200)
self.assertEqual(len(page["items"]), 100)
self.assertEqual(len({message["id"] for message in page["items"]}), 100)
if __name__ == "__main__":
unittest.main()