ЛР2 - БВТ2303 - Самсонов - 4 #25

Closed
Samsonov_BVT2303 wants to merge 2 commits from Samsonov_BVT2303/mtusi2026lab2:master into master
15 changed files with 1578 additions and 26 deletions
+7 -3
View File
@@ -4,8 +4,12 @@ APP_HOST=0.0.0.0
APP_PORT=8080
APP_ENV=lab
SESSION_TTL_SECONDS=3600
# Choose credentials locally. File paths contain no secret values.
# Sessions of this stand live in Valkey, not in PostgreSQL.
SESSION_STORE=valkey
POSTGRES_USER=messenger
POSTGRES_PASSWORD=local-only-change-me
POSTGRES_DB=messenger
# Optional file paths. The API reads a file when it exists, otherwise DATABASE_URL / REDIS_URL.
# Compose injects those URLs itself; do not put the database password into Git beyond this placeholder.
DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
+29
View File
@@ -0,0 +1,29 @@
# syntax=docker/dockerfile:1.7
FROM python:3.12.6-slim-bookworm
RUN groupadd --system --gid 10001 app \
&& useradd --system --uid 10001 --gid app --no-create-home app
WORKDIR /app
COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt
COPY --chown=app:app src/ /app/
COPY --chown=app:app migrations/ /app/migrations/
USER app
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
APP_HOST=0.0.0.0 \
APP_PORT=8080 \
APP_ENV=lab
EXPOSE 8080
HEALTHCHECK --interval=10s --timeout=3s --start-period=10s --retries=5 \
CMD python3 -c "import urllib.request,sys; \
urllib.request.urlopen('http://127.0.0.1:8080/health/live',timeout=2); \
sys.exit(0)" || exit 1
CMD ["python3", "/app/server.py"]
+4 -1
View File
@@ -4,7 +4,7 @@ CA_FILE ?=
ACTION_SCRIPT ?=
TLS_ARGS = $(if $(CA_FILE),--ca-file "$(CA_FILE)",)
.PHONY: help check validate test load test-persistence
.PHONY: help check validate test load test-persistence test-student
help:
@echo "check: offline structure; validate: full OpenAPI (requirements-dev.txt); test: running API; load: synthetic writes"
@@ -24,3 +24,6 @@ load:
test-persistence:
@test -n "$(ACTION_SCRIPT)" || (echo "Set ACTION_SCRIPT to your implemented executable restart hook"; exit 2)
$(PYTHON) tests/resilience.py --mode persistence --action-script "$(ACTION_SCRIPT)" --base-url "$(BASE_URL)" $(TLS_ARGS)
test-student:
$(PYTHON) tests/student_level4.py --base-url "$(BASE_URL)"
+123 -22
View File
@@ -1,37 +1,138 @@
# Отчёт — лабораторная 2
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Автор, группа: Самсонов Александр, БВТ2303
- Целевая оценка: 4
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Использованные источники, библиотеки и помощь ИИ: PostgreSQL transactions, PostgreSQL EXPLAIN, Valkey persistence, RFC 7617, hashlib.scrypt, psycopg 3, redis-py. Помощь ИИ в реализации сервера, миграции, Compose и тестов.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Нужны Docker Desktop и Python 3.9+ (для публичных тестов; приложение в контейнере — Python 3.12.6).
```powershell
git clone <url-этого-форка>
cd lb2
Copy-Item .env.example .env
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build
docker compose --env-file .env ps
```
Готовность: `GET http://localhost:8080/health/ready` возвращает `{"status":"ok"}`. Миграции применяет сам API при старте, повторный запуск не вставляет данные второй раз: версия записана в `schema_migrations`.
Проверки:
```powershell
python tests/smoke.py --base-url http://localhost:8080
python tests/student_level4.py --base-url http://localhost:8080 --deps
docker compose --env-file .env exec -T api python3 /app/explain_history.py
```
Сохранность после рестарта API, PostgreSQL и Valkey (volumes не удаляются):
```powershell
# Git Bash или WSL, у scripts/restart.sh должен быть executable bit
chmod +x scripts/restart.sh
python tests/resilience.py --mode persistence --action-script scripts/restart.sh --base-url http://localhost:8080
```
Короткий TTL сессии:
```powershell
# в .env: SESSION_TTL_SECONDS=2
docker compose --env-file .env up -d --force-recreate api
python tests/student_level4.py --base-url http://localhost:8080 --expiry
# вернуть SESSION_TTL_SECONDS=3600 и снова up -d --force-recreate api
```
Остановка без удаления данных:
```powershell
docker compose --env-file .env stop
```
Удаление учебных данных вместе с volumes:
```powershell
docker compose --env-file .env down -v
```
Образы закреплены тегами: `python:3.12.6-slim-bookworm`, `postgres:16.6-bookworm`, `valkey/valkey:8.0.2`. Зависимости Python закреплены в `requirements.txt`.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab1, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
```
Клиент --HTTP--> API (src/server.py) --SQL--> PostgreSQL
|
+--сессия--> Valkey (AOF + volume)
```
Из lab1 перенесены пользователи, чаты, членство, сообщения, keyset-пагинация, формат ошибок и `X-Request-Id`. `X-User-Id` больше не выбирает пользователя. Регистрация требует `password`. Вход — только `POST /api/v1/auth/sessions` с `Authorization: Basic`. Остальные защищённые запросы требуют `Authorization: Bearer`. Basic на бизнес-запросе даёт 401.
### Источник истины
PostgreSQL хранит пользователей, чаты, участников и сообщения. Пароль хранится как scrypt (`hashlib.scrypt`, параметры `N=16384,r=8,p=1`), не как SHA-256 и не как plaintext. bcrypt не используется: он обрезает пароль на 72 байтах и не покрывает весь диапазон UTF-8 до 128 code points.
Сессии хранит только Valkey. Ключ — `session:` + SHA-256 токена, значение — id пользователя, TTL равен `SESSION_TTL_SECONDS` (по умолчанию 3600). Токен — `secrets.token_urlsafe(32)`, 256 бит, клиенту отдаётся один раз. Сам токен в Redis и в логи не пишется. TTL не продлевается при рестарте: Redis восстанавливает ключ вместе с оставшимся сроком из AOF. Один TTL без диска сессию после рестарта не сохранил бы, поэтому у Valkey включены `--appendonly yes --appendfsync everysec` и volume `valkeydata`.
Инвалидация: logout удаляет ключ. Истёкший TTL удаляет ключ сам. Повтор с отозванным токеном — 401.
### Транзакционные границы
- Регистрация: один `INSERT`. Два параллельных запроса с одним username упираются в `UNIQUE`; второй получает 409, а не 500.
- Создание чата: одна транзакция. Сначала вставляется чат, затем проверяются участники и вставляются `chat_members`. Неизвестный участник поднимает ошибку, транзакция откатывается, чат в списке не появляется.
- Сообщение: один `INSERT ... WHERE EXISTS` участника. Чужой чат не создаёт строку и отвечает 404.
- Сессия не входит в транзакцию PostgreSQL.
Пул: `psycopg_pool`, максимум 10 соединений, таймаут выдачи 5 с, `connect_timeout` 3 с, `statement_timeout` 5 с. Все запросы параметризованы (`%s`), значения в SQL не подставляются.
История читается так:
```sql
SELECT ...
FROM messages
WHERE chat_id = $1
AND (created_at, id) > ($2, $3)
ORDER BY created_at, id
LIMIT $4
```
Индекс `messages_history_idx (chat_id, created_at, id)` позволяет открыть страницу диапазоном, а не сканом всей таблицы. Это проверяет `src/explain_history.py` на 10000 сообщениях командой `EXPLAIN (ANALYZE, BUFFERS)`.
### Готовность и отказ
`/health/live` проверяет только процесс. `/health/ready` проверяет PostgreSQL и Valkey: без любого из них ответ 503 `unavailable`. Запрос без токена при недоступном Valkey остаётся 401, а не успешным анонимным доступом. Запрос с токеном при недоступном Valkey или PostgreSQL — 503, без текста SQL и без токена в ответе.
### Соответствие `.env.example`
| Переменная | Где используется |
|---|---|
| `APP_HOST`, `APP_PORT`, `APP_ENV` | bind и строка старта API |
| `SESSION_TTL_SECONDS` | TTL новой сессии в Valkey и `expires_at` |
| `SESSION_STORE=valkey` | сессии не пишутся в PostgreSQL |
| `POSTGRES_*` | контейнер PostgreSQL и сборка `DATABASE_URL` в Compose |
| `DATABASE_URL` | API, если нет файла `DATABASE_URL_FILE` |
| `REDIS_URL` | API, если нет файла `CACHE_URL_FILE` |
`.env` в Git не входит. Volumes `pgdata` и `valkeydata` тоже.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Критерий | Команда | Что смотреть |
|---|---|---|
| Публичный smoke | `python tests/smoke.py --base-url http://localhost:8080` | строка `PASS: lab2 public API scenario` |
| Рестарт с тем же токеном | `python tests/resilience.py --mode persistence --action-script scripts/restart.sh --base-url http://localhost:8080` | `PASS: persistence preserved confirmed data and existing session` |
| Гонка username, rollback чата, logout, неверный пароль, битый cursor | `python tests/student_level4.py --base-url http://localhost:8080` | `ALL STUDENT TESTS PASSED` |
| Недоступность Valkey и PostgreSQL | `python tests/student_level4.py --base-url http://localhost:8080 --deps` | `PASS dependencies`: ready/me дают 503, запрос без токена даёт 401, после старта прежний токен снова 200 |
| Expiry при коротком TTL | `SESSION_TTL_SECONDS=2`, recreate api, `python tests/student_level4.py --expiry` | `PASS expiry` |
| План истории | `docker compose --env-file .env exec -T api python3 /app/explain_history.py` | в плане Index Scan по `messages_history_idx`, без Seq Scan |
| Повтор миграций | второй `docker compose up` | в `schema_migrations` одна строка `001_init.sql`, пользователи не задвоены |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
На этой машине Docker в PATH не найден, поэтому smoke, persistence и EXPLAIN здесь не запускались. Команды выше — то, чем закрываются критерии на стенде.
## Эксперименты и отказы
## Ограничения
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
- Рестарт выдерживают данные PostgreSQL и ещё не истёкшие сессии Valkey, пока volumes не удалены. `down -v` стирает и сообщения, и сессии.
- Потеря Valkey не пускает анонима в API, но и не даёт войти, пока сессии некому хранить: это 503, не обход в PostgreSQL. Отдельной копии сессий в PostgreSQL нет.
- `appendfsync everysec` может потерять последнюю секунду записи при жёстком падении процесса Valkey. Обычный `docker compose restart` делает штатный shutdown, и AOF сбрасывается.
- Нагрузка больше лабораторного smoke не измерялась.
+63
View File
@@ -0,0 +1,63 @@
# Локальный стенд lab2: API, PostgreSQL, Valkey.
# Сессии живут в Valkey. AOF и volume сохраняют их при restart без удаления данных.
services:
api:
build: .
image: mtusi-messenger:lab2
ports:
- "127.0.0.1:8080:8080"
environment:
APP_HOST: 0.0.0.0
APP_PORT: "8080"
APP_ENV: ${APP_ENV:-lab}
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
SESSION_STORE: valkey
DATABASE_URL: postgresql://${POSTGRES_USER:-messenger}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-messenger}
REDIS_URL: redis://valkey:6379/0
depends_on:
postgres:
condition: service_healthy
valkey:
condition: service_healthy
healthcheck:
test:
- CMD
- python3
- -c
- import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/ready', timeout=2)
interval: 5s
timeout: 3s
retries: 12
start_period: 10s
restart: unless-stopped
postgres:
image: postgres:16.6-bookworm
environment:
POSTGRES_USER: ${POSTGRES_USER:-messenger}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB:-messenger}
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-messenger} -d ${POSTGRES_DB:-messenger}"]
interval: 5s
timeout: 3s
retries: 12
restart: unless-stopped
valkey:
image: valkey/valkey:8.0.2
command: ["valkey-server", "--appendonly", "yes", "--appendfsync", "everysec"]
volumes:
- valkeydata:/data
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 5s
timeout: 3s
retries: 12
restart: unless-stopped
volumes:
pgdata:
valkeydata:
+49
View File
@@ -0,0 +1,49 @@
-- Источник истины lab2: пользователи, чаты, участники, сообщения.
-- Сессии здесь не хранятся: их источник истины — Valkey (AOF + volume).
--
-- Транзакции приложения:
-- регистрация — один INSERT, уникальность username держит UNIQUE
-- создание чата — одна транзакция: чат и все chat_members, иначе откат
-- сообщение — один INSERT, только если отправитель уже участник
--
-- Индекс истории messages_history_idx покрывает keyset-запрос
-- WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
-- ORDER BY created_at, id
-- LIMIT $4
-- Страница читается диапазоном индекса, а не сканом всей таблицы.
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL,
display_name text NOT NULL,
password_hash text NOT NULL,
created_at timestamptz NOT NULL,
CONSTRAINT users_username_unique UNIQUE (username)
);
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL,
created_at timestamptz NOT NULL
);
CREATE INDEX chats_created_idx ON chats (created_at, id);
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats (id),
user_id uuid NOT NULL REFERENCES users (id),
position integer NOT NULL,
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX chat_members_user_idx ON chat_members (user_id, chat_id);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats (id),
sender_id uuid NOT NULL REFERENCES users (id),
text text NOT NULL,
created_at timestamptz NOT NULL
);
CREATE INDEX messages_history_idx ON messages (chat_id, created_at, id);
+3
View File
@@ -0,0 +1,3 @@
psycopg[binary]==3.2.3
psycopg-pool==3.2.3
redis==5.1.1
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
docker compose --env-file .env restart api postgres valkey
+365
View File
@@ -0,0 +1,365 @@
"""PostgreSQL: пользователи, чаты, участники, сообщения. Запросы только с %s."""
from __future__ import annotations
import os
import time
import uuid
from contextlib import contextmanager
from datetime import datetime, timezone
from pathlib import Path
import psycopg
from psycopg.errors import UniqueViolation
from psycopg_pool import ConnectionPool, PoolTimeout
from errors import ApiError
_pool: ConnectionPool | None = None
def migrations_dir() -> Path:
here = Path(__file__).resolve().parent
for candidate in (here / "migrations", here.parent / "migrations"):
if candidate.is_dir():
return candidate
return here.parent / "migrations"
def database_url() -> str:
path = os.environ.get("DATABASE_URL_FILE", "").strip()
if path and Path(path).is_file():
return Path(path).read_text(encoding="utf-8").strip()
return os.environ.get("DATABASE_URL", "").strip()
def init() -> None:
global _pool
url = database_url()
if not url:
raise RuntimeError("database is not configured")
_pool = ConnectionPool(
url,
min_size=0,
max_size=10,
timeout=5,
open=True,
kwargs={
"connect_timeout": 3,
"autocommit": False,
"options": "-c statement_timeout=5000",
},
)
def migrate_with_retry() -> None:
last = None
for _ in range(30):
try:
migrate()
return
except Exception as exc:
last = exc
time.sleep(1)
raise RuntimeError("migrations failed") from last
def migrate() -> None:
url = database_url()
if not url:
raise RuntimeError("database is not configured")
with psycopg.connect(url, connect_timeout=3, autocommit=False) as conn:
conn.execute("SELECT pg_advisory_lock(%s)", (842021,))
conn.execute(
"""
CREATE TABLE IF NOT EXISTS schema_migrations (
version text PRIMARY KEY,
applied_at timestamptz NOT NULL DEFAULT now()
)
"""
)
conn.commit()
applied = {
row[0]
for row in conn.execute("SELECT version FROM schema_migrations").fetchall()
}
for path in sorted(migrations_dir().glob("*.sql")):
if path.name in applied:
continue
sql = path.read_text(encoding="utf-8")
with conn.transaction():
for statement in sql.split(";"):
if statement.strip():
conn.execute(statement)
conn.execute(
"INSERT INTO schema_migrations (version) VALUES (%s)",
(path.name,),
)
@contextmanager
def connection():
if _pool is None:
raise ApiError(503, "unavailable", "База данных недоступна")
try:
with _pool.connection() as conn:
yield conn
except ApiError:
raise
except (PoolTimeout, psycopg.OperationalError, psycopg.InterfaceError) as exc:
raise ApiError(503, "unavailable", "База данных недоступна") from exc
def ping() -> bool:
try:
with connection() as conn:
conn.execute("SELECT 1")
return True
except ApiError:
return False
def close() -> None:
if _pool is not None:
_pool.close()
def user_json(row) -> dict:
return {
"id": str(row[0]),
"username": row[1],
"display_name": row[2],
"created_at": format_ts(row[3]),
}
def format_ts(value: datetime) -> str:
value = value.astimezone(timezone.utc)
return value.strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z"
def create_user(username: str, display_name: str, password_hash: str, created: datetime) -> dict:
uid = uuid.uuid4()
try:
with connection() as conn:
with conn.transaction():
row = conn.execute(
"""
INSERT INTO users (id, username, display_name, password_hash, created_at)
VALUES (%s, %s, %s, %s, %s)
RETURNING id, username, display_name, created_at
""",
(uid, username, display_name, password_hash, created),
).fetchone()
except UniqueViolation as exc:
raise ApiError(409, "conflict", "username занят") from exc
return user_json(row)
def user_for_login(username: str):
with connection() as conn:
return conn.execute(
"""
SELECT id, username, display_name, password_hash, created_at
FROM users
WHERE username = %s
""",
(username,),
).fetchone()
def get_user(user_id: str) -> dict | None:
with connection() as conn:
row = conn.execute(
"""
SELECT id, username, display_name, created_at
FROM users
WHERE id = %s
""",
(uuid.UUID(user_id),),
).fetchone()
if row is None:
return None
return user_json(row)
def create_chat(title: str, member_ids: list[str], created: datetime) -> dict:
cid = uuid.uuid4()
members = [uuid.UUID(item) for item in member_ids]
with connection() as conn:
with conn.transaction():
conn.execute(
"INSERT INTO chats (id, title, created_at) VALUES (%s, %s, %s)",
(cid, title, created),
)
found = conn.execute(
"SELECT id FROM users WHERE id = ANY(%s::uuid[])",
(members,),
).fetchall()
if {row[0] for row in found} != set(members):
raise ApiError(400, "invalid_request", "неизвестный участник")
for position, member in enumerate(members):
conn.execute(
"""
INSERT INTO chat_members (chat_id, user_id, position)
VALUES (%s, %s, %s)
""",
(cid, member, position),
)
return {
"id": str(cid),
"title": title,
"member_ids": list(member_ids),
"created_at": format_ts(created),
}
def _chat_row(row) -> dict:
return {
"id": str(row[0]),
"title": row[1],
"created_at": format_ts(row[2]),
"member_ids": [str(item) for item in row[3]],
}
def get_chat(chat_id: str, user_id: str) -> dict:
with connection() as conn:
row = conn.execute(
"""
SELECT c.id, c.title, c.created_at,
array_agg(cm.user_id ORDER BY cm.position)
FROM chats c
JOIN chat_members mine
ON mine.chat_id = c.id AND mine.user_id = %s
JOIN chat_members cm ON cm.chat_id = c.id
WHERE c.id = %s
GROUP BY c.id
""",
(uuid.UUID(user_id), uuid.UUID(chat_id)),
).fetchone()
if row is None:
raise ApiError(404, "not_found", "Чат не найден")
chat = _chat_row(row)
return {
"id": chat["id"],
"title": chat["title"],
"member_ids": chat["member_ids"],
"created_at": chat["created_at"],
}
def list_chats(user_id: str, limit: int, cursor) -> dict:
params: list = [uuid.UUID(user_id)]
where = ""
if cursor is not None:
created, chat_id = cursor
where = "WHERE (c.created_at, c.id) > (%s, %s)"
params.extend([created, uuid.UUID(chat_id)])
params.append(limit + 1)
sql = f"""
SELECT c.id, c.title, c.created_at,
array_agg(cm.user_id ORDER BY cm.position)
FROM chats c
JOIN chat_members mine
ON mine.chat_id = c.id AND mine.user_id = %s
JOIN chat_members cm ON cm.chat_id = c.id
{where}
GROUP BY c.id
ORDER BY c.created_at, c.id
LIMIT %s
"""
with connection() as conn:
rows = conn.execute(sql, params).fetchall()
page = rows[:limit]
next_cursor = None
if len(rows) > limit and page:
last = page[-1]
next_cursor = (last[2], str(last[0]))
return {
"items": [
{
"id": str(row[0]),
"title": row[1],
"member_ids": [str(item) for item in row[3]],
"created_at": format_ts(row[2]),
}
for row in page
],
"next_cursor_parts": next_cursor,
}
def create_message(chat_id: str, sender_id: str, text: str, created: datetime) -> dict:
mid = uuid.uuid4()
with connection() as conn:
with conn.transaction():
row = conn.execute(
"""
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
SELECT %s, %s, %s, %s, %s
WHERE EXISTS (
SELECT 1 FROM chat_members
WHERE chat_id = %s AND user_id = %s
)
RETURNING id, chat_id, sender_id, text, created_at
""",
(
mid, uuid.UUID(chat_id), uuid.UUID(sender_id), text, created,
uuid.UUID(chat_id), uuid.UUID(sender_id),
),
).fetchone()
if row is None:
raise ApiError(404, "not_found", "Чат не найден")
return message_json(row)
def list_messages(chat_id: str, user_id: str, limit: int, cursor) -> dict:
params: list = [uuid.UUID(chat_id), uuid.UUID(chat_id), uuid.UUID(user_id)]
cursor_sql = ""
if cursor is not None:
created, message_id = cursor
cursor_sql = "AND (m.created_at, m.id) > (%s, %s)"
params.extend([created, uuid.UUID(message_id)])
params.append(limit + 1)
sql = f"""
SELECT m.id, m.chat_id, m.sender_id, m.text, m.created_at
FROM messages m
WHERE m.chat_id = %s
AND EXISTS (
SELECT 1 FROM chat_members
WHERE chat_id = %s AND user_id = %s
)
{cursor_sql}
ORDER BY m.created_at, m.id
LIMIT %s
"""
with connection() as conn:
rows = conn.execute(sql, params).fetchall()
if not rows:
with connection() as conn:
member = conn.execute(
"""
SELECT 1 FROM chat_members
WHERE chat_id = %s AND user_id = %s
""",
(uuid.UUID(chat_id), uuid.UUID(user_id)),
).fetchone()
if member is None:
raise ApiError(404, "not_found", "Чат не найден")
page = rows[:limit]
next_cursor = None
if len(rows) > limit and page:
last = page[-1]
next_cursor = (last[4], str(last[0]))
return {
"items": [message_json(row) for row in page],
"next_cursor_parts": next_cursor,
}
def message_json(row) -> dict:
return {
"id": str(row[0]),
"chat_id": str(row[1]),
"sender_id": str(row[2]),
"text": row[3],
"created_at": format_ts(row[4]),
}
+8
View File
@@ -0,0 +1,8 @@
"""Ошибки API. Текст не содержит SQL, паролей и токенов."""
class ApiError(Exception):
def __init__(self, status: int, code: str, message: str):
self.status = status
self.code = code
self.message = message
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env python3
"""EXPLAIN (ANALYZE, BUFFERS) keyset-страницы на >= 10000 сообщениях одного чата."""
from __future__ import annotations
import os
import sys
import uuid
from datetime import datetime, timedelta, timezone
import psycopg
START = datetime(2020, 1, 1, tzinfo=timezone.utc)
def main() -> None:
url = os.environ.get("DATABASE_URL", "").strip()
if not url:
sys.stderr.write("DATABASE_URL is not set\n")
raise SystemExit(2)
with psycopg.connect(url, connect_timeout=5, autocommit=False) as conn:
user_id = uuid.uuid5(uuid.NAMESPACE_URL, "lab2-explain-user")
chat_id = uuid.uuid5(uuid.NAMESPACE_URL, "lab2-explain-chat")
with conn.transaction():
conn.execute(
"""
INSERT INTO users (id, username, display_name, password_hash, created_at)
VALUES (%s, %s, %s, %s, %s)
ON CONFLICT (username) DO NOTHING
""",
(user_id, "explain_hist", "explain", "not-a-login-hash", START),
)
conn.execute(
"""
INSERT INTO chats (id, title, created_at)
VALUES (%s, %s, %s)
ON CONFLICT (id) DO NOTHING
""",
(chat_id, "explain", START),
)
conn.execute(
"""
INSERT INTO chat_members (chat_id, user_id, position)
VALUES (%s, %s, 0)
ON CONFLICT (chat_id, user_id) DO NOTHING
""",
(chat_id, user_id),
)
count = conn.execute(
"SELECT count(*) FROM messages WHERE chat_id = %s",
(chat_id,),
).fetchone()[0]
missing = 10000 - count
if missing > 0:
conn.execute(
"""
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
SELECT gen_random_uuid(), %s, %s, 'm', %s + (g || ' seconds')::interval
FROM generate_series(%s::bigint, %s::bigint) AS g
""",
(chat_id, user_id, START, count + 1, 10000),
)
midpoint = START + timedelta(seconds=5000)
plan = conn.execute(
"""
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, created_at
FROM messages
WHERE chat_id = %s
AND (created_at, id) > (%s, %s)
ORDER BY created_at, id
LIMIT 50
""",
(chat_id, midpoint, uuid.UUID(int=0)),
).fetchall()
text = "\n".join(row[0] for row in plan)
sys.stdout.write(text + "\n")
if "Index" not in text or "Seq Scan" in text:
sys.stderr.write("expected an index range scan, not a sequential scan\n")
raise SystemExit(1)
if __name__ == "__main__":
main()
+43
View File
@@ -0,0 +1,43 @@
"""Пароли: scrypt из hashlib. Не bcrypt — он режет пароль на 72 байтах."""
from __future__ import annotations
import base64
import hashlib
import hmac
import secrets
N = 2**14
R = 8
P = 1
DKLEN = 32
def hash_password(password: str) -> str:
salt = secrets.token_bytes(16)
digest = hashlib.scrypt(
password.encode("utf-8"), salt=salt, n=N, r=R, p=P, dklen=DKLEN,
maxmem=64 * 1024 * 1024)
return "scrypt$%d$%d$%d$%s$%s" % (
N, R, P,
base64.b64encode(salt).decode("ascii"),
base64.b64encode(digest).decode("ascii"),
)
def verify_password(password: str, stored: str) -> bool:
try:
algo, n, r, p, salt_b64, digest_b64 = stored.split("$")
if algo != "scrypt":
return False
salt = base64.b64decode(salt_b64)
expected = base64.b64decode(digest_b64)
actual = hashlib.scrypt(
password.encode("utf-8"), salt=salt, n=int(n), r=int(r), p=int(p),
dklen=len(expected), maxmem=64 * 1024 * 1024)
except (ValueError, TypeError):
return False
return hmac.compare_digest(actual, expected)
# Одинаковая стоимость проверки, когда пользователя нет.
DUMMY_HASH = hash_password("dummy-password-not-used")
+473
View File
@@ -0,0 +1,473 @@
#!/usr/bin/env python3
"""Messenger lab2. PostgreSQL хранит данные, Valkey хранит сессии."""
from __future__ import annotations
import base64
import json
import os
import re
import secrets
import signal
import socketserver
import sys
import threading
import uuid
from datetime import datetime, timedelta, timezone
from http.server import BaseHTTPRequestHandler
from urllib.parse import parse_qs, urlparse
import db
import passwords
import sessions
from errors import ApiError
APP_HOST = os.environ.get("APP_HOST", "0.0.0.0")
APP_PORT = int(os.environ.get("APP_PORT", "8080"))
APP_ENV = os.environ.get("APP_ENV", "lab")
USERNAME_RE = re.compile(r"^[a-z0-9_]{3,32}\Z")
UUID_RE = re.compile(
r"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-"
r"[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\Z")
TIMESTAMP_RE = re.compile(
r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{1,6})?Z\Z")
MAX_BODY = 16 * 1024 * 1024
def now_utc() -> datetime:
current = datetime.now(timezone.utc)
return current.replace(microsecond=(current.microsecond // 1000) * 1000)
def is_blank(value: str) -> bool:
return value.strip() == ""
def require_object(body):
if not isinstance(body, dict):
raise ApiError(400, "invalid_request", "Тело должно быть JSON-объектом")
def reject_extra(body: dict, allowed: set):
extra = set(body) - allowed
if extra:
raise ApiError(400, "invalid_request",
"Лишние поля: " + ", ".join(sorted(extra)))
def parse_uuid(value, field: str) -> str:
if not isinstance(value, str) or not UUID_RE.match(value):
raise ApiError(400, "invalid_request", f"Некорректный UUID: {field}")
return value.lower()
def parse_display_name(value) -> str:
if not isinstance(value, str):
raise ApiError(400, "invalid_request", "display_name должен быть строкой")
if not (1 <= len(value) <= 100):
raise ApiError(400, "invalid_request", "display_name: 1..100 code points")
if is_blank(value):
raise ApiError(400, "invalid_request", "display_name не может быть пустым")
return value
def parse_username(value) -> str:
if not isinstance(value, str) or not USERNAME_RE.match(value):
raise ApiError(400, "invalid_request", "username: ^[a-z0-9_]{3,32}$")
return value
def parse_password(value) -> str:
if not isinstance(value, str):
raise ApiError(400, "invalid_request", "password должен быть строкой")
if not (12 <= len(value) <= 128):
raise ApiError(400, "invalid_request", "password: 12..128 code points")
if is_blank(value):
raise ApiError(400, "invalid_request", "password не может быть пустым")
return value
def parse_title(value) -> str:
if not isinstance(value, str):
raise ApiError(400, "invalid_request", "title должен быть строкой")
if not (1 <= len(value) <= 100):
raise ApiError(400, "invalid_request", "title: 1..100 code points")
if is_blank(value):
raise ApiError(400, "invalid_request", "title не может быть пустым")
return value
def parse_text(value) -> str:
if not isinstance(value, str):
raise ApiError(400, "invalid_request", "text должен быть строкой")
if not (1 <= len(value) <= 4000):
raise ApiError(400, "invalid_request", "text: 1..4000 code points")
if is_blank(value):
raise ApiError(400, "invalid_request", "text не может быть пустым")
return value
def parse_member_ids(value) -> list:
if not isinstance(value, list):
raise ApiError(400, "invalid_request", "member_ids должен быть массивом")
if not (1 <= len(value) <= 100):
raise ApiError(400, "invalid_request", "member_ids: 1..100")
seen = set()
result = []
for item in value:
uid = parse_uuid(item, "member_ids[]")
if uid in seen:
raise ApiError(400, "invalid_request", "member_ids содержит дубликаты")
seen.add(uid)
result.append(uid)
return result
def parse_limit(raw, default=50) -> int:
if raw is None:
return default
if isinstance(raw, str) and raw.strip() == "":
raise ApiError(400, "invalid_request", "limit должен быть целым")
try:
value = int(raw)
except (TypeError, ValueError):
raise ApiError(400, "invalid_request", "limit должен быть целым")
if not (1 <= value <= 100):
raise ApiError(400, "invalid_request", "limit: 1..100")
return value
def parse_optional_cursor(raw):
if raw is None:
return None
if not isinstance(raw, str) or raw.strip() == "":
raise ApiError(400, "invalid_request", "Некорректный cursor")
return raw
def parse_cursor_timestamp(value: str) -> datetime:
if not isinstance(value, str) or not TIMESTAMP_RE.match(value):
raise ApiError(400, "invalid_request", "Некорректный cursor")
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise ApiError(400, "invalid_request", "Некорректный cursor") from exc
return parsed
def challenge_for(path: str) -> str:
if path == "/api/v1/auth/sessions":
return 'Basic realm="messenger", charset="UTF-8"'
return 'Bearer realm="messenger"'
def bearer_token(headers) -> str:
auth = headers.get("Authorization") or ""
parts = auth.split(None, 1)
if len(parts) != 2 or parts[0].lower() != "bearer" or not parts[1].strip():
raise ApiError(401, "unauthorized", "Требуется Bearer")
return parts[1].strip()
def current_user(headers) -> dict:
token = bearer_token(headers)
user = db.get_user(sessions.user_id(token))
if user is None:
raise ApiError(401, "unauthorized", "Сессия недействительна")
return user
def basic_credentials(headers) -> tuple[str, str]:
auth = headers.get("Authorization") or ""
parts = auth.split(None, 1)
if len(parts) != 2 or parts[0].lower() != "basic" or not parts[1].strip():
raise ApiError(401, "unauthorized", "Требуется Basic")
try:
decoded = base64.b64decode(parts[1].strip(), validate=True).decode("utf-8")
except (ValueError, UnicodeDecodeError) as exc:
raise ApiError(401, "unauthorized", "Неверные учётные данные") from exc
if ":" not in decoded:
raise ApiError(401, "unauthorized", "Неверные учётные данные")
username, password = decoded.split(":", 1)
return username, password
def encode_cursor(chat_id: str, created_at: str, msg_id: str) -> str:
raw = f"{chat_id}|{created_at}|{msg_id}".encode("utf-8")
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
def decode_cursor(cursor: str, chat_id: str):
pad = "=" * (-len(cursor) % 4)
try:
raw = base64.urlsafe_b64decode(cursor + pad).decode("utf-8")
except Exception as exc:
raise ApiError(400, "invalid_request", "Некорректный cursor") from exc
parts = raw.split("|")
if len(parts) != 3:
raise ApiError(400, "invalid_request", "Некорректный cursor")
c_chat, c_created, c_id = parts
if not UUID_RE.match(c_chat):
raise ApiError(400, "invalid_request", "Некорректный cursor")
if c_chat.lower() != chat_id:
raise ApiError(400, "invalid_request", "cursor от другого чата")
return parse_cursor_timestamp(c_created), parse_uuid(c_id, "cursor")
def encode_chat_cursor(created_at: str, chat_id: str) -> str:
raw = f"chat|{created_at}|{chat_id}".encode("utf-8")
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
def decode_chat_cursor(cursor: str):
pad = "=" * (-len(cursor) % 4)
try:
raw = base64.urlsafe_b64decode(cursor + pad).decode("utf-8")
except Exception as exc:
raise ApiError(400, "invalid_request", "Некорректный cursor") from exc
parts = raw.split("|")
if len(parts) != 3 or parts[0] != "chat":
raise ApiError(400, "invalid_request", "Некорректный cursor")
return parse_cursor_timestamp(parts[1]), parse_uuid(parts[2], "cursor")
def create_user(body) -> dict:
require_object(body)
reject_extra(body, {"username", "display_name", "password"})
username = parse_username(body.get("username"))
display_name = parse_display_name(body.get("display_name"))
password = parse_password(body.get("password"))
return db.create_user(username, display_name, passwords.hash_password(password), now_utc())
def create_session(headers) -> dict:
username, password = basic_credentials(headers)
row = None
try:
row = db.user_for_login(username)
except ApiError:
raise
stored = row[3] if row is not None else passwords.DUMMY_HASH
password_ok = passwords.verify_password(password, stored)
if row is None or not password_ok:
raise ApiError(401, "unauthorized", "Неверные учётные данные")
token = secrets.token_urlsafe(32)
ttl = sessions.session_ttl()
expires = now_utc() + timedelta(seconds=ttl)
sessions.create(token, str(row[0]), ttl)
return {
"token": token,
"token_type": "Bearer",
"expires_at": db.format_ts(expires),
"user": db.user_json((row[0], row[1], row[2], row[4])),
}
def logout(headers) -> None:
token = bearer_token(headers)
sessions.user_id(token)
sessions.revoke(token)
def create_chat(body, actor: dict) -> dict:
require_object(body)
reject_extra(body, {"title", "member_ids"})
title = parse_title(body.get("title"))
member_ids = parse_member_ids(body.get("member_ids"))
if actor["id"] not in member_ids:
raise ApiError(400, "invalid_request", "вызывающий должен быть в member_ids")
return db.create_chat(title, member_ids, now_utc())
def list_chats(actor: dict, limit: int, cursor) -> dict:
decoded = decode_chat_cursor(cursor) if cursor else None
page = db.list_chats(actor["id"], limit, decoded)
next_cursor = None
if page["next_cursor_parts"] is not None:
created, chat_id = page["next_cursor_parts"]
next_cursor = encode_chat_cursor(db.format_ts(created), chat_id)
return {"items": page["items"], "next_cursor": next_cursor}
def create_message(chat_id: str, body, actor: dict) -> dict:
cid = parse_uuid(chat_id, "chat_id")
require_object(body)
reject_extra(body, {"text"})
text = parse_text(body.get("text"))
return db.create_message(cid, actor["id"], text, now_utc())
def list_messages(chat_id: str, actor: dict, limit: int, cursor) -> dict:
cid = parse_uuid(chat_id, "chat_id")
decoded = decode_cursor(cursor, cid) if cursor else None
page = db.list_messages(cid, actor["id"], limit, decoded)
next_cursor = None
if page["next_cursor_parts"] is not None:
created, message_id = page["next_cursor_parts"]
next_cursor = encode_cursor(cid, db.format_ts(created), message_id)
return {"items": page["items"], "next_cursor": next_cursor}
class FastThreadingHTTPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
daemon_threads = True
block_on_close = False
allow_reuse_address = True
request_queue_size = 256
class Handler(BaseHTTPRequestHandler):
server_version = "mtusi-messenger/2.0"
protocol_version = "HTTP/1.0"
def log_message(self, fmt, *args):
return
def _request_id(self) -> str:
return self.headers.get("X-Request-Id") or str(uuid.uuid4())
def _send_json(self, status: int, payload, request_id: str, extra_headers=None):
body = b""
if status != 204:
body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
self.send_response(status)
if status != 204:
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.send_header("X-Request-Id", request_id)
for key, value in (extra_headers or {}).items():
self.send_header(key, value)
self.end_headers()
if body:
self.wfile.write(body)
def _send_error(self, status: int, code: str, message: str, request_id: str, extra_headers=None):
self._send_json(
status,
{"error": {"code": code, "message": message, "request_id": request_id}},
request_id,
extra_headers,
)
def _read_body(self):
length = self.headers.get("Content-Length")
if not length:
return None
try:
size = int(length)
except ValueError as exc:
raise ApiError(400, "invalid_request", "Некорректный Content-Length") from exc
if size > MAX_BODY:
raise ApiError(413, "payload_too_large", "Тело слишком большое")
raw = self.rfile.read(size)
ctype = (self.headers.get("Content-Type") or "").split(";")[0].strip()
if ctype and ctype != "application/json":
raise ApiError(415, "unsupported_media_type", "Ожидается application/json")
if not raw:
raise ApiError(400, "invalid_request", "Пустое тело")
try:
return json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ApiError(400, "invalid_request", "Malformed JSON") from exc
def do_GET(self):
self._dispatch("GET")
def do_POST(self):
self._dispatch("POST")
def do_DELETE(self):
self._dispatch("DELETE")
def _dispatch(self, method: str):
rid = self._request_id()
parsed = urlparse(self.path)
path = parsed.path.rstrip("/") or "/"
query = parse_qs(parsed.query, keep_blank_values=True)
try:
self._route(method, path, query, rid)
except ApiError as exc:
headers = {}
if exc.status == 401:
headers["WWW-Authenticate"] = challenge_for(path)
self._send_error(exc.status, exc.code, exc.message, rid, headers)
except BrokenPipeError:
pass
except Exception:
sys.stderr.write("internal error on %s %s\n" % (method, path))
self._send_error(500, "internal_error", "Внутренняя ошибка", rid)
def _route(self, method: str, path: str, query: dict, rid: str):
if method == "GET" and path == "/health/live":
return self._send_json(200, {"status": "ok"}, rid)
if method == "GET" and path == "/health/ready":
if db.ping() and sessions.ping():
return self._send_json(200, {"status": "ok"}, rid)
raise ApiError(503, "unavailable", "Зависимость недоступна")
if method == "POST" and path == "/api/v1/users":
return self._send_json(201, create_user(self._read_body()), rid)
if method == "POST" and path == "/api/v1/auth/sessions":
return self._send_json(201, create_session(self.headers), rid)
if method == "DELETE" and path == "/api/v1/auth/sessions/current":
logout(self.headers)
return self._send_json(204, None, rid)
if method == "GET" and path == "/api/v1/users/me":
return self._send_json(200, current_user(self.headers), rid)
if method == "POST" and path == "/api/v1/chats":
actor = current_user(self.headers)
return self._send_json(201, create_chat(self._read_body(), actor), rid)
if method == "GET" and path == "/api/v1/chats":
actor = current_user(self.headers)
limit = parse_limit(query.get("limit", [None])[0])
cursor = parse_optional_cursor(query.get("cursor", [None])[0])
return self._send_json(200, list_chats(actor, limit, cursor), rid)
match = re.fullmatch(r"/api/v1/chats/([^/]+)", path)
if match and method == "GET":
actor = current_user(self.headers)
return self._send_json(200, db.get_chat(parse_uuid(match.group(1), "chat_id"), actor["id"]), rid)
match = re.fullmatch(r"/api/v1/chats/([^/]+)/messages", path)
if match and method == "POST":
actor = current_user(self.headers)
return self._send_json(201, create_message(match.group(1), self._read_body(), actor), rid)
if match and method == "GET":
actor = current_user(self.headers)
limit = parse_limit(query.get("limit", [None])[0])
cursor = parse_optional_cursor(query.get("cursor", [None])[0])
return self._send_json(200, list_messages(match.group(1), actor, limit, cursor), rid)
raise ApiError(404, "not_found", "Маршрут не найден")
def main():
try:
db.init()
db.migrate_with_retry()
except Exception:
sys.stderr.write("database is not ready at startup\n")
try:
sessions.init()
except Exception:
sys.stderr.write("session store is not configured\n")
server = FastThreadingHTTPServer((APP_HOST, APP_PORT), Handler)
def shutdown(signum, frame):
sys.stderr.write("SIGTERM received, shutting down\n")
threading.Thread(target=server.shutdown, daemon=True).start()
signal.signal(signal.SIGTERM, shutdown)
signal.signal(signal.SIGINT, shutdown)
sys.stderr.write("listening on %s:%s env=%s\n" % (APP_HOST, APP_PORT, APP_ENV))
try:
server.serve_forever(poll_interval=0.05)
finally:
server.server_close()
db.close()
sys.stderr.write("stopped\n")
if __name__ == "__main__":
main()
+85
View File
@@ -0,0 +1,85 @@
"""Сессии в Valkey. Ключ — SHA-256 токена, TTL равен остатку жизни сессии.
Источник истины сессий — Valkey. Перезапуск переживается за счёт AOF и volume,
а не за счёт одного TTL: TTL только ограничивает срок, диск сохраняет ключ.
"""
from __future__ import annotations
import hashlib
import os
from pathlib import Path
import redis
from errors import ApiError
_client: redis.Redis | None = None
def session_ttl() -> int:
raw = os.environ.get("SESSION_TTL_SECONDS", "3600")
try:
value = int(raw)
except ValueError:
return 3600
return value if value >= 1 else 1
def redis_url() -> str:
path = os.environ.get("CACHE_URL_FILE", "").strip()
if path and Path(path).is_file():
return Path(path).read_text(encoding="utf-8").strip()
return os.environ.get("REDIS_URL", "").strip()
def init() -> None:
global _client
url = redis_url()
if not url:
_client = None
return
_client = redis.Redis.from_url(
url, socket_connect_timeout=0.5, socket_timeout=0.5,
decode_responses=True)
def _key(token: str) -> str:
digest = hashlib.sha256(token.encode("utf-8")).hexdigest()
return "session:" + digest
def _require() -> redis.Redis:
if _client is None:
raise ApiError(503, "unavailable", "Хранилище сессий недоступно")
return _client
def ping() -> bool:
try:
return bool(_require().ping())
except (ApiError, redis.RedisError):
return False
def create(token: str, user_id: str, ttl: int) -> None:
try:
_require().set(_key(token), user_id, ex=ttl)
except (ApiError, redis.RedisError) as exc:
raise ApiError(503, "unavailable", "Хранилище сессий недоступно") from exc
def user_id(token: str) -> str:
try:
value = _require().get(_key(token))
except (ApiError, redis.RedisError) as exc:
raise ApiError(503, "unavailable", "Хранилище сессий недоступно") from exc
if not value:
raise ApiError(401, "unauthorized", "Сессия недействительна")
return value
def revoke(token: str) -> None:
try:
_require().delete(_key(token))
except (ApiError, redis.RedisError) as exc:
raise ApiError(503, "unavailable", "Хранилище сессий недоступно") from exc
+239
View File
@@ -0,0 +1,239 @@
#!/usr/bin/env python3
"""Проверки уровня 4 для lab2. Сервер уже должен слушать BASE_URL."""
from __future__ import annotations
import argparse
import base64
import json
import subprocess
import time
import uuid
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime, timezone
from pathlib import Path
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen
ROOT = Path(__file__).resolve().parents[1]
class Failure(AssertionError):
pass
def call(base, method, path, body=None, headers=None, expected=None):
data = None
h = {"Accept": "application/json"}
if body is not None:
data = json.dumps(body, ensure_ascii=False).encode("utf-8")
h["Content-Type"] = "application/json"
h.update(headers or {})
request = Request(base + path, data=data, headers=h, method=method)
try:
response = urlopen(request, timeout=20)
status, payload_bytes, rh = response.status, response.read(), response.headers
except HTTPError as exc:
status, payload_bytes, rh = exc.code, exc.read(), exc.headers
payload = json.loads(payload_bytes) if payload_bytes else None
if expected is not None and status != expected:
raise Failure("%s %s: expected %s, got %s" % (method, path.split("?")[0], expected, status))
if not rh.get("X-Request-Id"):
raise Failure("missing X-Request-Id")
return status, rh, payload
def register(base, username=None, password=None):
username = username or ("u_" + uuid.uuid4().hex[:12])
password = password or ("T3st-" + uuid.uuid4().hex)
_, _, user = call(base, "POST", "/api/v1/users", {
"username": username,
"display_name": "Тест " + username[:12],
"password": password,
}, expected=201)
return user, password
def login(base, username, password):
raw = base64.b64encode((username + ":" + password).encode()).decode()
_, rh, session = call(
base, "POST", "/api/v1/auth/sessions",
headers={"Authorization": "Basic " + raw}, expected=201)
return session, rh
def auth(token):
return {"Authorization": "Bearer " + token}
def test_wrong_password(base):
user, _password = register(base)
bad = base64.b64encode((user["username"] + ":wrong-password").encode()).decode()
_, rh, body = call(
base, "POST", "/api/v1/auth/sessions",
headers={"Authorization": "Basic " + bad}, expected=401)
if not rh.get("WWW-Authenticate", "").lower().startswith("basic"):
raise Failure("wrong password must challenge Basic")
if body["error"]["code"] != "unauthorized":
raise Failure("wrong password code")
print("PASS wrong_password")
def test_logout(base):
user, password = register(base)
session, _rh = login(base, user["username"], password)
call(base, "DELETE", "/api/v1/auth/sessions/current",
headers=auth(session["token"]), expected=204)
_, rh, _body = call(base, "GET", "/api/v1/users/me",
headers=auth(session["token"]), expected=401)
if not rh.get("WWW-Authenticate", "").lower().startswith("bearer"):
raise Failure("revoked session must challenge Bearer")
print("PASS logout")
def test_x_user_id_ignored(base):
alice, alice_password = register(base)
eve, _eve_password = register(base)
session, _rh = login(base, alice["username"], alice_password)
headers = auth(session["token"])
headers["X-User-Id"] = eve["id"]
_, _, me = call(base, "GET", "/api/v1/users/me", headers=headers, expected=200)
if me["id"] != alice["id"]:
raise Failure("X-User-Id changed the session user")
call(base, "GET", "/api/v1/users/me", headers={"X-User-Id": alice["id"]}, expected=401)
print("PASS x_user_id")
def test_duplicate_race(base):
username = "u_" + uuid.uuid4().hex[:12]
password = "T3st-" + uuid.uuid4().hex
def once(_idx):
return call(base, "POST", "/api/v1/users", {
"username": username,
"display_name": "Гонка",
"password": password,
})[0]
with ThreadPoolExecutor(max_workers=8) as pool:
codes = list(pool.map(once, range(8)))
if codes.count(201) != 1 or codes.count(409) != 7:
raise Failure("duplicate username race: %s" % codes)
print("PASS duplicate_race")
def test_chat_rollback(base):
user, password = register(base)
session, _rh = login(base, user["username"], password)
title = "rollback-" + uuid.uuid4().hex[:8]
missing = str(uuid.uuid4())
call(base, "POST", "/api/v1/chats", {
"title": title,
"member_ids": [user["id"], missing],
}, headers=auth(session["token"]), expected=400)
_, _, page = call(base, "GET", "/api/v1/chats?limit=100",
headers=auth(session["token"]), expected=200)
if any(item["title"] == title for item in page["items"]):
raise Failure("rolled back chat is still visible")
print("PASS chat_rollback")
def test_bad_cursor_and_limit(base):
user, password = register(base)
session, _rh = login(base, user["username"], password)
_, _, chat = call(base, "POST", "/api/v1/chats", {
"title": "Страницы",
"member_ids": [user["id"]],
}, headers=auth(session["token"]), expected=201)
headers = auth(session["token"])
call(base, "GET", "/api/v1/chats/%s/messages?limit=" % chat["id"],
headers=headers, expected=400)
token = base64.urlsafe_b64encode(
("%s|not-a-timestamp|not-a-uuid" % chat["id"]).encode()
).decode().rstrip("=")
call(base, "GET", "/api/v1/chats/%s/messages?cursor=%s" % (chat["id"], quote(token, safe="")),
headers=headers, expected=400)
print("PASS bad_cursor_and_limit")
def test_expiry(base):
user, password = register(base)
session, _rh = login(base, user["username"], password)
expires = datetime.fromisoformat(session["expires_at"].replace("Z", "+00:00"))
wait = (expires - datetime.now(timezone.utc)).total_seconds() + 1
if wait > 30:
raise Failure("SESSION_TTL_SECONDS is too large for the expiry check (%.0fs)" % wait)
time.sleep(max(wait, 0))
call(base, "GET", "/api/v1/users/me", headers=auth(session["token"]), expected=401)
print("PASS expiry")
def compose(args):
subprocess.run(
["docker", "compose", "--env-file", ".env", *args],
cwd=ROOT, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
def wait_ready(base, timeout=60):
deadline = time.time() + timeout
while time.time() < deadline:
try:
call(base, "GET", "/health/ready", expected=200)
return
except Failure:
time.sleep(0.5)
raise Failure("service did not become ready")
def test_dependencies(base):
user, password = register(base)
session, _rh = login(base, user["username"], password)
headers = auth(session["token"])
try:
compose(["stop", "valkey"])
time.sleep(1)
call(base, "GET", "/health/ready", expected=503)
call(base, "GET", "/api/v1/users/me", headers=headers, expected=503)
call(base, "GET", "/api/v1/users/me", expected=401)
finally:
compose(["start", "valkey"])
wait_ready(base)
call(base, "GET", "/api/v1/users/me", headers=headers, expected=200)
try:
compose(["stop", "postgres"])
time.sleep(1)
call(base, "GET", "/health/ready", expected=503)
call(base, "GET", "/api/v1/users/me", headers=headers, expected=503)
finally:
compose(["start", "postgres"])
wait_ready(base)
call(base, "GET", "/api/v1/users/me", headers=headers, expected=200)
print("PASS dependencies")
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--base-url", default="http://localhost:8080")
parser.add_argument("--expiry", action="store_true")
parser.add_argument("--deps", action="store_true")
args = parser.parse_args()
base = args.base_url.rstrip("/")
test_wrong_password(base)
test_logout(base)
test_x_user_id_ignored(base)
test_duplicate_race(base)
test_chat_rollback(base)
test_bad_cursor_and_limit(base)
if args.expiry:
test_expiry(base)
if args.deps:
test_dependencies(base)
print("ALL STUDENT TESTS PASSED")
if __name__ == "__main__":
try:
main()
except (Failure, OSError, subprocess.CalledProcessError) as exc:
print("FAIL:", exc)
raise SystemExit(1)