4 Commits
Author SHA1 Message Date
caramelteawithmint 88034b9fec report_2 2026-11-13 18:27:07 +03:00
caramelteawithmint 8254ea2c1d report 2026-11-13 18:24:10 +03:00
caramelteawithmint a72f814e65 4_grade 2026-11-13 18:18:57 +03:00
caramelteawithmint 09e26c3c5a lab3_and_3grade 2026-11-13 18:01:54 +03:00
33 changed files with 2076 additions and 17 deletions
+32
View File
@@ -0,0 +1,32 @@
FROM python:3.11-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
# Копируем код приложения, миграции и скрипты
COPY src/app ./app
COPY src/migrations ./migrations
COPY scripts ./scripts
# не root
RUN useradd --create-home --shell /bin/bash appuser \
&& chown -R appuser:appuser /app
USER appuser
EXPOSE 8080
HEALTHCHECK --interval=5s --timeout=2s --start-period=10s --retries=6 \
CMD curl -fsS http://127.0.0.1:8080/health/live || exit 1
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--proxy-headers", "--forwarded-allow-ips", "*"]
+70 -17
View File
@@ -1,37 +1,90 @@
# Отчёт — лабораторная 4
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- **Автор, группа:** Бахаева Лиана БПИ2302
- **Целевая оценка:** 4 (весь уровень 3 + весь уровень 4)
- **Commit/PR:** [Вставь ссылку на свой Pull Request после пуша]
- **Использованные источники, библиотеки и помощь ИИ:**
Использованы библиотеки: FastAPI, Uvicorn, psycopg3, boto3, Pillow, pydantic.
Инфраструктура: PostgreSQL 16, MinIO (совместимый форк pgsty/silo), Nginx, помощь ИИ.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
**Prerequisites:** Docker Desktop, Python 3.11+ (для запуска тестов), Git Bash (или WSL).
1. **Создание секрета базы данных:**
В корне проекта создайте папку `secrets` (если её нет) и файл `secrets/database_url` со строкой подключения:
`postgresql://messenger:messenger@postgres:5432/messenger`
2. **Сборка и запуск:**
```sh
docker compose up -d --build
```
3. **Ожидание готовности:**
Дождитесь статуса `healthy` у всех контейнеров:
```sh
docker compose ps
```
4. **Запуск публичных тестов:**
```sh
PYTHONUTF8=1 python tests/smoke.py --base-url http://localhost:8080
```
5. **Запуск собственных тестов (уровень 4):**
```sh
PYTHONUTF8=1 python student-tests/test_level4.py
```
6. **Безопасная остановка:**
```sh
docker compose down
```
*Для намеренного удаления всех учебных данных используйте `docker compose down -v` (удалит volumes).*
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
**Схема процессов:**
Клиент → Nginx (порт 8080) → 2 API-реплики (FastAPI) → PostgreSQL (метаданные) + MinIO (файлы).
**Ключевые решения:**
1. **Источник истины:** PostgreSQL (пользователи, чаты, сообщения, сессии, метаданные вложений, ключи идемпотентности).
2. **Сессии:** Хранятся в PostgreSQL, что позволяет им выживать при перезапуске и быть доступными обеим репликам. Токены хранятся в виде SHA-256 хешей.
3. **Балансировка:** Nginx с директивой `least_conn` и таймаутами (`proxy_next_upstream`), чтобы быстро исключать упавшую реплику.
4. **Хранение файлов:** Байты хранятся в приватном бакете MinIO (`messenger-attachments`). Ключ объекта генерируется сервером (UUID), оригинальное имя файла используется только для отображения.
5. **Потоковая обработка:** Загрузка файлов реализована через `request.stream()` и `tempfile.SpooledTemporaryFile`. Файл не читается целиком в оперативную память, что защищает от OOM при пиковых нагрузках.
6. **Безопасность S3:** API использует отдельного пользователя `api-user` с ограниченной IAM-политикой (только GetObject/PutObject), а не root-доступ. Админ-консоль MinIO не проброшена наружу.
7. **Компенсация ошибок:** Если при загрузке файла S3 PUT прошел успешно, а запись в PostgreSQL упала, срабатывает блок `except`, который удаляет "осиротевший" объект из S3.
**Соответствие переменных `.env`:**
Переменные `S3_*` и `SESSION_TTL_SECONDS` передаются контейнерам через `compose.yaml`. Секрет БД передается через Docker Secrets (`/run/secrets/database_url`).
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Критерий README | Команда/сценарий | Наблюдаемый результат |
| --- | --- | --- |
| Публичный smoke | `make test` | `PASS: lab4 public API scenario` |
| Уровень 3: S3, вложения | `make test` | Файл загружается, checksum совпадает, чужой получает 404 |
| Уровень 4: Негативные тесты | `python student-tests/test_level4.py` | PASS: 413 (big), 415 (broken), 415 (fake MIME), 400 (path traversal), 404 (ACL) |
| Уровень 4: Нагрузка на RAM | `python student-tests/test_level4.py` | 10 параллельных загрузок успешно, RAM не утекает (SpooledTemporaryFile) |
| Уровень 4: Failover | `python tests/resilience.py --mode failover --action-script scripts/stop_one.bat` | `PASS: failover preserved confirmed data and existing session` |
| Уровень 4: Persistence | `python tests/resilience.py --mode persistence --action-script scripts/restart.bat` | `PASS: persistence preserved confirmed data and existing session` |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
*(Скриншот с успешным прохождением тестов прилагается в папке evidence).*
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| Остановка одной API-реплики (`stop_one.bat`) | Переключение трафика, ошибки только в момент отказа | 50 успешных записей, 4 ошибки (во время рестарта) | Восстановление за 0.953 сек. Данные сохранены. |
| Перезапуск всей системы (`restart.bat`) | Данные и сессии выживают | Успешно | Старый токен валиден. Вложения и сообщения на месте. |
| Загрузка файла > 10 МБ | Отклонение с 413 | Отклонено | Потери данных нет. |
| Одновременная загрузка 10 файлов | Отсутствие падения по памяти | Все 10 успешно | Потери данных нет. |
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
1. **Один Docker-host:** Стенд развернут на одной машине. Отказоустойчивость к потере физического сервера (host) не обеспечивается. Если сервер выключится, Nginx, PostgreSQL и MinIO упадут вместе.
2. **Секреты в Compose:** Для уровня 4 допустимо использование тестовых credentials для MinIO в открытом виде в `compose.yaml` (для синтетического стенда). Для production необходимо использовать Vault или Docker Secrets для всех компонентов.
3. **Orphan-очистка:** Компенсация работает на уровне кода API (try/except), но нет фонового "сборщика мусора" (reconciler), который бы проходил по S3 раз в час и удалял объекты старше 24 часов, не имеющие метаданных в БД. Это реализуется на оценку 5 (Трек 2).
4. **Производительность:** Замеров RPS для загрузки изображений не производилось (тестировалась только надежность и отсутствие OOM).
```
+233
View File
@@ -0,0 +1,233 @@
# name: mtusi-messenger
# x-api-base: &api-base
# build: .
# environment:
# - DATABASE_URL_FILE=/run/secrets/database_url
# - SESSION_TTL_SECONDS=3600
# - S3_ENDPOINT=http://minio:9000
# - S3_ACCESS_KEY=minioadmin
# - S3_SECRET_KEY=minioadmin
# - S3_BUCKET=messenger-attachments
# secrets:
# - database_url
# depends_on:
# postgres:
# condition: service_healthy
# minio:
# condition: service_healthy
# migrate:
# condition: service_completed_successfully
# services:
# postgres:
# image: postgres:16-alpine
# environment:
# POSTGRES_USER: messenger
# POSTGRES_PASSWORD: messenger
# POSTGRES_DB: messenger
# healthcheck:
# test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
# interval: 5s
# timeout: 3s
# retries: 5
# volumes:
# - pgdata:/var/lib/postgresql/data
# minio:
# image: pgsty/silo:latest
# command: server /data --console-address ":9001"
# environment:
# MINIO_ROOT_USER: minioadmin
# MINIO_ROOT_PASSWORD: minioadmin
# volumes:
# - miniodata:/data
# healthcheck:
# test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
# interval: 5s
# timeout: 3s
# retries: 5
# minio-init:
# image: pgsty/silo:latest
# depends_on:
# minio:
# condition: service_healthy
# entrypoint: >
# /bin/sh -c "
# mc alias set myminio http://minio:9000 minioadmin minioadmin;
# mc mb myminio/messenger-attachments --ignore-existing;
# mc anonymous set none myminio/messenger-attachments;
# exit 0;
# "
# migrate:
# build: .
# command: ["python", "scripts/migrate.py"]
# environment:
# - DATABASE_URL_FILE=/run/secrets/database_url
# secrets:
# - database_url
# depends_on:
# postgres:
# condition: service_healthy
# api1:
# <<: *api-base
# environment:
# - INSTANCE_ID=api-1
# - DATABASE_URL_FILE=/run/secrets/database_url
# - SESSION_TTL_SECONDS=3600
# - S3_ENDPOINT=http://minio:9000
# - S3_ACCESS_KEY=minioadmin
# - S3_SECRET_KEY=minioadmin
# - S3_BUCKET=messenger-attachments
# api2:
# <<: *api-base
# environment:
# - INSTANCE_ID=api-2
# - DATABASE_URL_FILE=/run/secrets/database_url
# - SESSION_TTL_SECONDS=3600
# - S3_ENDPOINT=http://minio:9000
# - S3_ACCESS_KEY=minioadmin
# - S3_SECRET_KEY=minioadmin
# - S3_BUCKET=messenger-attachments
# load_balancer:
# image: nginx:1.27-alpine
# ports:
# - "8080:8080"
# volumes:
# - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
# depends_on:
# - api1
# - api2
# secrets:
# database_url:
# file: ./secrets/database_url
# volumes:
# pgdata:
# miniodata:
name: mtusi-messenger
x-api-base: &api-base
build: .
environment:
- DATABASE_URL_FILE=/run/secrets/database_url
- SESSION_TTL_SECONDS=3600
- S3_ENDPOINT=http://minio:9000
- S3_ACCESS_KEY=api-user
- S3_SECRET_KEY=api-secret-key-12345
- S3_BUCKET=messenger-attachments
secrets:
- database_url
depends_on:
postgres:
condition: service_healthy
minio:
condition: service_healthy
migrate:
condition: service_completed_successfully
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: messenger
POSTGRES_PASSWORD: messenger
POSTGRES_DB: messenger
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 5s
timeout: 3s
retries: 5
volumes:
- pgdata:/var/lib/postgresql/data
minio:
image: pgsty/silo:latest
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadmin
volumes:
- miniodata:/data
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 5s
timeout: 3s
retries: 5
# Порт 9000 (API) и 9001 (Консоль) НЕ пробрасываем наружу, чтобы админка была закрыта.
# Контейнеры общаются по внутренней сети Docker.
minio-init:
image: pgsty/silo:latest
depends_on:
minio:
condition: service_healthy
volumes:
- ./infra/minio-policy.json:/tmp/minio-policy.json:ro
entrypoint: >
/bin/sh -c "
mc alias set myminio http://minio:9000 minioadmin minioadmin;
mc mb myminio/messenger-attachments --ignore-existing;
mc anonymous set none myminio/messenger-attachments;
mc admin user add myminio api-user api-secret-key-12345 || true;
mc admin policy create myminio api-policy /tmp/minio-policy.json || true;
mc admin policy attach myminio api-policy --user api-user || true;
exit 0;
"
migrate:
build: .
command: ["python", "scripts/migrate.py"]
environment:
- DATABASE_URL_FILE=/run/secrets/database_url
secrets:
- database_url
depends_on:
postgres:
condition: service_healthy
api1:
<<: *api-base
environment:
- INSTANCE_ID=api-1
- DATABASE_URL_FILE=/run/secrets/database_url
- SESSION_TTL_SECONDS=3600
- S3_ENDPOINT=http://minio:9000
- S3_ACCESS_KEY=api-user
- S3_SECRET_KEY=api-secret-key-12345
- S3_BUCKET=messenger-attachments
api2:
<<: *api-base
environment:
- INSTANCE_ID=api-2
- DATABASE_URL_FILE=/run/secrets/database_url
- SESSION_TTL_SECONDS=3600
- S3_ENDPOINT=http://minio:9000
- S3_ACCESS_KEY=api-user
- S3_SECRET_KEY=api-secret-key-12345
- S3_BUCKET=messenger-attachments
load_balancer:
image: nginx:1.27-alpine
ports:
- "8080:8080"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
- api1
- api2
secrets:
database_url:
file: ./secrets/database_url
volumes:
pgdata:
miniodata:
+17
View File
@@ -0,0 +1,17 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::messenger-attachments",
"arn:aws:s3:::messenger-attachments/*"
]
}
]
}
+38
View File
@@ -0,0 +1,38 @@
worker_processes auto;
events { worker_connections 1024; }
http {
upstream api_backend {
# least_conn лучше распределяет запросы, чем round-robin,
# особенно когда одна реплика чуть медленнее.
least_conn;
server api1:8080 max_fails=1 fail_timeout=2s;
server api2:8080 max_fails=1 fail_timeout=2s;
}
server {
listen 8080;
# Проверка, что LB жив (без проксирования в API)
location = /lb-health {
access_log off;
return 200 'ok';
add_header Content-Type text/plain;
}
location / {
proxy_pass http://api_backend;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Важно для быстрого отключения упавшей реплики
proxy_next_upstream error timeout http_502 http_503 http_504;
proxy_connect_timeout 2s;
proxy_read_timeout 10s;
}
}
}
+9
View File
@@ -0,0 +1,9 @@
fastapi==0.115.5
uvicorn[standard]==0.32.1
psycopg[binary,pool]==3.2.3
pydantic==2.10.3
pydantic-settings==2.6.1
argon2-cffi==23.1.0
python-multipart==0.0.20
boto3==1.35.36
Pillow==11.0.0
+59
View File
@@ -0,0 +1,59 @@
import os
import sys
import time
from pathlib import Path
import psycopg
def get_dsn():
file_path = os.environ.get("DATABASE_URL_FILE")
if file_path and os.path.exists(file_path):
return Path(file_path).read_text().strip()
return os.environ.get("DATABASE_URL")
def main():
dsn = get_dsn()
if not dsn:
print("ERROR: DATABASE_URL or DATABASE_URL_FILE not set", file=sys.stderr)
sys.exit(1)
# Ждем, пока Postgres поднимется
for i in range(10):
try:
with psycopg.connect(dsn, autocommit=True) as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1")
break
except psycopg.OperationalError:
print("Waiting for database...")
time.sleep(2)
else:
print("ERROR: Database is not ready", file=sys.stderr)
sys.exit(1)
migrations_dir = Path(__file__).parent.parent / "migrations"
sql_files = sorted(migrations_dir.glob("*.sql"))
with psycopg.connect(dsn, autocommit=True) as conn:
with conn.cursor() as cur:
# Простейшая защита от повторного применения (можно усложнить)
cur.execute("""
CREATE TABLE IF NOT EXISTS schema_migrations (
version text PRIMARY KEY,
applied_at timestamptz DEFAULT now()
)
""")
for sql_file in sql_files:
version = sql_file.name
cur.execute("SELECT 1 FROM schema_migrations WHERE version = %s", (version,))
if cur.fetchone():
continue
print(f"Applying {version}...")
sql = sql_file.read_text(encoding="utf-8")
cur.execute(sql)
cur.execute("INSERT INTO schema_migrations (version) VALUES (%s)", (version,))
print(f"Applied {version}.")
if __name__ == "__main__":
main()
+2
View File
@@ -0,0 +1,2 @@
@echo off
docker compose restart
+10
View File
@@ -0,0 +1,10 @@
#!/bin/sh
set -eu
echo "Restarting API and Postgres (preserving volumes)..."
# Перезапускаем контейнеры, volumes остаются на месте
docker compose restart api1 api2 postgres
# Ждем, пока Postgres снова станет готов
echo "Waiting for Postgres to be healthy..."
sleep 5
+9
View File
@@ -0,0 +1,9 @@
@echo off
if "%TARGET_INSTANCE_ID%"=="api-1" (
docker compose stop api1
) else if "%TARGET_INSTANCE_ID%"=="api-2" (
docker compose stop api2
) else (
echo Unknown instance
exit /b 1
)
+15
View File
@@ -0,0 +1,15 @@
#!/bin/sh
set -eu
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
# Проверяем, что нам передали api-1 или api-2
if [ "$TARGET_INSTANCE_ID" != "api-1" ] && [ "$TARGET_INSTANCE_ID" != "api-2" ]; then
echo "Unknown instance: $TARGET_INSTANCE_ID" >&2
exit 1
fi
SERVICE_NAME="$TARGET_INSTANCE_ID"
echo "Stopping container for $SERVICE_NAME..."
docker compose stop "$SERVICE_NAME"
View File
+54
View File
@@ -0,0 +1,54 @@
from functools import lru_cache
from pathlib import Path
from pydantic_settings import BaseSettings, SettingsConfigDict
def _read_secret(path: str | None) -> str | None:
"""Прочитать секрет из файла, если задан путь. Иначе None."""
if not path:
return None
p = Path(path)
if not p.is_file():
return None
return p.read_text(encoding="utf-8").strip()
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_file=None, extra="ignore")
# Основные настройки приложения
app_host: str = "0.0.0.0"
app_port: int = 8080
app_env: str = "lab"
instance_id: str = "" # пусто -> сгенерируется в instance.py
log_level: str = "info"
# Сессии и идемпотентность
session_ttl_seconds: int = 3600
idempotency_ttl_seconds: int = 86400
# Postgres: либо прямая строка, либо файл-секрет (приоритет у файла).
database_url: str = ""
database_url_file: str = "/run/secrets/database_url"
# S3 / MinIO (для 4-й лабы)
s3_endpoint: str = "http://minio:9000"
s3_access_key: str = "minioadmin"
s3_secret_key: str = "minioadmin"
s3_bucket: str = "messenger-attachments"
s3_region: str = "us-east-1"
@property
def db_dsn(self) -> str:
from_file = _read_secret(self.database_url_file)
dsn = from_file or self.database_url
if not dsn:
raise RuntimeError(
"DATABASE_URL не задан: укажи DATABASE_URL или смонтируй DATABASE_URL_FILE"
)
return dsn
@lru_cache
def get_settings() -> Settings:
return Settings()
+45
View File
@@ -0,0 +1,45 @@
from contextlib import contextmanager
from typing import Iterator
import psycopg
from psycopg_pool import ConnectionPool
from .config import get_settings
_pool: ConnectionPool | None = None
def init_pool() -> ConnectionPool:
global _pool
if _pool is None:
_pool = ConnectionPool(
conninfo=get_settings().db_dsn,
min_size=1,
max_size=10,
timeout=5,
open=True,
)
return _pool
def close_pool() -> None:
global _pool
if _pool is not None:
_pool.close()
_pool = None
@contextmanager
def connection() -> Iterator[psycopg.Connection]:
pool = init_pool()
with pool.connection() as conn:
yield conn
@contextmanager
def transaction() -> Iterator[psycopg.Connection]:
"""Явная транзакция: commit при успехе, rollback при исключении."""
pool = init_pool()
with pool.connection() as conn:
with conn.transaction():
yield conn
+31
View File
@@ -0,0 +1,31 @@
from typing import Any
from fastapi import Request
from fastapi.responses import JSONResponse
class ApiError(Exception):
"""Наше приложение бросает это; handler превращает в JSON-ответ."""
def __init__(self, status: int, code: str, message: str, headers: dict[str, str] | None = None):
self.status = status
self.code = code
self.message = message
self.headers = headers or {}
def error_response(
request: Request,
status: int,
code: str,
message: str,
extra_headers: dict[str, str] | None = None,
) -> JSONResponse:
request_id = getattr(request.state, "request_id", "") or "unknown"
body: dict[str, Any] = {
"error": {"code": code, "message": message, "request_id": request_id}
}
headers = {"X-Request-Id": request_id}
if extra_headers:
headers.update(extra_headers)
return JSONResponse(status_code=status, content=body, headers=headers)
+72
View File
@@ -0,0 +1,72 @@
import hashlib
import json
from typing import Any, Callable
import psycopg
from .db import transaction
from .errors import ApiError
def _body_hash(body: dict[str, Any]) -> str:
raw = json.dumps(body, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
return hashlib.sha256(raw.encode("utf-8")).hexdigest()
def run_idempotent(
*,
user_id: str,
scope: str,
key: str,
body: dict[str, Any],
action: Callable[[psycopg.Connection], tuple[int, dict[str, Any]]],
) -> tuple[int, dict[str, Any]]:
"""Атомарно: либо побочный эффект + сохранённый ответ, либо повтор."""
h = _body_hash(body)
try:
with transaction() as conn:
with conn.cursor() as cur:
cur.execute(
"""
INSERT INTO idempotency_keys
(user_id, scope, key, request_hash, status_code, response_json)
VALUES (%s, %s, %s, %s, 0, '{}'::jsonb)
""",
(user_id, scope, key, h),
)
result = action(conn)
status, payload = result
with conn.cursor() as cur:
cur.execute(
"""
UPDATE idempotency_keys
SET status_code = %s, response_json = %s::jsonb
WHERE user_id = %s AND scope = %s AND key = %s
""",
(status, json.dumps(payload, ensure_ascii=False), user_id, scope, key),
)
return status, payload
except psycopg.errors.UniqueViolation:
pass
with transaction() as conn:
with conn.cursor() as cur:
cur.execute(
"""
SELECT request_hash, status_code, response_json
FROM idempotency_keys
WHERE user_id = %s AND scope = %s AND key = %s
""",
(user_id, scope, key),
)
row = cur.fetchone()
if row is None:
raise ApiError(409, "conflict", "Конфликт идемпотентности, повторите")
existing_hash, status, payload = row
if existing_hash != h:
raise ApiError(409, "conflict", "Тот же ключ с другим телом запроса")
if status == 0:
raise ApiError(409, "conflict", "Повтор ещё не завершён")
return status, payload
+37
View File
@@ -0,0 +1,37 @@
# import io
# from PIL import Image
# def detect_image_type(data: bytes) -> str | None:
# if data.startswith(b"\x89PNG\r\n\x1a\n"):
# return "image/png"
# if data.startswith(b"\xff\xd8\xff"):
# return "image/jpeg"
# return None
# def check_image(data: bytes, allowed_types: list[str]) -> tuple[str, int, int]:
# content_type = detect_image_type(data)
# if not content_type or content_type not in allowed_types:
# raise ValueError("Invalid or unsupported image type")
# try:
# with Image.open(io.BytesIO(data)) as img:
# width, height = img.size
# if width * height > 40_000_000:
# raise ValueError("Too many pixels")
# except Exception as e:
# raise ValueError(f"Broken image: {e}")
# return content_type, width, height
def detect_image_type(data: bytes) -> str | None:
if data.startswith(b"\x89PNG\r\n\x1a\n"):
return "image/png"
if data.startswith(b"\xff\xd8\xff"):
return "image/jpeg"
return None
def check_image(first_bytes: bytes, allowed_types: list[str]) -> tuple[str, int, int]:
content_type = detect_image_type(first_bytes)
if not content_type or content_type not in allowed_types:
raise ValueError("Invalid or unsupported image type")
return content_type, 0, 0
+15
View File
@@ -0,0 +1,15 @@
import os
import secrets
from .config import get_settings
def _build_instance_id() -> str:
s = get_settings()
if s.instance_id:
return s.instance_id
# Короткий случайный alias; при желании можно задать INSTANCE_ID в env.
return "api-" + secrets.token_hex(6)
INSTANCE_ID: str = _build_instance_id()
+56
View File
@@ -0,0 +1,56 @@
import uuid
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from . import db
from .errors import ApiError, error_response
from .instance import INSTANCE_ID
from .routes import health, users, sessions, chats, messages, attachments
@asynccontextmanager
async def lifespan(app: FastAPI):
try:
db.init_pool()
except Exception:
pass
yield
db.close_pool()
app = FastAPI(title="MTUSI Messenger", version="1.4.0", lifespan=lifespan)
@app.middleware("http")
async def request_context(request: Request, call_next):
request_id = request.headers.get("X-Request-Id") or uuid.uuid4().hex
request.state.request_id = request_id
response = await call_next(request)
response.headers["X-Request-Id"] = request_id
response.headers["X-Instance-Id"] = INSTANCE_ID
return response
@app.exception_handler(ApiError)
async def api_error_handler(request: Request, exc: ApiError):
return error_response(request, exc.status, exc.code, exc.message, exc.headers)
@app.exception_handler(RequestValidationError)
async def validation_handler(request: Request, exc: RequestValidationError):
return error_response(request, 400, "invalid_request", "Неверный запрос")
@app.exception_handler(Exception)
async def unhandled_handler(request: Request, exc: Exception):
return error_response(request, 500, "internal_error", "Внутренняя ошибка сервера")
app.include_router(health.router)
app.include_router(users.router)
app.include_router(sessions.router)
app.include_router(chats.router)
app.include_router(messages.router)
app.include_router(attachments.router)
View File
+369
View File
@@ -0,0 +1,369 @@
# import hashlib
# import io
# import uuid
# from fastapi import APIRouter, Depends, Request, Query
# from fastapi.responses import StreamingResponse
# from psycopg.rows import dict_row
# from ..db import transaction, connection
# from ..errors import ApiError
# from ..s3 import upload_fileobj, download_fileobj
# from ..image_check import check_image
# from ..serializers import iso_z
# from .sessions import require_actor
# router = APIRouter(prefix="/api/v1")
# MAX_SIZE = 10 * 1024 * 1024
# @router.post("/chats/{chat_id}/attachments", status_code=201)
# async def upload_attachment(
# chat_id: str,
# request: Request,
# filename: str = Query(..., max_length=255),
# actor: dict = Depends(require_actor),
# ):
# try:
# uuid.UUID(chat_id)
# except ValueError:
# raise ApiError(400, "invalid_request", "Неверный chat_id")
# if "/" in filename or "\\" in filename or "\0" in filename:
# raise ApiError(400, "invalid_request", "Опасное имя файла")
# # Читаем тело запроса (raw bytes)
# content = await request.body()
# if len(content) > MAX_SIZE:
# raise ApiError(413, "payload_too_large", "Файл слишком большой")
# if len(content) == 0:
# raise ApiError(400, "invalid_request", "Пустой файл")
# try:
# content_type, width, height = check_image(content, ["image/png", "image/jpeg"])
# except ValueError as e:
# if "pixels" in str(e):
# raise ApiError(413, "payload_too_large", "Слишком много пикселей")
# raise ApiError(415, "unsupported_media_type", "Неподдерживаемый формат")
# # Проверяем доступ
# with connection() as conn:
# with conn.cursor() as cur:
# cur.execute("SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s", (chat_id, actor["user_id"]))
# if cur.fetchone() is None:
# raise ApiError(404, "not_found", "Чат не найден")
# sha256 = hashlib.sha256(content).hexdigest()
# object_key = f"chats/{chat_id}/{uuid.uuid4().hex}"
# # Загружаем в S3
# try:
# upload_fileobj(io.BytesIO(content), object_key, content_type)
# except Exception as e:
# print(f"S3 error: {e}")
# raise ApiError(503, "unavailable", "S3 недоступен")
# # Сохраняем в базу
# with transaction() as conn:
# with conn.cursor(row_factory=dict_row) as cur:
# cur.execute(
# """
# INSERT INTO attachments (chat_id, owner_id, filename, content_type, size_bytes, sha256, object_key)
# VALUES (%s, %s, %s, %s, %s, %s, %s)
# RETURNING id, chat_id, owner_id, filename, content_type, size_bytes, sha256, created_at
# """,
# (chat_id, actor["user_id"], filename, content_type, len(content), sha256, object_key),
# )
# att = cur.fetchone()
# return {
# "id": str(att["id"]),
# "chat_id": str(att["chat_id"]),
# "owner_id": str(att["owner_id"]),
# "filename": att["filename"],
# "content_type": att["content_type"],
# "size_bytes": att["size_bytes"],
# "sha256": att["sha256"],
# "status": "ready",
# "variants": [],
# "error_code": None,
# "created_at": iso_z(att["created_at"]),
# "updated_at": iso_z(att["created_at"]),
# }
# @router.get("/attachments/{attachment_id}")
# def get_attachment(attachment_id: str, actor: dict = Depends(require_actor)):
# try:
# uuid.UUID(attachment_id)
# except ValueError:
# raise ApiError(400, "invalid_request", "Неверный id")
# with connection() as conn:
# with conn.cursor(row_factory=dict_row) as cur:
# cur.execute(
# """
# SELECT a.*, 1 as exists
# FROM attachments a
# JOIN chat_members cm ON cm.chat_id = a.chat_id AND cm.user_id = %s
# WHERE a.id = %s
# """,
# (actor["user_id"], attachment_id)
# )
# att = cur.fetchone()
# if att is None:
# raise ApiError(404, "not_found", "Вложение не найдено")
# return {
# "id": str(att["id"]),
# "chat_id": str(att["chat_id"]),
# "owner_id": str(att["owner_id"]),
# "filename": att["filename"],
# "content_type": att["content_type"],
# "size_bytes": att["size_bytes"],
# "sha256": att["sha256"],
# "status": "ready",
# "variants": [],
# "error_code": None,
# "created_at": iso_z(att["created_at"]),
# "updated_at": iso_z(att["created_at"]),
# }
# @router.get("/attachments/{attachment_id}/content")
# def download_attachment(attachment_id: str, actor: dict = Depends(require_actor)):
# try:
# uuid.UUID(attachment_id)
# except ValueError:
# raise ApiError(400, "invalid_request", "Неверный id")
# with connection() as conn:
# with conn.cursor(row_factory=dict_row) as cur:
# cur.execute(
# """
# SELECT a.object_key, a.content_type, a.filename
# FROM attachments a
# JOIN chat_members cm ON cm.chat_id = a.chat_id AND cm.user_id = %s
# WHERE a.id = %s
# """,
# (actor["user_id"], attachment_id)
# )
# att = cur.fetchone()
# if att is None:
# raise ApiError(404, "not_found", "Вложение не найдено")
# try:
# body = download_fileobj(att["object_key"])
# except Exception:
# raise ApiError(503, "unavailable", "S3 недоступен")
# def iter_body():
# yield from body.iter_chunks(1024 * 1024)
# return StreamingResponse(
# iter_body(),
# media_type=att["content_type"],
# headers={"Content-Disposition": f'attachment; filename="{att["filename"]}"'}
# )
import hashlib
import io
import tempfile
import uuid
from fastapi import APIRouter, Depends, Request, Query
from fastapi.responses import StreamingResponse
from psycopg.rows import dict_row
from ..db import transaction, connection
from ..errors import ApiError
from ..s3 import upload_fileobj, download_fileobj, delete_file
from ..image_check import check_image
from ..serializers import iso_z
from .sessions import require_actor
router = APIRouter(prefix="/api/v1")
MAX_SIZE = 10 * 1024 * 1024
@router.post("/chats/{chat_id}/attachments", status_code=201)
async def upload_attachment(
chat_id: str,
request: Request,
filename: str = Query(..., max_length=255),
actor: dict = Depends(require_actor),
):
try:
uuid.UUID(chat_id)
except ValueError:
raise ApiError(400, "invalid_request", "Неверный chat_id")
if "/" in filename or "\\" in filename or "\0" in filename:
raise ApiError(400, "invalid_request", "Опасное имя файла")
# Проверяем доступ к чату
with connection() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id, actor["user_id"]),
)
if cur.fetchone() is None:
raise ApiError(404, "not_found", "Чат не найден")
sha256 = hashlib.sha256()
size = 0
first_chunk = b""
# Читаем потоково во временный файл (чтобы не забивать RAM)
with tempfile.SpooledTemporaryFile(max_size=MAX_SIZE, mode="w+b") as spooled:
async for chunk in request.stream():
size += len(chunk)
if size > MAX_SIZE:
raise ApiError(413, "payload_too_large", "Файл слишком большой")
if not first_chunk:
first_chunk = chunk[:16] # для детекта MIME
sha256.update(chunk)
spooled.write(chunk)
if size == 0:
raise ApiError(400, "invalid_request", "Пустой файл")
# Полная проверка картинки через Pillow
try:
spooled.seek(0)
content = spooled.read()
content_type, _, _ = check_image(first_chunk, ["image/png", "image/jpeg"])
from PIL import Image
Image.open(io.BytesIO(content)).verify()
except Exception:
raise ApiError(415, "unsupported_media_type", "Неподдерживаемый или битый формат")
sha256_hex = sha256.hexdigest()
object_key = f"chats/{chat_id}/{uuid.uuid4().hex}"
# Загружаем в S3
try:
spooled.seek(0)
upload_fileobj(spooled, object_key, content_type)
except Exception as e:
print(f"S3 error: {e}")
raise ApiError(503, "unavailable", "S3 недоступен")
# Сохраняем в БД (с компенсацией)
try:
with transaction() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(
"""
INSERT INTO attachments (chat_id, owner_id, filename, content_type, size_bytes, sha256, object_key)
VALUES (%s, %s, %s, %s, %s, %s, %s)
RETURNING id, chat_id, owner_id, filename, content_type, size_bytes, sha256, created_at
""",
(chat_id, actor["user_id"], filename, content_type, size, sha256_hex, object_key),
)
att = cur.fetchone()
except Exception as e:
# КОМПЕНСАЦИЯ: удаляем из S3, если БД упала
print(f"DB failed, compensating S3: {e}")
try:
delete_file(object_key)
except Exception as s3_e:
print(f"Failed to compensate S3: {s3_e}")
raise ApiError(500, "internal_error", "Ошибка сохранения")
return {
"id": str(att["id"]),
"chat_id": str(att["chat_id"]),
"owner_id": str(att["owner_id"]),
"filename": att["filename"],
"content_type": att["content_type"],
"size_bytes": att["size_bytes"],
"sha256": att["sha256"],
"status": "ready",
"variants": [],
"error_code": None,
"created_at": iso_z(att["created_at"]),
"updated_at": iso_z(att["created_at"]),
}
@router.get("/attachments/{attachment_id}")
def get_attachment(attachment_id: str, actor: dict = Depends(require_actor)):
try:
uuid.UUID(attachment_id)
except ValueError:
raise ApiError(400, "invalid_request", "Неверный id")
with connection() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(
"""
SELECT a.*
FROM attachments a
JOIN chat_members cm ON cm.chat_id = a.chat_id AND cm.user_id = %s
WHERE a.id = %s
""",
(actor["user_id"], attachment_id),
)
att = cur.fetchone()
if att is None:
raise ApiError(404, "not_found", "Вложение не найдено")
return {
"id": str(att["id"]),
"chat_id": str(att["chat_id"]),
"owner_id": str(att["owner_id"]),
"filename": att["filename"],
"content_type": att["content_type"],
"size_bytes": att["size_bytes"],
"sha256": att["sha256"],
"status": "ready",
"variants": [],
"error_code": None,
"created_at": iso_z(att["created_at"]),
"updated_at": iso_z(att["created_at"]),
}
@router.get("/attachments/{attachment_id}/content")
def download_attachment(attachment_id: str, actor: dict = Depends(require_actor)):
try:
uuid.UUID(attachment_id)
except ValueError:
raise ApiError(400, "invalid_request", "Неверный id")
with connection() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(
"""
SELECT a.object_key, a.content_type, a.filename
FROM attachments a
JOIN chat_members cm ON cm.chat_id = a.chat_id AND cm.user_id = %s
WHERE a.id = %s
""",
(actor["user_id"], attachment_id),
)
att = cur.fetchone()
if att is None:
raise ApiError(404, "not_found", "Вложение не найдено")
try:
body = download_fileobj(att["object_key"])
except Exception:
raise ApiError(503, "unavailable", "S3 недоступен")
def iter_body():
yield from body.iter_chunks(1024 * 1024)
return StreamingResponse(
iter_body(),
media_type=att["content_type"],
headers={"Content-Disposition": f'attachment; filename="{att["filename"]}"'},
)
+169
View File
@@ -0,0 +1,169 @@
import base64
import uuid
from datetime import datetime
from typing import Optional
import psycopg
from fastapi import APIRouter, Depends, Query
from pydantic import BaseModel, ConfigDict, Field, field_validator
from psycopg.rows import dict_row
from ..db import connection, transaction
from ..errors import ApiError
from ..serializers import row_to_chat
from .sessions import require_actor
router = APIRouter(prefix="/api/v1")
class CreateChat(BaseModel):
model_config = ConfigDict(extra="forbid")
title: str = Field(min_length=1, max_length=100)
member_ids: list[str] = Field(min_length=1, max_length=100)
@field_validator("title")
@classmethod
def title_not_blank(cls, v: str) -> str:
if not v.strip():
raise ValueError("title must not be blank")
return v
@field_validator("member_ids")
@classmethod
def unique_members(cls, v: list[str]) -> list[str]:
if len(set(v)) != len(v):
raise ValueError("duplicate member_ids")
for s in v:
try:
uuid.UUID(s)
except ValueError:
raise ValueError("invalid uuid in member_ids")
return v
def _encode_cursor(created_at: datetime, id_) -> str:
raw = f"{created_at.isoformat()}|{id_}"
return base64.urlsafe_b64encode(raw.encode("utf-8")).decode("ascii").rstrip("=")
def _decode_cursor(cursor: str):
pad = "=" * (-len(cursor) % 4)
raw = base64.urlsafe_b64decode(cursor + pad).decode("utf-8")
ts, id_ = raw.split("|", 1)
return datetime.fromisoformat(ts), id_
@router.post("/chats", status_code=201)
def create_chat(body: CreateChat, actor: dict = Depends(require_actor)):
me = actor["user_id"]
if str(me) not in body.member_ids:
raise ApiError(400, "invalid_request", "Вызывающий должен входить в member_ids")
try:
with transaction() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(
"SELECT id FROM users WHERE id = ANY(%s::uuid[])",
(body.member_ids,),
)
found = {str(r["id"]) for r in cur.fetchall()}
missing = set(body.member_ids) - found
if missing:
raise ApiError(400, "invalid_request", "Неизвестные участники")
cur.execute(
"INSERT INTO chats (title) VALUES (%s) RETURNING id, title, created_at",
(body.title,),
)
chat = cur.fetchone()
cur.executemany(
"INSERT INTO chat_members (chat_id, user_id) VALUES (%s, %s)",
[(chat["id"], m) for m in body.member_ids],
)
except ApiError:
raise
except psycopg.errors.ForeignKeyViolation:
raise ApiError(400, "invalid_request", "Неизвестные участники")
chat["member_ids"] = body.member_ids
return row_to_chat(chat)
@router.get("/chats")
def list_chats(
limit: int = Query(default=50, ge=1, le=100),
cursor: Optional[str] = Query(default=None),
actor: dict = Depends(require_actor),
):
params: list = []
where = ["cm.user_id = %s"]
params.append(actor["user_id"])
if cursor:
try:
c_ts, c_id = _decode_cursor(cursor)
except Exception:
raise ApiError(400, "invalid_request", "Неверный cursor")
where.append("(c.created_at, c.id) > (%s, %s)")
params.extend([c_ts, c_id])
sql = f"""
SELECT c.id, c.title, c.created_at,
ARRAY(
SELECT cm2.user_id::text
FROM chat_members cm2
WHERE cm2.chat_id = c.id
ORDER BY cm2.user_id
) AS member_ids
FROM chats c
JOIN chat_members cm ON cm.chat_id = c.id
WHERE {' AND '.join(where)}
ORDER BY c.created_at, c.id
LIMIT %s
"""
params.append(limit + 1)
with connection() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(sql, params)
rows = cur.fetchall()
next_cursor = None
if len(rows) > limit:
last = rows[limit - 1]
next_cursor = _encode_cursor(last["created_at"], str(last["id"]))
rows = rows[:limit]
return {"items": [row_to_chat(r) for r in rows], "next_cursor": next_cursor}
@router.get("/chats/{chat_id}")
def get_chat(chat_id: str, actor: dict = Depends(require_actor)):
try:
uuid.UUID(chat_id)
except ValueError:
raise ApiError(400, "invalid_request", "Неверный chat_id")
with connection() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(
"""
SELECT c.id, c.title, c.created_at,
ARRAY(
SELECT cm2.user_id::text
FROM chat_members cm2
WHERE cm2.chat_id = c.id
ORDER BY cm2.user_id
) AS member_ids
FROM chats c
JOIN chat_members cm ON cm.chat_id = c.id
WHERE c.id = %s AND cm.user_id = %s
""",
(chat_id, actor["user_id"]),
)
row = cur.fetchone()
if row is None:
raise ApiError(404, "not_found", "Чат не найден")
return row_to_chat(row)
+26
View File
@@ -0,0 +1,26 @@
from fastapi import APIRouter, Request
from fastapi.responses import JSONResponse
from ..db import connection
from ..errors import error_response
router = APIRouter()
@router.get("/health/live")
def live():
return {"status": "ok"}
@router.get("/health/ready")
def ready(request: Request):
try:
with connection() as conn:
with conn.cursor() as cur:
cur.execute("SELECT 1")
cur.fetchone()
except Exception:
return error_response(
request, 503, "unavailable", "Зависимость недоступна"
)
return {"status": "ok"}
+171
View File
@@ -0,0 +1,171 @@
import re
import uuid
from datetime import datetime
from typing import Optional
from fastapi import APIRouter, Depends, Header, Query
from pydantic import BaseModel, ConfigDict, Field
from psycopg.rows import dict_row
from ..db import connection
from ..errors import ApiError
from ..idempotency import run_idempotent
from ..serializers import row_to_message
from .sessions import require_actor
router = APIRouter(prefix="/api/v1")
KEY_RE = re.compile(r"^[A-Za-z0-9._:-]{1,128}$")
class CreateMessage(BaseModel):
model_config = ConfigDict(extra="forbid")
text: str = Field(max_length=4000)
attachment_ids: list[str] = Field(default_factory=list, max_length=10)
def _check_chat_member(conn, chat_id: str, user_id: str) -> None:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id, user_id),
)
if cur.fetchone() is None:
raise ApiError(404, "not_found", "Чат не найден")
def _encode_cursor(created_at: datetime, id_) -> str:
import base64
raw = f"{created_at.isoformat()}|{id_}"
return base64.urlsafe_b64encode(raw.encode("utf-8")).decode("ascii").rstrip("=")
def _decode_cursor(cursor: str):
import base64
pad = "=" * (-len(cursor) % 4)
raw = base64.urlsafe_b64decode(cursor + pad).decode("utf-8")
ts, id_ = raw.split("|", 1)
return datetime.fromisoformat(ts), id_
@router.post("/chats/{chat_id}/messages", status_code=201)
def create_message(
chat_id: str,
body: CreateMessage,
idempotency_key: str = Header(..., alias="Idempotency-Key"),
actor: dict = Depends(require_actor),
):
try:
uuid.UUID(chat_id)
except ValueError:
raise ApiError(400, "invalid_request", "Неверный chat_id")
if not KEY_RE.fullmatch(idempotency_key or ""):
raise ApiError(400, "invalid_request", "Неверный Idempotency-Key")
if not body.text.strip() and not body.attachment_ids:
raise ApiError(400, "invalid_request", "Нужен текст или вложение")
scope = f"POST /api/v1/chats/{chat_id}/messages"
def action(conn):
_check_chat_member(conn, chat_id, actor["user_id"])
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(
"""
INSERT INTO messages (chat_id, sender_id, text)
VALUES (%s, %s, %s)
RETURNING id, chat_id, sender_id, text, created_at
""",
(chat_id, actor["user_id"], body.text),
)
row = cur.fetchone()
if body.attachment_ids:
for att_id in body.attachment_ids:
cur.execute(
"SELECT 1 FROM attachments WHERE id = %s AND chat_id = %s",
(att_id, chat_id),
)
if cur.fetchone() is None:
raise ApiError(404, "not_found", "Вложение не найдено")
cur.executemany(
"INSERT INTO message_attachments (message_id, attachment_id) VALUES (%s, %s)",
[(row["id"], att_id) for att_id in body.attachment_ids]
)
row["attachment_ids"] = body.attachment_ids
return 201, row_to_message(row)
_, payload = run_idempotent(
user_id=actor["user_id"],
scope=scope,
key=idempotency_key,
body=body.model_dump(),
action=action,
)
return payload
@router.get("/chats/{chat_id}/messages")
def list_messages(
chat_id: str,
limit: int = Query(default=50, ge=1, le=100),
cursor: Optional[str] = Query(default=None),
actor: dict = Depends(require_actor),
):
try:
uuid.UUID(chat_id)
except ValueError:
raise ApiError(400, "invalid_request", "Неверный chat_id")
with connection() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
(chat_id, actor["user_id"]),
)
if cur.fetchone() is None:
raise ApiError(404, "not_found", "Чат не найден")
params: list = [chat_id]
where = ["m.chat_id = %s"]
if cursor:
try:
c_ts, c_id = _decode_cursor(cursor)
except Exception:
raise ApiError(400, "invalid_request", "Неверный cursor")
where.append("(m.created_at, m.id) > (%s, %s)")
params.extend([c_ts, c_id])
sql = f"""
SELECT m.id, m.chat_id, m.sender_id, m.text, m.created_at,
COALESCE(
ARRAY(
SELECT ma.attachment_id::text
FROM message_attachments ma
WHERE ma.message_id = m.id
), ARRAY[]::text[]
) AS attachment_ids
FROM messages m
WHERE {' AND '.join(where)}
ORDER BY m.created_at, m.id
LIMIT %s
"""
params.append(limit + 1)
with connection() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(sql, params)
rows = cur.fetchall()
next_cursor = None
if len(rows) > limit:
last = rows[limit - 1]
next_cursor = _encode_cursor(last["created_at"], str(last["id"]))
rows = rows[:limit]
return {"items": [row_to_message(r) for r in rows], "next_cursor": next_cursor}
+131
View File
@@ -0,0 +1,131 @@
import base64
from fastapi import APIRouter, Depends, Request
from fastapi.responses import Response
from psycopg.rows import dict_row
from ..config import get_settings
from ..db import connection, transaction
from ..errors import ApiError
from ..security import (
new_session_token,
session_expiry,
token_digest,
verify_password,
)
from ..serializers import row_to_user
router = APIRouter(prefix="/api/v1")
WWW_BASIC = 'Basic realm="messenger", charset="UTF-8"'
WWW_BEARER = 'Bearer realm="messenger"'
def _parse_basic(header: str | None) -> tuple[str, str]:
if not header or not header.lower().startswith("basic "):
raise ApiError(401, "unauthorized", "Требуется Basic", {"WWW-Authenticate": WWW_BASIC})
try:
decoded = base64.b64decode(header[6:].strip(), validate=True).decode("utf-8")
except Exception:
raise ApiError(401, "unauthorized", "Неверные учётные данные", {"WWW-Authenticate": WWW_BASIC})
if ":" not in decoded:
raise ApiError(401, "unauthorized", "Неверные учётные данные", {"WWW-Authenticate": WWW_BASIC})
username, password = decoded.split(":", 1)
return username, password
@router.post("/auth/sessions", status_code=201)
def create_session(request: Request):
header = request.headers.get("Authorization")
username, password = _parse_basic(header)
with connection() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(
"SELECT id, username, display_name, created_at, password_hash "
"FROM users WHERE username = %s",
(username,),
)
row = cur.fetchone()
# Одинаковый 401 для «нет пользователя» и «неверный пароль».
if row is None or not verify_password(password, row["password_hash"]):
raise ApiError(401, "unauthorized", "Неверные учётные данные", {"WWW-Authenticate": WWW_BASIC})
settings = get_settings()
token = new_session_token()
expires_at = session_expiry(settings.session_ttl_seconds)
with transaction() as conn:
with conn.cursor() as cur:
cur.execute(
"""
INSERT INTO sessions (user_id, token_hash, expires_at)
VALUES (%s, %s, %s)
""",
(row["id"], token_digest(token), expires_at),
)
return {
"token": token,
"token_type": "Bearer",
"expires_at": expires_at.isoformat().replace("+00:00", "Z"),
"user": row_to_user(row),
}
def _extract_bearer(request: Request) -> str:
header = request.headers.get("Authorization")
if not header or not header.lower().startswith("bearer "):
raise ApiError(401, "unauthorized", "Требуется Bearer", {"WWW-Authenticate": WWW_BEARER})
token = header[7:].strip()
if not token:
raise ApiError(401, "unauthorized", "Требуется Bearer", {"WWW-Authenticate": WWW_BEARER})
return token
def require_actor(request: Request) -> dict:
"""Зависимость: валидирует Bearer и возвращает actor с user и token_hash."""
token = _extract_bearer(request)
digest = token_digest(token)
with connection() as conn:
with conn.cursor(row_factory=dict_row) as cur:
cur.execute(
"""
SELECT s.user_id, s.expires_at,
u.id, u.username, u.display_name, u.created_at
FROM sessions s
JOIN users u ON u.id = s.user_id
WHERE s.token_hash = %s
""",
(digest,),
)
row = cur.fetchone()
if row is None:
raise ApiError(401, "unauthorized", "Сессия недействительна", {"WWW-Authenticate": WWW_BEARER})
# Проверка срока на стороне БД
with connection() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT 1 FROM sessions WHERE token_hash = %s AND expires_at > now()",
(digest,),
)
if cur.fetchone() is None:
raise ApiError(401, "unauthorized", "Сессия истекла", {"WWW-Authenticate": WWW_BEARER})
return {
"user": row_to_user(row),
"user_id": row["user_id"],
"token_hash": digest,
}
@router.delete("/auth/sessions/current", status_code=204)
def revoke_session(request: Request, actor: dict = Depends(require_actor)):
with transaction() as conn:
with conn.cursor() as cur:
cur.execute("DELETE FROM sessions WHERE token_hash = %s", (actor["token_hash"],))
return Response(status_code=204)
+52
View File
@@ -0,0 +1,52 @@
from fastapi import APIRouter, Depends
from pydantic import BaseModel, ConfigDict, Field, field_validator
import psycopg
from ..db import transaction
from ..errors import ApiError
from ..security import hash_password
from ..serializers import row_to_user
from .sessions import require_actor
router = APIRouter(prefix="/api/v1")
class CreateUser(BaseModel):
model_config = ConfigDict(extra="forbid")
username: str = Field(pattern=r"^[a-z0-9_]{3,32}$")
display_name: str = Field(min_length=1, max_length=100)
password: str = Field(min_length=12, max_length=128)
@field_validator("display_name")
@classmethod
def not_blank(cls, v: str) -> str:
if not v.strip():
raise ValueError("display_name must not be blank")
return v
@router.post("/users", status_code=201)
def create_user(body: CreateUser):
pwd_hash = hash_password(body.password)
try:
with transaction() as conn:
with conn.cursor(row_factory=psycopg.rows.dict_row) as cur:
cur.execute(
"""
INSERT INTO users (username, display_name, password_hash)
VALUES (%s, %s, %s)
RETURNING id, username, display_name, created_at
""",
(body.username, body.display_name, pwd_hash),
)
row = cur.fetchone()
except psycopg.errors.UniqueViolation:
raise ApiError(409, "conflict", "Имя пользователя занято")
return row_to_user(row)
@router.get("/users/me")
def current_user(actor: dict = Depends(require_actor)):
return actor["user"]
+69
View File
@@ -0,0 +1,69 @@
# import boto3
# from botocore.config import Config
# from .config import get_settings
# def get_s3_client():
# s = get_settings()
# return boto3.client(
# "s3",
# endpoint_url=s.s3_endpoint,
# aws_access_key_id=s.s3_access_key,
# aws_secret_access_key=s.s3_secret_key,
# region_name=s.s3_region,
# config=Config(signature_version="s3v4"),
# )
# def upload_fileobj(fileobj, key: str, content_type: str):
# client = get_s3_client()
# s = get_settings()
# client.upload_fileobj(
# fileobj,
# s.s3_bucket,
# key,
# ExtraArgs={"ContentType": content_type},
# )
# def download_fileobj(key: str):
# client = get_s3_client()
# s = get_settings()
# obj = client.get_object(Bucket=s.s3_bucket, Key=key)
# return obj["Body"]
import boto3
from botocore.config import Config
from .config import get_settings
def get_s3_client():
s = get_settings()
return boto3.client(
"s3",
endpoint_url=s.s3_endpoint,
aws_access_key_id=s.s3_access_key,
aws_secret_access_key=s.s3_secret_key,
region_name=s.s3_region,
config=Config(signature_version="s3v4"),
)
def upload_fileobj(fileobj, key: str, content_type: str):
client = get_s3_client()
s = get_settings()
client.upload_fileobj(
fileobj,
s.s3_bucket,
key,
ExtraArgs={"ContentType": content_type},
)
def download_fileobj(key: str):
client = get_s3_client()
s = get_settings()
obj = client.get_object(Bucket=s.s3_bucket, Key=key)
return obj["Body"]
def delete_file(key: str):
client = get_s3_client()
s = get_settings()
client.delete_object(Bucket=s.s3_bucket, Key=key)
+34
View File
@@ -0,0 +1,34 @@
import hashlib
import secrets
from datetime import datetime, timedelta, timezone
from argon2 import PasswordHasher
from argon2.exceptions import VerifyMismatchError
_ph = PasswordHasher() # Argon2id по умолчанию
def hash_password(password: str) -> str:
return _ph.hash(password)
def verify_password(password: str, password_hash: str) -> bool:
try:
return _ph.verify(password_hash, password)
except VerifyMismatchError:
return False
except Exception:
return False
def new_session_token() -> str:
# 32 байта = 256 бит энтропии; url-safe hex без паддинга.
return secrets.token_urlsafe(32)
def token_digest(token: str) -> str:
return hashlib.sha256(token.encode("utf-8")).hexdigest()
def session_expiry(ttl_seconds: int) -> datetime:
return datetime.now(timezone.utc) + timedelta(seconds=ttl_seconds)
+35
View File
@@ -0,0 +1,35 @@
from datetime import datetime
def iso_z(dt: datetime) -> str:
"""RFC3339 UTC с 'Z'. Postgres отдаёт timestamptz с '+00:00'."""
return dt.isoformat().replace("+00:00", "Z")
def row_to_user(row) -> dict:
return {
"id": str(row["id"]),
"username": row["username"],
"display_name": row["display_name"],
"created_at": iso_z(row["created_at"]),
}
def row_to_chat(row) -> dict:
return {
"id": str(row["id"]),
"title": row["title"],
"member_ids": row["member_ids"],
"created_at": iso_z(row["created_at"]),
}
def row_to_message(row) -> dict:
return {
"id": str(row["id"]),
"chat_id": str(row["chat_id"]),
"sender_id": str(row["sender_id"]),
"text": row["text"],
"created_at": iso_z(row["created_at"]),
"attachment_ids": row.get("attachment_ids", []),
}
+52
View File
@@ -0,0 +1,52 @@
-- 001_init.sql
CREATE EXTENSION IF NOT EXISTS pgcrypto;
CREATE TABLE IF NOT EXISTS users (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
username text NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
display_name text NOT NULL CHECK (char_length(display_name) BETWEEN 1 AND 100),
password_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS sessions (
token_hash text PRIMARY KEY,
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
expires_at timestamptz NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS sessions_user_id_idx ON sessions(user_id);
CREATE INDEX IF NOT EXISTS sessions_expires_at_idx ON sessions(expires_at);
CREATE TABLE IF NOT EXISTS chats (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
title text NOT NULL CHECK (char_length(title) BETWEEN 1 AND 100),
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS chat_members (
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
PRIMARY KEY (chat_id, user_id)
);
CREATE INDEX IF NOT EXISTS chat_members_user_idx ON chat_members(user_id);
CREATE TABLE IF NOT EXISTS messages (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
sender_id uuid NOT NULL REFERENCES users(id),
text text NOT NULL DEFAULT '',
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS messages_chat_order_idx ON messages(chat_id, created_at, id);
CREATE TABLE IF NOT EXISTS idempotency_keys (
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
scope text NOT NULL, -- method + " " + path
key text NOT NULL,
request_hash text NOT NULL,
status_code int NOT NULL,
response_json jsonb NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (user_id, scope, key)
);
+20
View File
@@ -0,0 +1,20 @@
-- 002_attachments.sql
CREATE TABLE IF NOT EXISTS attachments (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
owner_id uuid NOT NULL REFERENCES users(id),
filename text NOT NULL,
content_type text NOT NULL,
size_bytes bigint NOT NULL,
sha256 text NOT NULL,
object_key text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS attachments_chat_idx ON attachments(chat_id);
CREATE TABLE IF NOT EXISTS message_attachments (
message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
attachment_id uuid NOT NULL REFERENCES attachments(id) ON DELETE CASCADE,
PRIMARY KEY (message_id, attachment_id)
);
+113
View File
@@ -0,0 +1,113 @@
import base64
import io
import json
import os
import random
import urllib.error
import urllib.request
import uuid
from concurrent.futures import ThreadPoolExecutor
BASE_URL = "http://localhost:8080"
def req(method, path, data=None, headers=None, raw=None):
r = urllib.request.Request(BASE_URL + path, method=method)
if headers:
for k, v in headers.items():
r.add_header(k, v)
if data:
r.add_header("Content-Type", "application/json")
r.data = json.dumps(data).encode("utf-8")
if raw:
r.data = raw
try:
with urllib.request.urlopen(r) as resp:
return resp.status, resp.read()
except urllib.error.HTTPError as e:
return e.code, e.read()
def setup():
u = "test4_" + str(random.randint(1000, 9999))
_, body = req("POST", "/api/v1/users", {"username": u, "display_name": "Тест", "password": "testpassword1234"})
user = json.loads(body)
cred = base64.b64encode((u + ":testpassword1234").encode()).decode()
_, body = req("POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic " + cred})
token = json.loads(body)["token"]
headers = {"Authorization": "Bearer " + token}
_, body = req("POST", "/api/v1/chats", {"title": "Тестовый чат", "member_ids": [user["id"]]}, headers)
chat = json.loads(body)
return user, chat, headers
def main():
user, chat, headers = setup()
print(f"Пользователь {user['id']} и чат {chat['id']} созданы.")
chat_url = f"/api/v1/chats/{chat['id']}/attachments"
# 1. Слишком большой файл (11 МБ)
print("1. Тест: слишком большой файл...")
big_data = b"\x89PNG\r\n\x1a\n" + b"A" * (11 * 1024 * 1024)
status, _ = req("POST", chat_url + "?filename=big.png", headers={**headers, "Content-Type": "image/png"}, raw=big_data)
assert status == 413, f"Ожидали 413, получили {status}"
print(" OK (413)")
# 2. Битый файл (MIME PNG, но внутри мусор)
print("2. Тест: битый файл...")
broken_data = b"\x89PNG\r\n\x1a\n" + b"broken data"
status, _ = req("POST", chat_url + "?filename=broken.png", headers={**headers, "Content-Type": "image/png"}, raw=broken_data)
assert status == 415, f"Ожидали 415, получили {status}"
print(" OK (415)")
# 3. Подмена MIME (говорим image/png, а внутри текст)
print("3. Тест: подмена MIME...")
text_data = b"Hello, this is not an image!"
status, _ = req("POST", chat_url + "?filename=fake.png", headers={**headers, "Content-Type": "image/png"}, raw=text_data)
assert status == 415, f"Ожидали 415, получили {status}"
print(" OK (415)")
# 4. Опасное имя файла (path traversal)
print("4. Тест: опасное имя файла...")
with open("tests/fixtures/sample.png", "rb") as f:
valid_png = f.read()
status, _ = req("POST", chat_url + "?filename=../../etc/passwd", headers={**headers, "Content-Type": "image/png"}, raw=valid_png)
assert status == 400, f"Ожидали 400, получили {status}"
print(" OK (400)")
# 5. Успешная загрузка (чтобы проверить доступ)
print("5. Тест: успешная загрузка...")
status, body = req("POST", chat_url + "?filename=ok.png", headers={**headers, "Content-Type": "image/png"}, raw=valid_png)
assert status == 201, f"Ожидали 201, получили {status}"
att = json.loads(body)
print(f" OK (201), ID: {att['id']}")
# 6. Чужой доступ
print("6. Тест: чужой доступ...")
u_eve = "eve_" + str(random.randint(1000, 9999))
req("POST", "/api/v1/users", {"username": u_eve, "display_name": "Eve", "password": "testpassword1234"})
cred_eve = base64.b64encode((u_eve + ":testpassword1234").encode()).decode()
_, body = req("POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic " + cred_eve})
token_eve = json.loads(body)["token"]
headers_eve = {"Authorization": "Bearer " + token_eve}
status, _ = req("GET", f"/api/v1/attachments/{att['id']}", headers=headers_eve)
assert status == 404, f"Ожидали 404 (чужой), получили {status}"
print(" OK (404)")
# 7. Нагрузочный тест (10 параллельных загрузок)
print("7. Тест: 10 параллельных загрузок (проверка RAM)...")
def upload_worker():
status, _ = req("POST", chat_url + "?filename=parallel.png", headers={**headers, "Content-Type": "image/png"}, raw=valid_png)
return status
with ThreadPoolExecutor(max_workers=10) as pool:
results = list(pool.map(lambda _: upload_worker(), range(10)))
assert all(s == 201 for s in results), f"Не все загрузки успешны: {results}"
print(" OK (все 10 успешно)")
print("\nPASS: Уровень 4 пройден! (Все негативные тесты и нагрузка работают)")
if __name__ == "__main__":
main()
+31
View File
@@ -0,0 +1,31 @@
import urllib.request, json, base64, random
BASE = "http://localhost:8080"
def req(path, method="GET", data=None, headers=None):
r = urllib.request.Request(BASE + path, method=method)
if headers:
for k, v in headers.items(): r.add_header(k, v)
if data:
r.add_header("Content-Type", "application/json")
r.data = json.dumps(data).encode("utf-8")
try:
with urllib.request.urlopen(r) as resp:
return resp.status, json.loads(resp.read())
except urllib.error.HTTPError as e:
return e.code, json.loads(e.read())
u = "test_" + str(random.randint(1000, 9999))
print("Создаём пользователя:", u)
s, user = req("/api/v1/users", "POST", {"username": u, "display_name": "Тест", "password": "testpassword1234"})
print("Пользователь:", s, user)
print("Логинимся...")
cred = base64.b64encode((u + ":testpassword1234").encode()).decode()
s, sess = req("/api/v1/auth/sessions", "POST", headers={"Authorization": "Basic " + cred})
print("Сессия:", s)
token = sess["token"]
print("Создаём чат...")
s, chat = req("/api/v1/chats", "POST", {"title": "Тест", "member_ids": [user["id"]]}, {"Authorization": "Bearer " + token})
print("Чат:", s, chat)