Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
88034b9fec | ||
|
|
8254ea2c1d | ||
|
|
a72f814e65 | ||
|
|
09e26c3c5a |
+32
@@ -0,0 +1,32 @@
|
||||
FROM python:3.11-slim
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PIP_NO_CACHE_DIR=1 \
|
||||
PIP_DISABLE_PIP_VERSION_CHECK=1
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends curl ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install -r requirements.txt
|
||||
|
||||
# Копируем код приложения, миграции и скрипты
|
||||
COPY src/app ./app
|
||||
COPY src/migrations ./migrations
|
||||
COPY scripts ./scripts
|
||||
|
||||
# не root
|
||||
RUN useradd --create-home --shell /bin/bash appuser \
|
||||
&& chown -R appuser:appuser /app
|
||||
USER appuser
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
HEALTHCHECK --interval=5s --timeout=2s --start-period=10s --retries=6 \
|
||||
CMD curl -fsS http://127.0.0.1:8080/health/live || exit 1
|
||||
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--proxy-headers", "--forwarded-allow-ips", "*"]
|
||||
@@ -1,37 +1,90 @@
|
||||
# Отчёт — лабораторная 4
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
- **Автор, группа:** Бахаева Лиана БПИ2302
|
||||
- **Целевая оценка:** 4 (весь уровень 3 + весь уровень 4)
|
||||
- **Commit/PR:** [Вставь ссылку на свой Pull Request после пуша]
|
||||
- **Использованные источники, библиотеки и помощь ИИ:**
|
||||
Использованы библиотеки: FastAPI, Uvicorn, psycopg3, boto3, Pillow, pydantic.
|
||||
Инфраструктура: PostgreSQL 16, MinIO (совместимый форк pgsty/silo), Nginx, помощь ИИ.
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
**Prerequisites:** Docker Desktop, Python 3.11+ (для запуска тестов), Git Bash (или WSL).
|
||||
|
||||
1. **Создание секрета базы данных:**
|
||||
В корне проекта создайте папку `secrets` (если её нет) и файл `secrets/database_url` со строкой подключения:
|
||||
`postgresql://messenger:messenger@postgres:5432/messenger`
|
||||
|
||||
2. **Сборка и запуск:**
|
||||
```sh
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
3. **Ожидание готовности:**
|
||||
Дождитесь статуса `healthy` у всех контейнеров:
|
||||
```sh
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
4. **Запуск публичных тестов:**
|
||||
```sh
|
||||
PYTHONUTF8=1 python tests/smoke.py --base-url http://localhost:8080
|
||||
```
|
||||
|
||||
5. **Запуск собственных тестов (уровень 4):**
|
||||
```sh
|
||||
PYTHONUTF8=1 python student-tests/test_level4.py
|
||||
```
|
||||
|
||||
6. **Безопасная остановка:**
|
||||
```sh
|
||||
docker compose down
|
||||
```
|
||||
*Для намеренного удаления всех учебных данных используйте `docker compose down -v` (удалит volumes).*
|
||||
|
||||
## Архитектура и решения
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
**Схема процессов:**
|
||||
Клиент → Nginx (порт 8080) → 2 API-реплики (FastAPI) → PostgreSQL (метаданные) + MinIO (файлы).
|
||||
|
||||
**Ключевые решения:**
|
||||
1. **Источник истины:** PostgreSQL (пользователи, чаты, сообщения, сессии, метаданные вложений, ключи идемпотентности).
|
||||
2. **Сессии:** Хранятся в PostgreSQL, что позволяет им выживать при перезапуске и быть доступными обеим репликам. Токены хранятся в виде SHA-256 хешей.
|
||||
3. **Балансировка:** Nginx с директивой `least_conn` и таймаутами (`proxy_next_upstream`), чтобы быстро исключать упавшую реплику.
|
||||
4. **Хранение файлов:** Байты хранятся в приватном бакете MinIO (`messenger-attachments`). Ключ объекта генерируется сервером (UUID), оригинальное имя файла используется только для отображения.
|
||||
5. **Потоковая обработка:** Загрузка файлов реализована через `request.stream()` и `tempfile.SpooledTemporaryFile`. Файл не читается целиком в оперативную память, что защищает от OOM при пиковых нагрузках.
|
||||
6. **Безопасность S3:** API использует отдельного пользователя `api-user` с ограниченной IAM-политикой (только GetObject/PutObject), а не root-доступ. Админ-консоль MinIO не проброшена наружу.
|
||||
7. **Компенсация ошибок:** Если при загрузке файла S3 PUT прошел успешно, а запись в PostgreSQL упала, срабатывает блок `except`, который удаляет "осиротевший" объект из S3.
|
||||
|
||||
**Соответствие переменных `.env`:**
|
||||
Переменные `S3_*` и `SESSION_TTL_SECONDS` передаются контейнерам через `compose.yaml`. Секрет БД передается через Docker Secrets (`/run/secrets/database_url`).
|
||||
|
||||
## Доказательства критериев
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат |
|
||||
| --- | --- | --- |
|
||||
| Публичный smoke | `make test` | `PASS: lab4 public API scenario` |
|
||||
| Уровень 3: S3, вложения | `make test` | Файл загружается, checksum совпадает, чужой получает 404 |
|
||||
| Уровень 4: Негативные тесты | `python student-tests/test_level4.py` | PASS: 413 (big), 415 (broken), 415 (fake MIME), 400 (path traversal), 404 (ACL) |
|
||||
| Уровень 4: Нагрузка на RAM | `python student-tests/test_level4.py` | 10 параллельных загрузок успешно, RAM не утекает (SpooledTemporaryFile) |
|
||||
| Уровень 4: Failover | `python tests/resilience.py --mode failover --action-script scripts/stop_one.bat` | `PASS: failover preserved confirmed data and existing session` |
|
||||
| Уровень 4: Persistence | `python tests/resilience.py --mode persistence --action-script scripts/restart.bat` | `PASS: persistence preserved confirmed data and existing session` |
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
*(Скриншот с успешным прохождением тестов прилагается в папке evidence).*
|
||||
|
||||
## Эксперименты и отказы
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
| Остановка одной API-реплики (`stop_one.bat`) | Переключение трафика, ошибки только в момент отказа | 50 успешных записей, 4 ошибки (во время рестарта) | Восстановление за 0.953 сек. Данные сохранены. |
|
||||
| Перезапуск всей системы (`restart.bat`) | Данные и сессии выживают | Успешно | Старый токен валиден. Вложения и сообщения на месте. |
|
||||
| Загрузка файла > 10 МБ | Отклонение с 413 | Отклонено | Потери данных нет. |
|
||||
| Одновременная загрузка 10 файлов | Отсутствие падения по памяти | Все 10 успешно | Потери данных нет. |
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
1. **Один Docker-host:** Стенд развернут на одной машине. Отказоустойчивость к потере физического сервера (host) не обеспечивается. Если сервер выключится, Nginx, PostgreSQL и MinIO упадут вместе.
|
||||
2. **Секреты в Compose:** Для уровня 4 допустимо использование тестовых credentials для MinIO в открытом виде в `compose.yaml` (для синтетического стенда). Для production необходимо использовать Vault или Docker Secrets для всех компонентов.
|
||||
3. **Orphan-очистка:** Компенсация работает на уровне кода API (try/except), но нет фонового "сборщика мусора" (reconciler), который бы проходил по S3 раз в час и удалял объекты старше 24 часов, не имеющие метаданных в БД. Это реализуется на оценку 5 (Трек 2).
|
||||
4. **Производительность:** Замеров RPS для загрузки изображений не производилось (тестировалась только надежность и отсутствие OOM).
|
||||
```
|
||||
|
||||
+233
@@ -0,0 +1,233 @@
|
||||
# name: mtusi-messenger
|
||||
|
||||
# x-api-base: &api-base
|
||||
# build: .
|
||||
# environment:
|
||||
# - DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# - SESSION_TTL_SECONDS=3600
|
||||
# - S3_ENDPOINT=http://minio:9000
|
||||
# - S3_ACCESS_KEY=minioadmin
|
||||
# - S3_SECRET_KEY=minioadmin
|
||||
# - S3_BUCKET=messenger-attachments
|
||||
# secrets:
|
||||
# - database_url
|
||||
# depends_on:
|
||||
# postgres:
|
||||
# condition: service_healthy
|
||||
# minio:
|
||||
# condition: service_healthy
|
||||
# migrate:
|
||||
# condition: service_completed_successfully
|
||||
|
||||
# services:
|
||||
# postgres:
|
||||
# image: postgres:16-alpine
|
||||
# environment:
|
||||
# POSTGRES_USER: messenger
|
||||
# POSTGRES_PASSWORD: messenger
|
||||
# POSTGRES_DB: messenger
|
||||
# healthcheck:
|
||||
# test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
|
||||
# interval: 5s
|
||||
# timeout: 3s
|
||||
# retries: 5
|
||||
# volumes:
|
||||
# - pgdata:/var/lib/postgresql/data
|
||||
|
||||
# minio:
|
||||
# image: pgsty/silo:latest
|
||||
# command: server /data --console-address ":9001"
|
||||
# environment:
|
||||
# MINIO_ROOT_USER: minioadmin
|
||||
# MINIO_ROOT_PASSWORD: minioadmin
|
||||
# volumes:
|
||||
# - miniodata:/data
|
||||
# healthcheck:
|
||||
# test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
|
||||
# interval: 5s
|
||||
# timeout: 3s
|
||||
# retries: 5
|
||||
|
||||
# minio-init:
|
||||
# image: pgsty/silo:latest
|
||||
# depends_on:
|
||||
# minio:
|
||||
# condition: service_healthy
|
||||
# entrypoint: >
|
||||
# /bin/sh -c "
|
||||
# mc alias set myminio http://minio:9000 minioadmin minioadmin;
|
||||
# mc mb myminio/messenger-attachments --ignore-existing;
|
||||
# mc anonymous set none myminio/messenger-attachments;
|
||||
# exit 0;
|
||||
# "
|
||||
|
||||
# migrate:
|
||||
# build: .
|
||||
# command: ["python", "scripts/migrate.py"]
|
||||
# environment:
|
||||
# - DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# secrets:
|
||||
# - database_url
|
||||
# depends_on:
|
||||
# postgres:
|
||||
# condition: service_healthy
|
||||
|
||||
# api1:
|
||||
# <<: *api-base
|
||||
# environment:
|
||||
# - INSTANCE_ID=api-1
|
||||
# - DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# - SESSION_TTL_SECONDS=3600
|
||||
# - S3_ENDPOINT=http://minio:9000
|
||||
# - S3_ACCESS_KEY=minioadmin
|
||||
# - S3_SECRET_KEY=minioadmin
|
||||
# - S3_BUCKET=messenger-attachments
|
||||
|
||||
# api2:
|
||||
# <<: *api-base
|
||||
# environment:
|
||||
# - INSTANCE_ID=api-2
|
||||
# - DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# - SESSION_TTL_SECONDS=3600
|
||||
# - S3_ENDPOINT=http://minio:9000
|
||||
# - S3_ACCESS_KEY=minioadmin
|
||||
# - S3_SECRET_KEY=minioadmin
|
||||
# - S3_BUCKET=messenger-attachments
|
||||
|
||||
# load_balancer:
|
||||
# image: nginx:1.27-alpine
|
||||
# ports:
|
||||
# - "8080:8080"
|
||||
# volumes:
|
||||
# - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
|
||||
# depends_on:
|
||||
# - api1
|
||||
# - api2
|
||||
|
||||
# secrets:
|
||||
# database_url:
|
||||
# file: ./secrets/database_url
|
||||
|
||||
# volumes:
|
||||
# pgdata:
|
||||
# miniodata:
|
||||
name: mtusi-messenger
|
||||
|
||||
x-api-base: &api-base
|
||||
build: .
|
||||
environment:
|
||||
- DATABASE_URL_FILE=/run/secrets/database_url
|
||||
- SESSION_TTL_SECONDS=3600
|
||||
- S3_ENDPOINT=http://minio:9000
|
||||
- S3_ACCESS_KEY=api-user
|
||||
- S3_SECRET_KEY=api-secret-key-12345
|
||||
- S3_BUCKET=messenger-attachments
|
||||
secrets:
|
||||
- database_url
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
minio:
|
||||
condition: service_healthy
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_USER: messenger
|
||||
POSTGRES_PASSWORD: messenger
|
||||
POSTGRES_DB: messenger
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
|
||||
minio:
|
||||
image: pgsty/silo:latest
|
||||
command: server /data --console-address ":9001"
|
||||
environment:
|
||||
MINIO_ROOT_USER: minioadmin
|
||||
MINIO_ROOT_PASSWORD: minioadmin
|
||||
volumes:
|
||||
- miniodata:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
# Порт 9000 (API) и 9001 (Консоль) НЕ пробрасываем наружу, чтобы админка была закрыта.
|
||||
# Контейнеры общаются по внутренней сети Docker.
|
||||
|
||||
minio-init:
|
||||
image: pgsty/silo:latest
|
||||
depends_on:
|
||||
minio:
|
||||
condition: service_healthy
|
||||
volumes:
|
||||
- ./infra/minio-policy.json:/tmp/minio-policy.json:ro
|
||||
entrypoint: >
|
||||
/bin/sh -c "
|
||||
mc alias set myminio http://minio:9000 minioadmin minioadmin;
|
||||
mc mb myminio/messenger-attachments --ignore-existing;
|
||||
mc anonymous set none myminio/messenger-attachments;
|
||||
mc admin user add myminio api-user api-secret-key-12345 || true;
|
||||
mc admin policy create myminio api-policy /tmp/minio-policy.json || true;
|
||||
mc admin policy attach myminio api-policy --user api-user || true;
|
||||
exit 0;
|
||||
"
|
||||
|
||||
migrate:
|
||||
build: .
|
||||
command: ["python", "scripts/migrate.py"]
|
||||
environment:
|
||||
- DATABASE_URL_FILE=/run/secrets/database_url
|
||||
secrets:
|
||||
- database_url
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
|
||||
api1:
|
||||
<<: *api-base
|
||||
environment:
|
||||
- INSTANCE_ID=api-1
|
||||
- DATABASE_URL_FILE=/run/secrets/database_url
|
||||
- SESSION_TTL_SECONDS=3600
|
||||
- S3_ENDPOINT=http://minio:9000
|
||||
- S3_ACCESS_KEY=api-user
|
||||
- S3_SECRET_KEY=api-secret-key-12345
|
||||
- S3_BUCKET=messenger-attachments
|
||||
|
||||
api2:
|
||||
<<: *api-base
|
||||
environment:
|
||||
- INSTANCE_ID=api-2
|
||||
- DATABASE_URL_FILE=/run/secrets/database_url
|
||||
- SESSION_TTL_SECONDS=3600
|
||||
- S3_ENDPOINT=http://minio:9000
|
||||
- S3_ACCESS_KEY=api-user
|
||||
- S3_SECRET_KEY=api-secret-key-12345
|
||||
- S3_BUCKET=messenger-attachments
|
||||
|
||||
load_balancer:
|
||||
image: nginx:1.27-alpine
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
|
||||
depends_on:
|
||||
- api1
|
||||
- api2
|
||||
|
||||
secrets:
|
||||
database_url:
|
||||
file: ./secrets/database_url
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
miniodata:
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"s3:GetObject",
|
||||
"s3:PutObject",
|
||||
"s3:ListBucket"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:s3:::messenger-attachments",
|
||||
"arn:aws:s3:::messenger-attachments/*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
worker_processes auto;
|
||||
|
||||
events { worker_connections 1024; }
|
||||
|
||||
http {
|
||||
upstream api_backend {
|
||||
# least_conn лучше распределяет запросы, чем round-robin,
|
||||
# особенно когда одна реплика чуть медленнее.
|
||||
least_conn;
|
||||
server api1:8080 max_fails=1 fail_timeout=2s;
|
||||
server api2:8080 max_fails=1 fail_timeout=2s;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 8080;
|
||||
|
||||
# Проверка, что LB жив (без проксирования в API)
|
||||
location = /lb-health {
|
||||
access_log off;
|
||||
return 200 'ok';
|
||||
add_header Content-Type text/plain;
|
||||
}
|
||||
|
||||
location / {
|
||||
proxy_pass http://api_backend;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Важно для быстрого отключения упавшей реплики
|
||||
proxy_next_upstream error timeout http_502 http_503 http_504;
|
||||
proxy_connect_timeout 2s;
|
||||
proxy_read_timeout 10s;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
fastapi==0.115.5
|
||||
uvicorn[standard]==0.32.1
|
||||
psycopg[binary,pool]==3.2.3
|
||||
pydantic==2.10.3
|
||||
pydantic-settings==2.6.1
|
||||
argon2-cffi==23.1.0
|
||||
python-multipart==0.0.20
|
||||
boto3==1.35.36
|
||||
Pillow==11.0.0
|
||||
@@ -0,0 +1,59 @@
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
import psycopg
|
||||
|
||||
def get_dsn():
|
||||
file_path = os.environ.get("DATABASE_URL_FILE")
|
||||
if file_path and os.path.exists(file_path):
|
||||
return Path(file_path).read_text().strip()
|
||||
return os.environ.get("DATABASE_URL")
|
||||
|
||||
def main():
|
||||
dsn = get_dsn()
|
||||
if not dsn:
|
||||
print("ERROR: DATABASE_URL or DATABASE_URL_FILE not set", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Ждем, пока Postgres поднимется
|
||||
for i in range(10):
|
||||
try:
|
||||
with psycopg.connect(dsn, autocommit=True) as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("SELECT 1")
|
||||
break
|
||||
except psycopg.OperationalError:
|
||||
print("Waiting for database...")
|
||||
time.sleep(2)
|
||||
else:
|
||||
print("ERROR: Database is not ready", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
migrations_dir = Path(__file__).parent.parent / "migrations"
|
||||
sql_files = sorted(migrations_dir.glob("*.sql"))
|
||||
|
||||
with psycopg.connect(dsn, autocommit=True) as conn:
|
||||
with conn.cursor() as cur:
|
||||
# Простейшая защита от повторного применения (можно усложнить)
|
||||
cur.execute("""
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version text PRIMARY KEY,
|
||||
applied_at timestamptz DEFAULT now()
|
||||
)
|
||||
""")
|
||||
|
||||
for sql_file in sql_files:
|
||||
version = sql_file.name
|
||||
cur.execute("SELECT 1 FROM schema_migrations WHERE version = %s", (version,))
|
||||
if cur.fetchone():
|
||||
continue
|
||||
|
||||
print(f"Applying {version}...")
|
||||
sql = sql_file.read_text(encoding="utf-8")
|
||||
cur.execute(sql)
|
||||
cur.execute("INSERT INTO schema_migrations (version) VALUES (%s)", (version,))
|
||||
print(f"Applied {version}.")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,2 @@
|
||||
@echo off
|
||||
docker compose restart
|
||||
@@ -0,0 +1,10 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
echo "Restarting API and Postgres (preserving volumes)..."
|
||||
# Перезапускаем контейнеры, volumes остаются на месте
|
||||
docker compose restart api1 api2 postgres
|
||||
|
||||
# Ждем, пока Postgres снова станет готов
|
||||
echo "Waiting for Postgres to be healthy..."
|
||||
sleep 5
|
||||
@@ -0,0 +1,9 @@
|
||||
@echo off
|
||||
if "%TARGET_INSTANCE_ID%"=="api-1" (
|
||||
docker compose stop api1
|
||||
) else if "%TARGET_INSTANCE_ID%"=="api-2" (
|
||||
docker compose stop api2
|
||||
) else (
|
||||
echo Unknown instance
|
||||
exit /b 1
|
||||
)
|
||||
@@ -0,0 +1,15 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
|
||||
|
||||
# Проверяем, что нам передали api-1 или api-2
|
||||
if [ "$TARGET_INSTANCE_ID" != "api-1" ] && [ "$TARGET_INSTANCE_ID" != "api-2" ]; then
|
||||
echo "Unknown instance: $TARGET_INSTANCE_ID" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
SERVICE_NAME="$TARGET_INSTANCE_ID"
|
||||
|
||||
echo "Stopping container for $SERVICE_NAME..."
|
||||
docker compose stop "$SERVICE_NAME"
|
||||
@@ -0,0 +1,54 @@
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
def _read_secret(path: str | None) -> str | None:
|
||||
"""Прочитать секрет из файла, если задан путь. Иначе None."""
|
||||
if not path:
|
||||
return None
|
||||
p = Path(path)
|
||||
if not p.is_file():
|
||||
return None
|
||||
return p.read_text(encoding="utf-8").strip()
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_file=None, extra="ignore")
|
||||
|
||||
# Основные настройки приложения
|
||||
app_host: str = "0.0.0.0"
|
||||
app_port: int = 8080
|
||||
app_env: str = "lab"
|
||||
instance_id: str = "" # пусто -> сгенерируется в instance.py
|
||||
log_level: str = "info"
|
||||
|
||||
# Сессии и идемпотентность
|
||||
session_ttl_seconds: int = 3600
|
||||
idempotency_ttl_seconds: int = 86400
|
||||
|
||||
# Postgres: либо прямая строка, либо файл-секрет (приоритет у файла).
|
||||
database_url: str = ""
|
||||
database_url_file: str = "/run/secrets/database_url"
|
||||
|
||||
# S3 / MinIO (для 4-й лабы)
|
||||
s3_endpoint: str = "http://minio:9000"
|
||||
s3_access_key: str = "minioadmin"
|
||||
s3_secret_key: str = "minioadmin"
|
||||
s3_bucket: str = "messenger-attachments"
|
||||
s3_region: str = "us-east-1"
|
||||
|
||||
@property
|
||||
def db_dsn(self) -> str:
|
||||
from_file = _read_secret(self.database_url_file)
|
||||
dsn = from_file or self.database_url
|
||||
if not dsn:
|
||||
raise RuntimeError(
|
||||
"DATABASE_URL не задан: укажи DATABASE_URL или смонтируй DATABASE_URL_FILE"
|
||||
)
|
||||
return dsn
|
||||
|
||||
|
||||
@lru_cache
|
||||
def get_settings() -> Settings:
|
||||
return Settings()
|
||||
@@ -0,0 +1,45 @@
|
||||
from contextlib import contextmanager
|
||||
from typing import Iterator
|
||||
|
||||
import psycopg
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
from .config import get_settings
|
||||
|
||||
_pool: ConnectionPool | None = None
|
||||
|
||||
|
||||
def init_pool() -> ConnectionPool:
|
||||
global _pool
|
||||
if _pool is None:
|
||||
_pool = ConnectionPool(
|
||||
conninfo=get_settings().db_dsn,
|
||||
min_size=1,
|
||||
max_size=10,
|
||||
timeout=5,
|
||||
open=True,
|
||||
)
|
||||
return _pool
|
||||
|
||||
|
||||
def close_pool() -> None:
|
||||
global _pool
|
||||
if _pool is not None:
|
||||
_pool.close()
|
||||
_pool = None
|
||||
|
||||
|
||||
@contextmanager
|
||||
def connection() -> Iterator[psycopg.Connection]:
|
||||
pool = init_pool()
|
||||
with pool.connection() as conn:
|
||||
yield conn
|
||||
|
||||
|
||||
@contextmanager
|
||||
def transaction() -> Iterator[psycopg.Connection]:
|
||||
"""Явная транзакция: commit при успехе, rollback при исключении."""
|
||||
pool = init_pool()
|
||||
with pool.connection() as conn:
|
||||
with conn.transaction():
|
||||
yield conn
|
||||
@@ -0,0 +1,31 @@
|
||||
from typing import Any
|
||||
|
||||
from fastapi import Request
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
|
||||
class ApiError(Exception):
|
||||
"""Наше приложение бросает это; handler превращает в JSON-ответ."""
|
||||
|
||||
def __init__(self, status: int, code: str, message: str, headers: dict[str, str] | None = None):
|
||||
self.status = status
|
||||
self.code = code
|
||||
self.message = message
|
||||
self.headers = headers or {}
|
||||
|
||||
|
||||
def error_response(
|
||||
request: Request,
|
||||
status: int,
|
||||
code: str,
|
||||
message: str,
|
||||
extra_headers: dict[str, str] | None = None,
|
||||
) -> JSONResponse:
|
||||
request_id = getattr(request.state, "request_id", "") or "unknown"
|
||||
body: dict[str, Any] = {
|
||||
"error": {"code": code, "message": message, "request_id": request_id}
|
||||
}
|
||||
headers = {"X-Request-Id": request_id}
|
||||
if extra_headers:
|
||||
headers.update(extra_headers)
|
||||
return JSONResponse(status_code=status, content=body, headers=headers)
|
||||
@@ -0,0 +1,72 @@
|
||||
import hashlib
|
||||
import json
|
||||
from typing import Any, Callable
|
||||
|
||||
import psycopg
|
||||
|
||||
from .db import transaction
|
||||
from .errors import ApiError
|
||||
|
||||
|
||||
def _body_hash(body: dict[str, Any]) -> str:
|
||||
raw = json.dumps(body, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
|
||||
return hashlib.sha256(raw.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def run_idempotent(
|
||||
*,
|
||||
user_id: str,
|
||||
scope: str,
|
||||
key: str,
|
||||
body: dict[str, Any],
|
||||
action: Callable[[psycopg.Connection], tuple[int, dict[str, Any]]],
|
||||
) -> tuple[int, dict[str, Any]]:
|
||||
"""Атомарно: либо побочный эффект + сохранённый ответ, либо повтор."""
|
||||
h = _body_hash(body)
|
||||
try:
|
||||
with transaction() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO idempotency_keys
|
||||
(user_id, scope, key, request_hash, status_code, response_json)
|
||||
VALUES (%s, %s, %s, %s, 0, '{}'::jsonb)
|
||||
""",
|
||||
(user_id, scope, key, h),
|
||||
)
|
||||
result = action(conn)
|
||||
status, payload = result
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
UPDATE idempotency_keys
|
||||
SET status_code = %s, response_json = %s::jsonb
|
||||
WHERE user_id = %s AND scope = %s AND key = %s
|
||||
""",
|
||||
(status, json.dumps(payload, ensure_ascii=False), user_id, scope, key),
|
||||
)
|
||||
return status, payload
|
||||
except psycopg.errors.UniqueViolation:
|
||||
pass
|
||||
|
||||
with transaction() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT request_hash, status_code, response_json
|
||||
FROM idempotency_keys
|
||||
WHERE user_id = %s AND scope = %s AND key = %s
|
||||
""",
|
||||
(user_id, scope, key),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
|
||||
if row is None:
|
||||
raise ApiError(409, "conflict", "Конфликт идемпотентности, повторите")
|
||||
|
||||
existing_hash, status, payload = row
|
||||
if existing_hash != h:
|
||||
raise ApiError(409, "conflict", "Тот же ключ с другим телом запроса")
|
||||
if status == 0:
|
||||
raise ApiError(409, "conflict", "Повтор ещё не завершён")
|
||||
return status, payload
|
||||
@@ -0,0 +1,37 @@
|
||||
# import io
|
||||
# from PIL import Image
|
||||
|
||||
# def detect_image_type(data: bytes) -> str | None:
|
||||
# if data.startswith(b"\x89PNG\r\n\x1a\n"):
|
||||
# return "image/png"
|
||||
# if data.startswith(b"\xff\xd8\xff"):
|
||||
# return "image/jpeg"
|
||||
# return None
|
||||
|
||||
# def check_image(data: bytes, allowed_types: list[str]) -> tuple[str, int, int]:
|
||||
# content_type = detect_image_type(data)
|
||||
# if not content_type or content_type not in allowed_types:
|
||||
# raise ValueError("Invalid or unsupported image type")
|
||||
|
||||
# try:
|
||||
# with Image.open(io.BytesIO(data)) as img:
|
||||
# width, height = img.size
|
||||
# if width * height > 40_000_000:
|
||||
# raise ValueError("Too many pixels")
|
||||
# except Exception as e:
|
||||
# raise ValueError(f"Broken image: {e}")
|
||||
|
||||
# return content_type, width, height
|
||||
def detect_image_type(data: bytes) -> str | None:
|
||||
if data.startswith(b"\x89PNG\r\n\x1a\n"):
|
||||
return "image/png"
|
||||
if data.startswith(b"\xff\xd8\xff"):
|
||||
return "image/jpeg"
|
||||
return None
|
||||
|
||||
|
||||
def check_image(first_bytes: bytes, allowed_types: list[str]) -> tuple[str, int, int]:
|
||||
content_type = detect_image_type(first_bytes)
|
||||
if not content_type or content_type not in allowed_types:
|
||||
raise ValueError("Invalid or unsupported image type")
|
||||
return content_type, 0, 0
|
||||
@@ -0,0 +1,15 @@
|
||||
import os
|
||||
import secrets
|
||||
|
||||
from .config import get_settings
|
||||
|
||||
|
||||
def _build_instance_id() -> str:
|
||||
s = get_settings()
|
||||
if s.instance_id:
|
||||
return s.instance_id
|
||||
# Короткий случайный alias; при желании можно задать INSTANCE_ID в env.
|
||||
return "api-" + secrets.token_hex(6)
|
||||
|
||||
|
||||
INSTANCE_ID: str = _build_instance_id()
|
||||
@@ -0,0 +1,56 @@
|
||||
import uuid
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from . import db
|
||||
from .errors import ApiError, error_response
|
||||
from .instance import INSTANCE_ID
|
||||
from .routes import health, users, sessions, chats, messages, attachments
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
try:
|
||||
db.init_pool()
|
||||
except Exception:
|
||||
pass
|
||||
yield
|
||||
db.close_pool()
|
||||
|
||||
|
||||
app = FastAPI(title="MTUSI Messenger", version="1.4.0", lifespan=lifespan)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def request_context(request: Request, call_next):
|
||||
request_id = request.headers.get("X-Request-Id") or uuid.uuid4().hex
|
||||
request.state.request_id = request_id
|
||||
response = await call_next(request)
|
||||
response.headers["X-Request-Id"] = request_id
|
||||
response.headers["X-Instance-Id"] = INSTANCE_ID
|
||||
return response
|
||||
|
||||
|
||||
@app.exception_handler(ApiError)
|
||||
async def api_error_handler(request: Request, exc: ApiError):
|
||||
return error_response(request, exc.status, exc.code, exc.message, exc.headers)
|
||||
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def validation_handler(request: Request, exc: RequestValidationError):
|
||||
return error_response(request, 400, "invalid_request", "Неверный запрос")
|
||||
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
async def unhandled_handler(request: Request, exc: Exception):
|
||||
return error_response(request, 500, "internal_error", "Внутренняя ошибка сервера")
|
||||
|
||||
|
||||
app.include_router(health.router)
|
||||
app.include_router(users.router)
|
||||
app.include_router(sessions.router)
|
||||
app.include_router(chats.router)
|
||||
app.include_router(messages.router)
|
||||
app.include_router(attachments.router)
|
||||
@@ -0,0 +1,369 @@
|
||||
# import hashlib
|
||||
# import io
|
||||
# import uuid
|
||||
|
||||
# from fastapi import APIRouter, Depends, Request, Query
|
||||
# from fastapi.responses import StreamingResponse
|
||||
# from psycopg.rows import dict_row
|
||||
|
||||
# from ..db import transaction, connection
|
||||
# from ..errors import ApiError
|
||||
# from ..s3 import upload_fileobj, download_fileobj
|
||||
# from ..image_check import check_image
|
||||
# from ..serializers import iso_z
|
||||
# from .sessions import require_actor
|
||||
|
||||
# router = APIRouter(prefix="/api/v1")
|
||||
|
||||
# MAX_SIZE = 10 * 1024 * 1024
|
||||
|
||||
|
||||
# @router.post("/chats/{chat_id}/attachments", status_code=201)
|
||||
# async def upload_attachment(
|
||||
# chat_id: str,
|
||||
# request: Request,
|
||||
# filename: str = Query(..., max_length=255),
|
||||
# actor: dict = Depends(require_actor),
|
||||
# ):
|
||||
# try:
|
||||
# uuid.UUID(chat_id)
|
||||
# except ValueError:
|
||||
# raise ApiError(400, "invalid_request", "Неверный chat_id")
|
||||
|
||||
# if "/" in filename or "\\" in filename or "\0" in filename:
|
||||
# raise ApiError(400, "invalid_request", "Опасное имя файла")
|
||||
|
||||
# # Читаем тело запроса (raw bytes)
|
||||
# content = await request.body()
|
||||
# if len(content) > MAX_SIZE:
|
||||
# raise ApiError(413, "payload_too_large", "Файл слишком большой")
|
||||
# if len(content) == 0:
|
||||
# raise ApiError(400, "invalid_request", "Пустой файл")
|
||||
|
||||
# try:
|
||||
# content_type, width, height = check_image(content, ["image/png", "image/jpeg"])
|
||||
# except ValueError as e:
|
||||
# if "pixels" in str(e):
|
||||
# raise ApiError(413, "payload_too_large", "Слишком много пикселей")
|
||||
# raise ApiError(415, "unsupported_media_type", "Неподдерживаемый формат")
|
||||
|
||||
# # Проверяем доступ
|
||||
# with connection() as conn:
|
||||
# with conn.cursor() as cur:
|
||||
# cur.execute("SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s", (chat_id, actor["user_id"]))
|
||||
# if cur.fetchone() is None:
|
||||
# raise ApiError(404, "not_found", "Чат не найден")
|
||||
|
||||
# sha256 = hashlib.sha256(content).hexdigest()
|
||||
# object_key = f"chats/{chat_id}/{uuid.uuid4().hex}"
|
||||
|
||||
# # Загружаем в S3
|
||||
# try:
|
||||
# upload_fileobj(io.BytesIO(content), object_key, content_type)
|
||||
# except Exception as e:
|
||||
# print(f"S3 error: {e}")
|
||||
# raise ApiError(503, "unavailable", "S3 недоступен")
|
||||
|
||||
# # Сохраняем в базу
|
||||
# with transaction() as conn:
|
||||
# with conn.cursor(row_factory=dict_row) as cur:
|
||||
# cur.execute(
|
||||
# """
|
||||
# INSERT INTO attachments (chat_id, owner_id, filename, content_type, size_bytes, sha256, object_key)
|
||||
# VALUES (%s, %s, %s, %s, %s, %s, %s)
|
||||
# RETURNING id, chat_id, owner_id, filename, content_type, size_bytes, sha256, created_at
|
||||
# """,
|
||||
# (chat_id, actor["user_id"], filename, content_type, len(content), sha256, object_key),
|
||||
# )
|
||||
# att = cur.fetchone()
|
||||
|
||||
# return {
|
||||
# "id": str(att["id"]),
|
||||
# "chat_id": str(att["chat_id"]),
|
||||
# "owner_id": str(att["owner_id"]),
|
||||
# "filename": att["filename"],
|
||||
# "content_type": att["content_type"],
|
||||
# "size_bytes": att["size_bytes"],
|
||||
# "sha256": att["sha256"],
|
||||
# "status": "ready",
|
||||
# "variants": [],
|
||||
# "error_code": None,
|
||||
# "created_at": iso_z(att["created_at"]),
|
||||
# "updated_at": iso_z(att["created_at"]),
|
||||
# }
|
||||
|
||||
|
||||
# @router.get("/attachments/{attachment_id}")
|
||||
# def get_attachment(attachment_id: str, actor: dict = Depends(require_actor)):
|
||||
# try:
|
||||
# uuid.UUID(attachment_id)
|
||||
# except ValueError:
|
||||
# raise ApiError(400, "invalid_request", "Неверный id")
|
||||
|
||||
# with connection() as conn:
|
||||
# with conn.cursor(row_factory=dict_row) as cur:
|
||||
# cur.execute(
|
||||
# """
|
||||
# SELECT a.*, 1 as exists
|
||||
# FROM attachments a
|
||||
# JOIN chat_members cm ON cm.chat_id = a.chat_id AND cm.user_id = %s
|
||||
# WHERE a.id = %s
|
||||
# """,
|
||||
# (actor["user_id"], attachment_id)
|
||||
# )
|
||||
# att = cur.fetchone()
|
||||
|
||||
# if att is None:
|
||||
# raise ApiError(404, "not_found", "Вложение не найдено")
|
||||
|
||||
# return {
|
||||
# "id": str(att["id"]),
|
||||
# "chat_id": str(att["chat_id"]),
|
||||
# "owner_id": str(att["owner_id"]),
|
||||
# "filename": att["filename"],
|
||||
# "content_type": att["content_type"],
|
||||
# "size_bytes": att["size_bytes"],
|
||||
# "sha256": att["sha256"],
|
||||
# "status": "ready",
|
||||
# "variants": [],
|
||||
# "error_code": None,
|
||||
# "created_at": iso_z(att["created_at"]),
|
||||
# "updated_at": iso_z(att["created_at"]),
|
||||
# }
|
||||
|
||||
|
||||
# @router.get("/attachments/{attachment_id}/content")
|
||||
# def download_attachment(attachment_id: str, actor: dict = Depends(require_actor)):
|
||||
# try:
|
||||
# uuid.UUID(attachment_id)
|
||||
# except ValueError:
|
||||
# raise ApiError(400, "invalid_request", "Неверный id")
|
||||
|
||||
# with connection() as conn:
|
||||
# with conn.cursor(row_factory=dict_row) as cur:
|
||||
# cur.execute(
|
||||
# """
|
||||
# SELECT a.object_key, a.content_type, a.filename
|
||||
# FROM attachments a
|
||||
# JOIN chat_members cm ON cm.chat_id = a.chat_id AND cm.user_id = %s
|
||||
# WHERE a.id = %s
|
||||
# """,
|
||||
# (actor["user_id"], attachment_id)
|
||||
# )
|
||||
# att = cur.fetchone()
|
||||
|
||||
# if att is None:
|
||||
# raise ApiError(404, "not_found", "Вложение не найдено")
|
||||
|
||||
# try:
|
||||
# body = download_fileobj(att["object_key"])
|
||||
# except Exception:
|
||||
# raise ApiError(503, "unavailable", "S3 недоступен")
|
||||
|
||||
# def iter_body():
|
||||
# yield from body.iter_chunks(1024 * 1024)
|
||||
|
||||
# return StreamingResponse(
|
||||
# iter_body(),
|
||||
# media_type=att["content_type"],
|
||||
# headers={"Content-Disposition": f'attachment; filename="{att["filename"]}"'}
|
||||
# )
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
import tempfile
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, Depends, Request, Query
|
||||
from fastapi.responses import StreamingResponse
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
from ..db import transaction, connection
|
||||
from ..errors import ApiError
|
||||
from ..s3 import upload_fileobj, download_fileobj, delete_file
|
||||
from ..image_check import check_image
|
||||
from ..serializers import iso_z
|
||||
from .sessions import require_actor
|
||||
|
||||
router = APIRouter(prefix="/api/v1")
|
||||
|
||||
MAX_SIZE = 10 * 1024 * 1024
|
||||
|
||||
|
||||
@router.post("/chats/{chat_id}/attachments", status_code=201)
|
||||
async def upload_attachment(
|
||||
chat_id: str,
|
||||
request: Request,
|
||||
filename: str = Query(..., max_length=255),
|
||||
actor: dict = Depends(require_actor),
|
||||
):
|
||||
try:
|
||||
uuid.UUID(chat_id)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Неверный chat_id")
|
||||
|
||||
if "/" in filename or "\\" in filename or "\0" in filename:
|
||||
raise ApiError(400, "invalid_request", "Опасное имя файла")
|
||||
|
||||
# Проверяем доступ к чату
|
||||
with connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
|
||||
(chat_id, actor["user_id"]),
|
||||
)
|
||||
if cur.fetchone() is None:
|
||||
raise ApiError(404, "not_found", "Чат не найден")
|
||||
|
||||
sha256 = hashlib.sha256()
|
||||
size = 0
|
||||
first_chunk = b""
|
||||
|
||||
# Читаем потоково во временный файл (чтобы не забивать RAM)
|
||||
with tempfile.SpooledTemporaryFile(max_size=MAX_SIZE, mode="w+b") as spooled:
|
||||
async for chunk in request.stream():
|
||||
size += len(chunk)
|
||||
if size > MAX_SIZE:
|
||||
raise ApiError(413, "payload_too_large", "Файл слишком большой")
|
||||
if not first_chunk:
|
||||
first_chunk = chunk[:16] # для детекта MIME
|
||||
sha256.update(chunk)
|
||||
spooled.write(chunk)
|
||||
|
||||
if size == 0:
|
||||
raise ApiError(400, "invalid_request", "Пустой файл")
|
||||
|
||||
# Полная проверка картинки через Pillow
|
||||
try:
|
||||
spooled.seek(0)
|
||||
content = spooled.read()
|
||||
content_type, _, _ = check_image(first_chunk, ["image/png", "image/jpeg"])
|
||||
from PIL import Image
|
||||
Image.open(io.BytesIO(content)).verify()
|
||||
except Exception:
|
||||
raise ApiError(415, "unsupported_media_type", "Неподдерживаемый или битый формат")
|
||||
|
||||
sha256_hex = sha256.hexdigest()
|
||||
object_key = f"chats/{chat_id}/{uuid.uuid4().hex}"
|
||||
|
||||
# Загружаем в S3
|
||||
try:
|
||||
spooled.seek(0)
|
||||
upload_fileobj(spooled, object_key, content_type)
|
||||
except Exception as e:
|
||||
print(f"S3 error: {e}")
|
||||
raise ApiError(503, "unavailable", "S3 недоступен")
|
||||
|
||||
# Сохраняем в БД (с компенсацией)
|
||||
try:
|
||||
with transaction() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO attachments (chat_id, owner_id, filename, content_type, size_bytes, sha256, object_key)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s)
|
||||
RETURNING id, chat_id, owner_id, filename, content_type, size_bytes, sha256, created_at
|
||||
""",
|
||||
(chat_id, actor["user_id"], filename, content_type, size, sha256_hex, object_key),
|
||||
)
|
||||
att = cur.fetchone()
|
||||
except Exception as e:
|
||||
# КОМПЕНСАЦИЯ: удаляем из S3, если БД упала
|
||||
print(f"DB failed, compensating S3: {e}")
|
||||
try:
|
||||
delete_file(object_key)
|
||||
except Exception as s3_e:
|
||||
print(f"Failed to compensate S3: {s3_e}")
|
||||
raise ApiError(500, "internal_error", "Ошибка сохранения")
|
||||
|
||||
return {
|
||||
"id": str(att["id"]),
|
||||
"chat_id": str(att["chat_id"]),
|
||||
"owner_id": str(att["owner_id"]),
|
||||
"filename": att["filename"],
|
||||
"content_type": att["content_type"],
|
||||
"size_bytes": att["size_bytes"],
|
||||
"sha256": att["sha256"],
|
||||
"status": "ready",
|
||||
"variants": [],
|
||||
"error_code": None,
|
||||
"created_at": iso_z(att["created_at"]),
|
||||
"updated_at": iso_z(att["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/attachments/{attachment_id}")
|
||||
def get_attachment(attachment_id: str, actor: dict = Depends(require_actor)):
|
||||
try:
|
||||
uuid.UUID(attachment_id)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Неверный id")
|
||||
|
||||
with connection() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT a.*
|
||||
FROM attachments a
|
||||
JOIN chat_members cm ON cm.chat_id = a.chat_id AND cm.user_id = %s
|
||||
WHERE a.id = %s
|
||||
""",
|
||||
(actor["user_id"], attachment_id),
|
||||
)
|
||||
att = cur.fetchone()
|
||||
|
||||
if att is None:
|
||||
raise ApiError(404, "not_found", "Вложение не найдено")
|
||||
|
||||
return {
|
||||
"id": str(att["id"]),
|
||||
"chat_id": str(att["chat_id"]),
|
||||
"owner_id": str(att["owner_id"]),
|
||||
"filename": att["filename"],
|
||||
"content_type": att["content_type"],
|
||||
"size_bytes": att["size_bytes"],
|
||||
"sha256": att["sha256"],
|
||||
"status": "ready",
|
||||
"variants": [],
|
||||
"error_code": None,
|
||||
"created_at": iso_z(att["created_at"]),
|
||||
"updated_at": iso_z(att["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/attachments/{attachment_id}/content")
|
||||
def download_attachment(attachment_id: str, actor: dict = Depends(require_actor)):
|
||||
try:
|
||||
uuid.UUID(attachment_id)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Неверный id")
|
||||
|
||||
with connection() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT a.object_key, a.content_type, a.filename
|
||||
FROM attachments a
|
||||
JOIN chat_members cm ON cm.chat_id = a.chat_id AND cm.user_id = %s
|
||||
WHERE a.id = %s
|
||||
""",
|
||||
(actor["user_id"], attachment_id),
|
||||
)
|
||||
att = cur.fetchone()
|
||||
|
||||
if att is None:
|
||||
raise ApiError(404, "not_found", "Вложение не найдено")
|
||||
|
||||
try:
|
||||
body = download_fileobj(att["object_key"])
|
||||
except Exception:
|
||||
raise ApiError(503, "unavailable", "S3 недоступен")
|
||||
|
||||
def iter_body():
|
||||
yield from body.iter_chunks(1024 * 1024)
|
||||
|
||||
return StreamingResponse(
|
||||
iter_body(),
|
||||
media_type=att["content_type"],
|
||||
headers={"Content-Disposition": f'attachment; filename="{att["filename"]}"'},
|
||||
)
|
||||
@@ -0,0 +1,169 @@
|
||||
import base64
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
import psycopg
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
from ..db import connection, transaction
|
||||
from ..errors import ApiError
|
||||
from ..serializers import row_to_chat
|
||||
from .sessions import require_actor
|
||||
|
||||
router = APIRouter(prefix="/api/v1")
|
||||
|
||||
|
||||
class CreateChat(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
title: str = Field(min_length=1, max_length=100)
|
||||
member_ids: list[str] = Field(min_length=1, max_length=100)
|
||||
|
||||
@field_validator("title")
|
||||
@classmethod
|
||||
def title_not_blank(cls, v: str) -> str:
|
||||
if not v.strip():
|
||||
raise ValueError("title must not be blank")
|
||||
return v
|
||||
|
||||
@field_validator("member_ids")
|
||||
@classmethod
|
||||
def unique_members(cls, v: list[str]) -> list[str]:
|
||||
if len(set(v)) != len(v):
|
||||
raise ValueError("duplicate member_ids")
|
||||
for s in v:
|
||||
try:
|
||||
uuid.UUID(s)
|
||||
except ValueError:
|
||||
raise ValueError("invalid uuid in member_ids")
|
||||
return v
|
||||
|
||||
|
||||
def _encode_cursor(created_at: datetime, id_) -> str:
|
||||
raw = f"{created_at.isoformat()}|{id_}"
|
||||
return base64.urlsafe_b64encode(raw.encode("utf-8")).decode("ascii").rstrip("=")
|
||||
|
||||
|
||||
def _decode_cursor(cursor: str):
|
||||
pad = "=" * (-len(cursor) % 4)
|
||||
raw = base64.urlsafe_b64decode(cursor + pad).decode("utf-8")
|
||||
ts, id_ = raw.split("|", 1)
|
||||
return datetime.fromisoformat(ts), id_
|
||||
|
||||
|
||||
@router.post("/chats", status_code=201)
|
||||
def create_chat(body: CreateChat, actor: dict = Depends(require_actor)):
|
||||
me = actor["user_id"]
|
||||
if str(me) not in body.member_ids:
|
||||
raise ApiError(400, "invalid_request", "Вызывающий должен входить в member_ids")
|
||||
|
||||
try:
|
||||
with transaction() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(
|
||||
"SELECT id FROM users WHERE id = ANY(%s::uuid[])",
|
||||
(body.member_ids,),
|
||||
)
|
||||
found = {str(r["id"]) for r in cur.fetchall()}
|
||||
missing = set(body.member_ids) - found
|
||||
if missing:
|
||||
raise ApiError(400, "invalid_request", "Неизвестные участники")
|
||||
|
||||
cur.execute(
|
||||
"INSERT INTO chats (title) VALUES (%s) RETURNING id, title, created_at",
|
||||
(body.title,),
|
||||
)
|
||||
chat = cur.fetchone()
|
||||
|
||||
cur.executemany(
|
||||
"INSERT INTO chat_members (chat_id, user_id) VALUES (%s, %s)",
|
||||
[(chat["id"], m) for m in body.member_ids],
|
||||
)
|
||||
except ApiError:
|
||||
raise
|
||||
except psycopg.errors.ForeignKeyViolation:
|
||||
raise ApiError(400, "invalid_request", "Неизвестные участники")
|
||||
|
||||
chat["member_ids"] = body.member_ids
|
||||
return row_to_chat(chat)
|
||||
|
||||
|
||||
@router.get("/chats")
|
||||
def list_chats(
|
||||
limit: int = Query(default=50, ge=1, le=100),
|
||||
cursor: Optional[str] = Query(default=None),
|
||||
actor: dict = Depends(require_actor),
|
||||
):
|
||||
params: list = []
|
||||
where = ["cm.user_id = %s"]
|
||||
params.append(actor["user_id"])
|
||||
|
||||
if cursor:
|
||||
try:
|
||||
c_ts, c_id = _decode_cursor(cursor)
|
||||
except Exception:
|
||||
raise ApiError(400, "invalid_request", "Неверный cursor")
|
||||
where.append("(c.created_at, c.id) > (%s, %s)")
|
||||
params.extend([c_ts, c_id])
|
||||
|
||||
sql = f"""
|
||||
SELECT c.id, c.title, c.created_at,
|
||||
ARRAY(
|
||||
SELECT cm2.user_id::text
|
||||
FROM chat_members cm2
|
||||
WHERE cm2.chat_id = c.id
|
||||
ORDER BY cm2.user_id
|
||||
) AS member_ids
|
||||
FROM chats c
|
||||
JOIN chat_members cm ON cm.chat_id = c.id
|
||||
WHERE {' AND '.join(where)}
|
||||
ORDER BY c.created_at, c.id
|
||||
LIMIT %s
|
||||
"""
|
||||
params.append(limit + 1)
|
||||
|
||||
with connection() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(sql, params)
|
||||
rows = cur.fetchall()
|
||||
|
||||
next_cursor = None
|
||||
if len(rows) > limit:
|
||||
last = rows[limit - 1]
|
||||
next_cursor = _encode_cursor(last["created_at"], str(last["id"]))
|
||||
rows = rows[:limit]
|
||||
|
||||
return {"items": [row_to_chat(r) for r in rows], "next_cursor": next_cursor}
|
||||
|
||||
|
||||
@router.get("/chats/{chat_id}")
|
||||
def get_chat(chat_id: str, actor: dict = Depends(require_actor)):
|
||||
try:
|
||||
uuid.UUID(chat_id)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Неверный chat_id")
|
||||
|
||||
with connection() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT c.id, c.title, c.created_at,
|
||||
ARRAY(
|
||||
SELECT cm2.user_id::text
|
||||
FROM chat_members cm2
|
||||
WHERE cm2.chat_id = c.id
|
||||
ORDER BY cm2.user_id
|
||||
) AS member_ids
|
||||
FROM chats c
|
||||
JOIN chat_members cm ON cm.chat_id = c.id
|
||||
WHERE c.id = %s AND cm.user_id = %s
|
||||
""",
|
||||
(chat_id, actor["user_id"]),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
if row is None:
|
||||
raise ApiError(404, "not_found", "Чат не найден")
|
||||
return row_to_chat(row)
|
||||
@@ -0,0 +1,26 @@
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from ..db import connection
|
||||
from ..errors import error_response
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/health/live")
|
||||
def live():
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
@router.get("/health/ready")
|
||||
def ready(request: Request):
|
||||
try:
|
||||
with connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("SELECT 1")
|
||||
cur.fetchone()
|
||||
except Exception:
|
||||
return error_response(
|
||||
request, 503, "unavailable", "Зависимость недоступна"
|
||||
)
|
||||
return {"status": "ok"}
|
||||
@@ -0,0 +1,171 @@
|
||||
import re
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, Query
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
from ..db import connection
|
||||
from ..errors import ApiError
|
||||
from ..idempotency import run_idempotent
|
||||
from ..serializers import row_to_message
|
||||
from .sessions import require_actor
|
||||
|
||||
router = APIRouter(prefix="/api/v1")
|
||||
|
||||
KEY_RE = re.compile(r"^[A-Za-z0-9._:-]{1,128}$")
|
||||
|
||||
|
||||
class CreateMessage(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
text: str = Field(max_length=4000)
|
||||
attachment_ids: list[str] = Field(default_factory=list, max_length=10)
|
||||
|
||||
|
||||
def _check_chat_member(conn, chat_id: str, user_id: str) -> None:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
|
||||
(chat_id, user_id),
|
||||
)
|
||||
if cur.fetchone() is None:
|
||||
raise ApiError(404, "not_found", "Чат не найден")
|
||||
|
||||
|
||||
def _encode_cursor(created_at: datetime, id_) -> str:
|
||||
import base64
|
||||
raw = f"{created_at.isoformat()}|{id_}"
|
||||
return base64.urlsafe_b64encode(raw.encode("utf-8")).decode("ascii").rstrip("=")
|
||||
|
||||
|
||||
def _decode_cursor(cursor: str):
|
||||
import base64
|
||||
pad = "=" * (-len(cursor) % 4)
|
||||
raw = base64.urlsafe_b64decode(cursor + pad).decode("utf-8")
|
||||
ts, id_ = raw.split("|", 1)
|
||||
return datetime.fromisoformat(ts), id_
|
||||
|
||||
|
||||
@router.post("/chats/{chat_id}/messages", status_code=201)
|
||||
def create_message(
|
||||
chat_id: str,
|
||||
body: CreateMessage,
|
||||
idempotency_key: str = Header(..., alias="Idempotency-Key"),
|
||||
actor: dict = Depends(require_actor),
|
||||
):
|
||||
try:
|
||||
uuid.UUID(chat_id)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Неверный chat_id")
|
||||
|
||||
if not KEY_RE.fullmatch(idempotency_key or ""):
|
||||
raise ApiError(400, "invalid_request", "Неверный Idempotency-Key")
|
||||
|
||||
if not body.text.strip() and not body.attachment_ids:
|
||||
raise ApiError(400, "invalid_request", "Нужен текст или вложение")
|
||||
|
||||
scope = f"POST /api/v1/chats/{chat_id}/messages"
|
||||
|
||||
def action(conn):
|
||||
_check_chat_member(conn, chat_id, actor["user_id"])
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO messages (chat_id, sender_id, text)
|
||||
VALUES (%s, %s, %s)
|
||||
RETURNING id, chat_id, sender_id, text, created_at
|
||||
""",
|
||||
(chat_id, actor["user_id"], body.text),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
|
||||
if body.attachment_ids:
|
||||
for att_id in body.attachment_ids:
|
||||
cur.execute(
|
||||
"SELECT 1 FROM attachments WHERE id = %s AND chat_id = %s",
|
||||
(att_id, chat_id),
|
||||
)
|
||||
if cur.fetchone() is None:
|
||||
raise ApiError(404, "not_found", "Вложение не найдено")
|
||||
|
||||
cur.executemany(
|
||||
"INSERT INTO message_attachments (message_id, attachment_id) VALUES (%s, %s)",
|
||||
[(row["id"], att_id) for att_id in body.attachment_ids]
|
||||
)
|
||||
|
||||
row["attachment_ids"] = body.attachment_ids
|
||||
|
||||
return 201, row_to_message(row)
|
||||
|
||||
_, payload = run_idempotent(
|
||||
user_id=actor["user_id"],
|
||||
scope=scope,
|
||||
key=idempotency_key,
|
||||
body=body.model_dump(),
|
||||
action=action,
|
||||
)
|
||||
return payload
|
||||
|
||||
|
||||
@router.get("/chats/{chat_id}/messages")
|
||||
def list_messages(
|
||||
chat_id: str,
|
||||
limit: int = Query(default=50, ge=1, le=100),
|
||||
cursor: Optional[str] = Query(default=None),
|
||||
actor: dict = Depends(require_actor),
|
||||
):
|
||||
try:
|
||||
uuid.UUID(chat_id)
|
||||
except ValueError:
|
||||
raise ApiError(400, "invalid_request", "Неверный chat_id")
|
||||
|
||||
with connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"SELECT 1 FROM chat_members WHERE chat_id = %s AND user_id = %s",
|
||||
(chat_id, actor["user_id"]),
|
||||
)
|
||||
if cur.fetchone() is None:
|
||||
raise ApiError(404, "not_found", "Чат не найден")
|
||||
|
||||
params: list = [chat_id]
|
||||
where = ["m.chat_id = %s"]
|
||||
if cursor:
|
||||
try:
|
||||
c_ts, c_id = _decode_cursor(cursor)
|
||||
except Exception:
|
||||
raise ApiError(400, "invalid_request", "Неверный cursor")
|
||||
where.append("(m.created_at, m.id) > (%s, %s)")
|
||||
params.extend([c_ts, c_id])
|
||||
|
||||
sql = f"""
|
||||
SELECT m.id, m.chat_id, m.sender_id, m.text, m.created_at,
|
||||
COALESCE(
|
||||
ARRAY(
|
||||
SELECT ma.attachment_id::text
|
||||
FROM message_attachments ma
|
||||
WHERE ma.message_id = m.id
|
||||
), ARRAY[]::text[]
|
||||
) AS attachment_ids
|
||||
FROM messages m
|
||||
WHERE {' AND '.join(where)}
|
||||
ORDER BY m.created_at, m.id
|
||||
LIMIT %s
|
||||
"""
|
||||
params.append(limit + 1)
|
||||
|
||||
with connection() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(sql, params)
|
||||
rows = cur.fetchall()
|
||||
|
||||
next_cursor = None
|
||||
if len(rows) > limit:
|
||||
last = rows[limit - 1]
|
||||
next_cursor = _encode_cursor(last["created_at"], str(last["id"]))
|
||||
rows = rows[:limit]
|
||||
|
||||
return {"items": [row_to_message(r) for r in rows], "next_cursor": next_cursor}
|
||||
@@ -0,0 +1,131 @@
|
||||
import base64
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi.responses import Response
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
from ..config import get_settings
|
||||
from ..db import connection, transaction
|
||||
from ..errors import ApiError
|
||||
from ..security import (
|
||||
new_session_token,
|
||||
session_expiry,
|
||||
token_digest,
|
||||
verify_password,
|
||||
)
|
||||
from ..serializers import row_to_user
|
||||
|
||||
router = APIRouter(prefix="/api/v1")
|
||||
|
||||
WWW_BASIC = 'Basic realm="messenger", charset="UTF-8"'
|
||||
WWW_BEARER = 'Bearer realm="messenger"'
|
||||
|
||||
|
||||
def _parse_basic(header: str | None) -> tuple[str, str]:
|
||||
if not header or not header.lower().startswith("basic "):
|
||||
raise ApiError(401, "unauthorized", "Требуется Basic", {"WWW-Authenticate": WWW_BASIC})
|
||||
try:
|
||||
decoded = base64.b64decode(header[6:].strip(), validate=True).decode("utf-8")
|
||||
except Exception:
|
||||
raise ApiError(401, "unauthorized", "Неверные учётные данные", {"WWW-Authenticate": WWW_BASIC})
|
||||
|
||||
if ":" not in decoded:
|
||||
raise ApiError(401, "unauthorized", "Неверные учётные данные", {"WWW-Authenticate": WWW_BASIC})
|
||||
|
||||
username, password = decoded.split(":", 1)
|
||||
return username, password
|
||||
|
||||
|
||||
@router.post("/auth/sessions", status_code=201)
|
||||
def create_session(request: Request):
|
||||
header = request.headers.get("Authorization")
|
||||
username, password = _parse_basic(header)
|
||||
|
||||
with connection() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(
|
||||
"SELECT id, username, display_name, created_at, password_hash "
|
||||
"FROM users WHERE username = %s",
|
||||
(username,),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
|
||||
# Одинаковый 401 для «нет пользователя» и «неверный пароль».
|
||||
if row is None or not verify_password(password, row["password_hash"]):
|
||||
raise ApiError(401, "unauthorized", "Неверные учётные данные", {"WWW-Authenticate": WWW_BASIC})
|
||||
|
||||
settings = get_settings()
|
||||
token = new_session_token()
|
||||
expires_at = session_expiry(settings.session_ttl_seconds)
|
||||
|
||||
with transaction() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO sessions (user_id, token_hash, expires_at)
|
||||
VALUES (%s, %s, %s)
|
||||
""",
|
||||
(row["id"], token_digest(token), expires_at),
|
||||
)
|
||||
|
||||
return {
|
||||
"token": token,
|
||||
"token_type": "Bearer",
|
||||
"expires_at": expires_at.isoformat().replace("+00:00", "Z"),
|
||||
"user": row_to_user(row),
|
||||
}
|
||||
|
||||
|
||||
def _extract_bearer(request: Request) -> str:
|
||||
header = request.headers.get("Authorization")
|
||||
if not header or not header.lower().startswith("bearer "):
|
||||
raise ApiError(401, "unauthorized", "Требуется Bearer", {"WWW-Authenticate": WWW_BEARER})
|
||||
token = header[7:].strip()
|
||||
if not token:
|
||||
raise ApiError(401, "unauthorized", "Требуется Bearer", {"WWW-Authenticate": WWW_BEARER})
|
||||
return token
|
||||
|
||||
|
||||
def require_actor(request: Request) -> dict:
|
||||
"""Зависимость: валидирует Bearer и возвращает actor с user и token_hash."""
|
||||
token = _extract_bearer(request)
|
||||
digest = token_digest(token)
|
||||
with connection() as conn:
|
||||
with conn.cursor(row_factory=dict_row) as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
SELECT s.user_id, s.expires_at,
|
||||
u.id, u.username, u.display_name, u.created_at
|
||||
FROM sessions s
|
||||
JOIN users u ON u.id = s.user_id
|
||||
WHERE s.token_hash = %s
|
||||
""",
|
||||
(digest,),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
if row is None:
|
||||
raise ApiError(401, "unauthorized", "Сессия недействительна", {"WWW-Authenticate": WWW_BEARER})
|
||||
|
||||
# Проверка срока на стороне БД
|
||||
with connection() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(
|
||||
"SELECT 1 FROM sessions WHERE token_hash = %s AND expires_at > now()",
|
||||
(digest,),
|
||||
)
|
||||
if cur.fetchone() is None:
|
||||
raise ApiError(401, "unauthorized", "Сессия истекла", {"WWW-Authenticate": WWW_BEARER})
|
||||
|
||||
return {
|
||||
"user": row_to_user(row),
|
||||
"user_id": row["user_id"],
|
||||
"token_hash": digest,
|
||||
}
|
||||
|
||||
|
||||
@router.delete("/auth/sessions/current", status_code=204)
|
||||
def revoke_session(request: Request, actor: dict = Depends(require_actor)):
|
||||
with transaction() as conn:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute("DELETE FROM sessions WHERE token_hash = %s", (actor["token_hash"],))
|
||||
return Response(status_code=204)
|
||||
@@ -0,0 +1,52 @@
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
import psycopg
|
||||
|
||||
from ..db import transaction
|
||||
from ..errors import ApiError
|
||||
from ..security import hash_password
|
||||
from ..serializers import row_to_user
|
||||
from .sessions import require_actor
|
||||
|
||||
router = APIRouter(prefix="/api/v1")
|
||||
|
||||
|
||||
class CreateUser(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
username: str = Field(pattern=r"^[a-z0-9_]{3,32}$")
|
||||
display_name: str = Field(min_length=1, max_length=100)
|
||||
password: str = Field(min_length=12, max_length=128)
|
||||
|
||||
@field_validator("display_name")
|
||||
@classmethod
|
||||
def not_blank(cls, v: str) -> str:
|
||||
if not v.strip():
|
||||
raise ValueError("display_name must not be blank")
|
||||
return v
|
||||
|
||||
|
||||
@router.post("/users", status_code=201)
|
||||
def create_user(body: CreateUser):
|
||||
pwd_hash = hash_password(body.password)
|
||||
try:
|
||||
with transaction() as conn:
|
||||
with conn.cursor(row_factory=psycopg.rows.dict_row) as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
INSERT INTO users (username, display_name, password_hash)
|
||||
VALUES (%s, %s, %s)
|
||||
RETURNING id, username, display_name, created_at
|
||||
""",
|
||||
(body.username, body.display_name, pwd_hash),
|
||||
)
|
||||
row = cur.fetchone()
|
||||
except psycopg.errors.UniqueViolation:
|
||||
raise ApiError(409, "conflict", "Имя пользователя занято")
|
||||
|
||||
return row_to_user(row)
|
||||
|
||||
|
||||
@router.get("/users/me")
|
||||
def current_user(actor: dict = Depends(require_actor)):
|
||||
return actor["user"]
|
||||
@@ -0,0 +1,69 @@
|
||||
# import boto3
|
||||
# from botocore.config import Config
|
||||
# from .config import get_settings
|
||||
|
||||
# def get_s3_client():
|
||||
# s = get_settings()
|
||||
# return boto3.client(
|
||||
# "s3",
|
||||
# endpoint_url=s.s3_endpoint,
|
||||
# aws_access_key_id=s.s3_access_key,
|
||||
# aws_secret_access_key=s.s3_secret_key,
|
||||
# region_name=s.s3_region,
|
||||
# config=Config(signature_version="s3v4"),
|
||||
# )
|
||||
|
||||
# def upload_fileobj(fileobj, key: str, content_type: str):
|
||||
# client = get_s3_client()
|
||||
# s = get_settings()
|
||||
# client.upload_fileobj(
|
||||
# fileobj,
|
||||
# s.s3_bucket,
|
||||
# key,
|
||||
# ExtraArgs={"ContentType": content_type},
|
||||
# )
|
||||
|
||||
# def download_fileobj(key: str):
|
||||
# client = get_s3_client()
|
||||
# s = get_settings()
|
||||
# obj = client.get_object(Bucket=s.s3_bucket, Key=key)
|
||||
# return obj["Body"]
|
||||
import boto3
|
||||
from botocore.config import Config
|
||||
from .config import get_settings
|
||||
|
||||
|
||||
def get_s3_client():
|
||||
s = get_settings()
|
||||
return boto3.client(
|
||||
"s3",
|
||||
endpoint_url=s.s3_endpoint,
|
||||
aws_access_key_id=s.s3_access_key,
|
||||
aws_secret_access_key=s.s3_secret_key,
|
||||
region_name=s.s3_region,
|
||||
config=Config(signature_version="s3v4"),
|
||||
)
|
||||
|
||||
|
||||
def upload_fileobj(fileobj, key: str, content_type: str):
|
||||
client = get_s3_client()
|
||||
s = get_settings()
|
||||
client.upload_fileobj(
|
||||
fileobj,
|
||||
s.s3_bucket,
|
||||
key,
|
||||
ExtraArgs={"ContentType": content_type},
|
||||
)
|
||||
|
||||
|
||||
def download_fileobj(key: str):
|
||||
client = get_s3_client()
|
||||
s = get_settings()
|
||||
obj = client.get_object(Bucket=s.s3_bucket, Key=key)
|
||||
return obj["Body"]
|
||||
|
||||
|
||||
def delete_file(key: str):
|
||||
client = get_s3_client()
|
||||
s = get_settings()
|
||||
client.delete_object(Bucket=s.s3_bucket, Key=key)
|
||||
@@ -0,0 +1,34 @@
|
||||
import hashlib
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from argon2 import PasswordHasher
|
||||
from argon2.exceptions import VerifyMismatchError
|
||||
|
||||
_ph = PasswordHasher() # Argon2id по умолчанию
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
return _ph.hash(password)
|
||||
|
||||
|
||||
def verify_password(password: str, password_hash: str) -> bool:
|
||||
try:
|
||||
return _ph.verify(password_hash, password)
|
||||
except VerifyMismatchError:
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def new_session_token() -> str:
|
||||
# 32 байта = 256 бит энтропии; url-safe hex без паддинга.
|
||||
return secrets.token_urlsafe(32)
|
||||
|
||||
|
||||
def token_digest(token: str) -> str:
|
||||
return hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def session_expiry(ttl_seconds: int) -> datetime:
|
||||
return datetime.now(timezone.utc) + timedelta(seconds=ttl_seconds)
|
||||
@@ -0,0 +1,35 @@
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
def iso_z(dt: datetime) -> str:
|
||||
"""RFC3339 UTC с 'Z'. Postgres отдаёт timestamptz с '+00:00'."""
|
||||
return dt.isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def row_to_user(row) -> dict:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"username": row["username"],
|
||||
"display_name": row["display_name"],
|
||||
"created_at": iso_z(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
def row_to_chat(row) -> dict:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"title": row["title"],
|
||||
"member_ids": row["member_ids"],
|
||||
"created_at": iso_z(row["created_at"]),
|
||||
}
|
||||
|
||||
|
||||
def row_to_message(row) -> dict:
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"chat_id": str(row["chat_id"]),
|
||||
"sender_id": str(row["sender_id"]),
|
||||
"text": row["text"],
|
||||
"created_at": iso_z(row["created_at"]),
|
||||
"attachment_ids": row.get("attachment_ids", []),
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
-- 001_init.sql
|
||||
CREATE EXTENSION IF NOT EXISTS pgcrypto;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
username text NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||
display_name text NOT NULL CHECK (char_length(display_name) BETWEEN 1 AND 100),
|
||||
password_hash text NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
token_hash text PRIMARY KEY,
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
expires_at timestamptz NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS sessions_user_id_idx ON sessions(user_id);
|
||||
CREATE INDEX IF NOT EXISTS sessions_expires_at_idx ON sessions(expires_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS chats (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
title text NOT NULL CHECK (char_length(title) BETWEEN 1 AND 100),
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS chat_members (
|
||||
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS chat_members_user_idx ON chat_members(user_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
sender_id uuid NOT NULL REFERENCES users(id),
|
||||
text text NOT NULL DEFAULT '',
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS messages_chat_order_idx ON messages(chat_id, created_at, id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS idempotency_keys (
|
||||
user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
scope text NOT NULL, -- method + " " + path
|
||||
key text NOT NULL,
|
||||
request_hash text NOT NULL,
|
||||
status_code int NOT NULL,
|
||||
response_json jsonb NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
PRIMARY KEY (user_id, scope, key)
|
||||
);
|
||||
@@ -0,0 +1,20 @@
|
||||
-- 002_attachments.sql
|
||||
CREATE TABLE IF NOT EXISTS attachments (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
chat_id uuid NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||
owner_id uuid NOT NULL REFERENCES users(id),
|
||||
filename text NOT NULL,
|
||||
content_type text NOT NULL,
|
||||
size_bytes bigint NOT NULL,
|
||||
sha256 text NOT NULL,
|
||||
object_key text NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS attachments_chat_idx ON attachments(chat_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS message_attachments (
|
||||
message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
|
||||
attachment_id uuid NOT NULL REFERENCES attachments(id) ON DELETE CASCADE,
|
||||
PRIMARY KEY (message_id, attachment_id)
|
||||
);
|
||||
@@ -0,0 +1,113 @@
|
||||
import base64
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
BASE_URL = "http://localhost:8080"
|
||||
|
||||
def req(method, path, data=None, headers=None, raw=None):
|
||||
r = urllib.request.Request(BASE_URL + path, method=method)
|
||||
if headers:
|
||||
for k, v in headers.items():
|
||||
r.add_header(k, v)
|
||||
if data:
|
||||
r.add_header("Content-Type", "application/json")
|
||||
r.data = json.dumps(data).encode("utf-8")
|
||||
if raw:
|
||||
r.data = raw
|
||||
try:
|
||||
with urllib.request.urlopen(r) as resp:
|
||||
return resp.status, resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, e.read()
|
||||
|
||||
def setup():
|
||||
u = "test4_" + str(random.randint(1000, 9999))
|
||||
_, body = req("POST", "/api/v1/users", {"username": u, "display_name": "Тест", "password": "testpassword1234"})
|
||||
user = json.loads(body)
|
||||
|
||||
cred = base64.b64encode((u + ":testpassword1234").encode()).decode()
|
||||
_, body = req("POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic " + cred})
|
||||
token = json.loads(body)["token"]
|
||||
|
||||
headers = {"Authorization": "Bearer " + token}
|
||||
_, body = req("POST", "/api/v1/chats", {"title": "Тестовый чат", "member_ids": [user["id"]]}, headers)
|
||||
chat = json.loads(body)
|
||||
return user, chat, headers
|
||||
|
||||
def main():
|
||||
user, chat, headers = setup()
|
||||
print(f"Пользователь {user['id']} и чат {chat['id']} созданы.")
|
||||
|
||||
chat_url = f"/api/v1/chats/{chat['id']}/attachments"
|
||||
|
||||
# 1. Слишком большой файл (11 МБ)
|
||||
print("1. Тест: слишком большой файл...")
|
||||
big_data = b"\x89PNG\r\n\x1a\n" + b"A" * (11 * 1024 * 1024)
|
||||
status, _ = req("POST", chat_url + "?filename=big.png", headers={**headers, "Content-Type": "image/png"}, raw=big_data)
|
||||
assert status == 413, f"Ожидали 413, получили {status}"
|
||||
print(" OK (413)")
|
||||
|
||||
# 2. Битый файл (MIME PNG, но внутри мусор)
|
||||
print("2. Тест: битый файл...")
|
||||
broken_data = b"\x89PNG\r\n\x1a\n" + b"broken data"
|
||||
status, _ = req("POST", chat_url + "?filename=broken.png", headers={**headers, "Content-Type": "image/png"}, raw=broken_data)
|
||||
assert status == 415, f"Ожидали 415, получили {status}"
|
||||
print(" OK (415)")
|
||||
|
||||
# 3. Подмена MIME (говорим image/png, а внутри текст)
|
||||
print("3. Тест: подмена MIME...")
|
||||
text_data = b"Hello, this is not an image!"
|
||||
status, _ = req("POST", chat_url + "?filename=fake.png", headers={**headers, "Content-Type": "image/png"}, raw=text_data)
|
||||
assert status == 415, f"Ожидали 415, получили {status}"
|
||||
print(" OK (415)")
|
||||
|
||||
# 4. Опасное имя файла (path traversal)
|
||||
print("4. Тест: опасное имя файла...")
|
||||
with open("tests/fixtures/sample.png", "rb") as f:
|
||||
valid_png = f.read()
|
||||
status, _ = req("POST", chat_url + "?filename=../../etc/passwd", headers={**headers, "Content-Type": "image/png"}, raw=valid_png)
|
||||
assert status == 400, f"Ожидали 400, получили {status}"
|
||||
print(" OK (400)")
|
||||
|
||||
# 5. Успешная загрузка (чтобы проверить доступ)
|
||||
print("5. Тест: успешная загрузка...")
|
||||
status, body = req("POST", chat_url + "?filename=ok.png", headers={**headers, "Content-Type": "image/png"}, raw=valid_png)
|
||||
assert status == 201, f"Ожидали 201, получили {status}"
|
||||
att = json.loads(body)
|
||||
print(f" OK (201), ID: {att['id']}")
|
||||
|
||||
# 6. Чужой доступ
|
||||
print("6. Тест: чужой доступ...")
|
||||
u_eve = "eve_" + str(random.randint(1000, 9999))
|
||||
req("POST", "/api/v1/users", {"username": u_eve, "display_name": "Eve", "password": "testpassword1234"})
|
||||
cred_eve = base64.b64encode((u_eve + ":testpassword1234").encode()).decode()
|
||||
_, body = req("POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic " + cred_eve})
|
||||
token_eve = json.loads(body)["token"]
|
||||
headers_eve = {"Authorization": "Bearer " + token_eve}
|
||||
|
||||
status, _ = req("GET", f"/api/v1/attachments/{att['id']}", headers=headers_eve)
|
||||
assert status == 404, f"Ожидали 404 (чужой), получили {status}"
|
||||
print(" OK (404)")
|
||||
|
||||
# 7. Нагрузочный тест (10 параллельных загрузок)
|
||||
print("7. Тест: 10 параллельных загрузок (проверка RAM)...")
|
||||
def upload_worker():
|
||||
status, _ = req("POST", chat_url + "?filename=parallel.png", headers={**headers, "Content-Type": "image/png"}, raw=valid_png)
|
||||
return status
|
||||
|
||||
with ThreadPoolExecutor(max_workers=10) as pool:
|
||||
results = list(pool.map(lambda _: upload_worker(), range(10)))
|
||||
|
||||
assert all(s == 201 for s in results), f"Не все загрузки успешны: {results}"
|
||||
print(" OK (все 10 успешно)")
|
||||
|
||||
print("\nPASS: Уровень 4 пройден! (Все негативные тесты и нагрузка работают)")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,31 @@
|
||||
import urllib.request, json, base64, random
|
||||
|
||||
BASE = "http://localhost:8080"
|
||||
|
||||
def req(path, method="GET", data=None, headers=None):
|
||||
r = urllib.request.Request(BASE + path, method=method)
|
||||
if headers:
|
||||
for k, v in headers.items(): r.add_header(k, v)
|
||||
if data:
|
||||
r.add_header("Content-Type", "application/json")
|
||||
r.data = json.dumps(data).encode("utf-8")
|
||||
try:
|
||||
with urllib.request.urlopen(r) as resp:
|
||||
return resp.status, json.loads(resp.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, json.loads(e.read())
|
||||
|
||||
u = "test_" + str(random.randint(1000, 9999))
|
||||
print("Создаём пользователя:", u)
|
||||
s, user = req("/api/v1/users", "POST", {"username": u, "display_name": "Тест", "password": "testpassword1234"})
|
||||
print("Пользователь:", s, user)
|
||||
|
||||
print("Логинимся...")
|
||||
cred = base64.b64encode((u + ":testpassword1234").encode()).decode()
|
||||
s, sess = req("/api/v1/auth/sessions", "POST", headers={"Authorization": "Basic " + cred})
|
||||
print("Сессия:", s)
|
||||
token = sess["token"]
|
||||
|
||||
print("Создаём чат...")
|
||||
s, chat = req("/api/v1/chats", "POST", {"title": "Тест", "member_ids": [user["id"]]}, {"Authorization": "Bearer " + token})
|
||||
print("Чат:", s, chat)
|
||||
Reference in New Issue
Block a user