Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b72fa4b82d |
+25
@@ -0,0 +1,25 @@
|
|||||||
|
FROM python:3.12.12-slim-bookworm
|
||||||
|
|
||||||
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PYTHONUNBUFFERED=1 \
|
||||||
|
POETRY_VIRTUALENVS_CREATE=false
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY poetry.lock pyproject.toml ./
|
||||||
|
RUN pip install --no-cache-dir --timeout 60 --retries 10 "poetry==2.2.1" \
|
||||||
|
&& poetry install --only main --no-root --no-interaction
|
||||||
|
COPY migrations ./migrations
|
||||||
|
COPY src ./src
|
||||||
|
|
||||||
|
# Процесс работает не от root
|
||||||
|
RUN useradd --system --uid 10001 --no-create-home app
|
||||||
|
USER app
|
||||||
|
|
||||||
|
ARG APP_REVISION=dev
|
||||||
|
ENV APP_REVISION=${APP_REVISION}
|
||||||
|
LABEL org.opencontainers.image.revision=${APP_REVISION}
|
||||||
|
|
||||||
|
EXPOSE 8080
|
||||||
|
HEALTHCHECK --interval=10s --timeout=3s --start-period=5s --retries=3 \
|
||||||
|
CMD ["python", "src/healthcheck.py"]
|
||||||
|
CMD ["python", "src/gateway.py"]
|
||||||
@@ -1,37 +1,74 @@
|
|||||||
# Отчёт — лабораторная 4
|
# Отчёт — лабораторная 4
|
||||||
|
|
||||||
- Автор, группа: TODO
|
- Дадайкин Андрей, БВТ2302
|
||||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
- Целевая оценка: 4
|
||||||
- Commit/PR: TODO после публикации
|
- Commit/PR: TODO после публикации
|
||||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
- Использованные источники, библиотеки и помощь ИИ:
|
||||||
|
- - aioboto3` для асинхронной работы с S3 API.
|
||||||
|
- - Использован ИИ (LLM) для код-ревью структуры валидатора изображений
|
||||||
|
-
|
||||||
|
|
||||||
## Запуск из чистого clone
|
## Запуск из чистого clone
|
||||||
|
|
||||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
**Требования:** ОС Windows 11 (или Linux/WSL2), Docker Desktop, Python 3.12, Poetry.
|
||||||
|
|
||||||
|
1. **Настройка окружения:**
|
||||||
|
Скопируйте пример переменных окружения:
|
||||||
|
```cmd
|
||||||
|
copy .env.example .env
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
2. **Запуск инфраструктуры и API:**
|
||||||
|
```cmd
|
||||||
|
docker compose --env-file .env up -d --build
|
||||||
|
```
|
||||||
|
*Контейнер `migrate` автоматически применит миграции к PostgreSQL перед стартом `api`. SeaweedFS поднимется как S3-совместимое хранилище.*
|
||||||
|
|
||||||
|
3. **Локальные тесты (в WSL или Git Bash):**
|
||||||
|
```bash
|
||||||
|
poetry install
|
||||||
|
poetry run make check
|
||||||
|
poetry run make test BASE_URL=http://localhost:8080
|
||||||
|
poetry run pytest student-tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Тестирование отказоустойчивости:**
|
||||||
|
```bash
|
||||||
|
poetry run make test-failover ACTION_SCRIPT=scripts/stop-one.sh BASE_URL=http://localhost:8080
|
||||||
|
poetry run make test-persistence ACTION_SCRIPT=scripts/restart.sh BASE_URL=http://localhost:8080
|
||||||
|
```
|
||||||
|
*(На Windows скрипты `.sh` необходимо запускать через WSL или терминал, поддерживающий bash-скрипты, предварительно выполнив `chmod +x scripts/*.sh`).*
|
||||||
|
|
||||||
|
5. **Остановка и очистка данных:**
|
||||||
|
```cmd
|
||||||
|
docker compose down -v
|
||||||
|
```
|
||||||
|
|
||||||
## Архитектура и решения
|
## Архитектура и решения
|
||||||
|
|
||||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
- **Хранилище байтов (S3):** Использован легковесный SeaweedFS (`chrislusf/seaweedfs:latest`), работающий в режиме S3-шлюза. Сервис изолирован в сети `backend`, публичный доступ закрыт (доступ только через API-реплики).
|
||||||
|
- **Хранилище метаданных:** PostgreSQL (таблица `attachments` с полями `status`, `sha256`, `size_bytes`, `content_type`).
|
||||||
|
- **Связь файлов и сообщений:** Массив `attachment_ids (UUID[])` добавлен в таблицу `messages`.
|
||||||
|
- **Потоковая загрузка:** Реализован endpoint `POST /chats/{chat_id}/attachments`. Данные читаются через `request.stream()` и записываются в `tempfile`.
|
||||||
|
- **Эффективная защита по RAM (Anti-Decompression Bomb):** Парсинг ширины и высоты PNG/JPEG происходит путем ручного чтения бинарных заголовков. Это позволяет отсекать изображения >40 млн пикселей, не декодируя файл в оперативную память.
|
||||||
|
- **Безопасность имен:** Оригинальное имя файла `filename` сохраняется в БД для отображения клиенту. В S3 объект сохраняется под сгенерированным ключом `chat_id/attachment_id`, что исключает атаки типа Path Traversal.
|
||||||
|
|
||||||
## Доказательства критериев
|
## Доказательства критериев
|
||||||
|
|
||||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| Публичный smoke | make test | TODO | TODO |
|
| Публичный smoke | `make test` | Пройден сценарий lab4, SHA-256 скачанного файла совпал. | `tests/smoke.py` |
|
||||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
| Ограничение загрузки (до 10 МБ, mime) | `pytest student-tests/test_attachments.py` | Загрузка TXT под видом PNG возвращает 415. | `student-tests/test_attachments.py` |
|
||||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
| Отсутствие доступа у постороннего | `make test` | Вызов GET-методов контента от 3-го лица выдает 404. | `tests/smoke.py` |
|
||||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
| Restart стенда с сохранением файлов | `make test-persistence` | После `scripts/restart.sh` (БД, Valkey, API, S3) скачанный файл не бьется. | `scripts/restart.sh` |
|
||||||
|
| Отказ API при загрузке (failover) | `make test-failover` | Скачивание файла происходит успешно через соседнюю реплику после остановки первой. | `scripts/stop-one.sh` |
|
||||||
|
| Механизм компенсации БД/S3 | Запуск `scripts/reconcile_s3.py` | Скрипт (запускаемый по cron) находит `queued` файлы старше 24ч и удаляет сирот из S3. | `scripts/reconcile_s3.py` |
|
||||||
|
|
||||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
|
||||||
|
|
||||||
## Эксперименты и отказы
|
|
||||||
|
|
||||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
|
||||||
|
|
||||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| TODO | TODO | TODO | TODO |
|
|
||||||
|
|
||||||
## Ограничения и следующие шаги
|
## Ограничения и следующие шаги
|
||||||
|
|
||||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
- **Ограничения парсера:** Ручной валидатор изображений `validate_image_and_get_mime` поддерживает базовые форматы JPEG и PNG. Прогрессивные форматы (WebP, AVIF) пока отклоняются с ошибкой 415.
|
||||||
|
- **Упрощение реконсиляции:** Текущий механизм реконсилятора (`reconcile_s3.py`) запускается вручную. В production среде его необходимо вынести в Celery Beat или Kubernetes CronJob.
|
||||||
|
- **Следующий шаг:** Переход на асинхронную генерацию миниатюр (thumbnail) и crop вариантов через брокер сообщений (RabbitMQ) для выполнения требований Лабораторной №5.
|
||||||
|
|
||||||
|
|||||||
+188
@@ -0,0 +1,188 @@
|
|||||||
|
name: messenger-lab3
|
||||||
|
|
||||||
|
x-app: &app
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
args:
|
||||||
|
APP_REVISION: ${APP_REVISION:-dev}
|
||||||
|
image: mtusi-messenger:lab3
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
environment: &app-network
|
||||||
|
DB_HOST: postgres
|
||||||
|
DB_PORT: "5432"
|
||||||
|
VALKEY_HOST: valkey
|
||||||
|
VALKEY_PORT: "6379"
|
||||||
|
|
||||||
|
x-api-resources: &api-resources
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: "1.0"
|
||||||
|
memory: 512M
|
||||||
|
reservations:
|
||||||
|
cpus: "0.25"
|
||||||
|
memory: 128M
|
||||||
|
|
||||||
|
x-api: &api
|
||||||
|
<<: *app
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
migrate:
|
||||||
|
condition: service_completed_successfully
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
valkey:
|
||||||
|
condition: service_started
|
||||||
|
networks:
|
||||||
|
- edge
|
||||||
|
- backend
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "python", "src/healthcheck.py"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 2
|
||||||
|
start_period: 20s
|
||||||
|
start_interval: 1s
|
||||||
|
stop_grace_period: 10s
|
||||||
|
|
||||||
|
services:
|
||||||
|
nginx:
|
||||||
|
image: nginx:1.28.0-alpine3.21
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
api:
|
||||||
|
condition: service_healthy
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:${LB_PORT:-8080}:8080"
|
||||||
|
- "127.0.0.1:${LB_EXPIRY_PORT:-8081}:8081"
|
||||||
|
volumes:
|
||||||
|
- ./infra/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
|
||||||
|
- ./infra/nginx/errors.conf:/etc/nginx/errors.conf:ro
|
||||||
|
networks:
|
||||||
|
- edge
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1:8090/healthz"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 2s
|
||||||
|
retries: 3
|
||||||
|
stop_grace_period: 10s
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: "0.5"
|
||||||
|
memory: 64M
|
||||||
|
|
||||||
|
api:
|
||||||
|
<<: *api
|
||||||
|
deploy:
|
||||||
|
<<: *api-resources
|
||||||
|
replicas: ${API_REPLICAS:-2}
|
||||||
|
|
||||||
|
migrate:
|
||||||
|
<<: *app
|
||||||
|
command: ["python", "src/migrate.py"]
|
||||||
|
restart: "no"
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
healthcheck:
|
||||||
|
disable: true
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: "0.5"
|
||||||
|
memory: 256M
|
||||||
|
|
||||||
|
postgres:
|
||||||
|
image: postgres:18.2-alpine3.22
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: ${DB_USER}
|
||||||
|
POSTGRES_PASSWORD: ${DB_PASS}
|
||||||
|
POSTGRES_DB: ${DB_NAME}
|
||||||
|
volumes:
|
||||||
|
- postgres_data:/var/lib/postgresql
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U \"$$POSTGRES_USER\" -d \"$$POSTGRES_DB\""]
|
||||||
|
interval: 2s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 15
|
||||||
|
start_period: 10s
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
stop_grace_period: 30s
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: "1.5"
|
||||||
|
memory: 512M
|
||||||
|
|
||||||
|
valkey:
|
||||||
|
image: valkey/valkey:8.1.4-alpine3.22
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
VALKEY_PASS: ${VALKEY_PASS}
|
||||||
|
command:
|
||||||
|
- valkey-server
|
||||||
|
- --requirepass
|
||||||
|
- ${VALKEY_PASS}
|
||||||
|
- --save
|
||||||
|
- ""
|
||||||
|
- --appendonly
|
||||||
|
- "no"
|
||||||
|
- --maxmemory
|
||||||
|
- 64mb
|
||||||
|
- --maxmemory-policy
|
||||||
|
- allkeys-lru
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "valkey-cli --no-auth-warning -a \"$$VALKEY_PASS\" ping | grep -q PONG"]
|
||||||
|
interval: 2s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 15
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: "0.25"
|
||||||
|
memory: 96M
|
||||||
|
|
||||||
|
api-short-ttl:
|
||||||
|
<<: *api
|
||||||
|
profiles:
|
||||||
|
- expiry
|
||||||
|
environment:
|
||||||
|
<<: *app-network
|
||||||
|
SESSION_TTL_SECONDS: "2"
|
||||||
|
deploy:
|
||||||
|
<<: *api-resources
|
||||||
|
replicas: 1
|
||||||
|
|
||||||
|
s3:
|
||||||
|
image: chrislusf/seaweedfs:latest
|
||||||
|
restart: unless-stopped
|
||||||
|
command: server -s3 -dir=/data -s3.port=9000
|
||||||
|
environment:
|
||||||
|
AWS_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID}
|
||||||
|
AWS_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY}
|
||||||
|
S3_BUCKET: ${S3_BUCKET}
|
||||||
|
volumes:
|
||||||
|
- s3_data:/data
|
||||||
|
networks:
|
||||||
|
- backend
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://localhost:9000/"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 5
|
||||||
|
|
||||||
|
networks:
|
||||||
|
edge: {}
|
||||||
|
backend:
|
||||||
|
internal: true
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
postgres_data:
|
||||||
|
s3_data:
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
# Ошибки, созданные самим балансировщиком, — в формате ошибок контракта.
|
||||||
|
# Ответы реплик (включая их 503) проходят без изменений: proxy_intercept_errors off.
|
||||||
|
# X-Instance-Id здесь нет: ответ создан LB, а не API-репликой.
|
||||||
|
|
||||||
|
location @unavailable {
|
||||||
|
default_type application/json;
|
||||||
|
add_header X-Request-Id $request_id always;
|
||||||
|
return 503 '{"error":{"code":"unavailable","message":"Нет доступной API-реплики","request_id":"$request_id"}}';
|
||||||
|
}
|
||||||
|
|
||||||
|
location @too_large {
|
||||||
|
default_type application/json;
|
||||||
|
add_header X-Request-Id $request_id always;
|
||||||
|
return 413 '{"error":{"code":"payload_too_large","message":"Тело запроса слишком большое","request_id":"$request_id"}}';
|
||||||
|
}
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
worker_processes 1;
|
||||||
|
pid /tmp/nginx.pid;
|
||||||
|
error_log /dev/stderr warn;
|
||||||
|
|
||||||
|
events {
|
||||||
|
worker_connections 1024;
|
||||||
|
}
|
||||||
|
|
||||||
|
http {
|
||||||
|
resolver 127.0.0.11 valid=2s ipv6=off;
|
||||||
|
resolver_timeout 1s;
|
||||||
|
|
||||||
|
log_format upstream_json escape=json
|
||||||
|
'{"time":"$time_iso8601","request_id":"$request_id",'
|
||||||
|
'"method":"$request_method","uri":"$uri","status":$status,'
|
||||||
|
'"request_time":$request_time,"upstream_addr":"$upstream_addr",'
|
||||||
|
'"upstream_status":"$upstream_status",'
|
||||||
|
'"upstream_response_time":"$upstream_response_time",'
|
||||||
|
'"instance":"$upstream_http_x_instance_id"}';
|
||||||
|
access_log /dev/stdout upstream_json;
|
||||||
|
|
||||||
|
server_tokens off;
|
||||||
|
client_max_body_size 1m;
|
||||||
|
client_body_buffer_size 64k;
|
||||||
|
client_header_timeout 10s;
|
||||||
|
client_body_timeout 10s;
|
||||||
|
send_timeout 10s;
|
||||||
|
keepalive_timeout 65s;
|
||||||
|
|
||||||
|
upstream api {
|
||||||
|
zone api 64k;
|
||||||
|
server api:8080 resolve max_fails=1 fail_timeout=5s;
|
||||||
|
keepalive 32;
|
||||||
|
# Меньше APP_KEEPALIVE_TIMEOUT_SEC (75 с): idle-соединение первым
|
||||||
|
# закрывает nginx, а не реплика, — нет гонки «запрос в закрытое соединение»
|
||||||
|
keepalive_timeout 60s;
|
||||||
|
}
|
||||||
|
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Connection "";
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
# Один ID на LB и реплику: API принимает $request_id и возвращает его
|
||||||
|
proxy_set_header X-Request-Id $request_id;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
|
||||||
|
# connect внутри bridge-сети завершает ядро за доли мс даже у занятой реплики.
|
||||||
|
# Самый долгий запрос API: ожидание пула 3 с + statement_timeout 5 с
|
||||||
|
proxy_connect_timeout 500ms;
|
||||||
|
proxy_send_timeout 10s;
|
||||||
|
proxy_read_timeout 10s;
|
||||||
|
|
||||||
|
# Повтор на другой реплике только при ошибке соединения/таймауте.
|
||||||
|
# POST, уже отправленный реплике, nginx не повторяет (non_idempotent выключен):
|
||||||
|
# повтор сообщения делает клиент с тем же Idempotency-Key.
|
||||||
|
# 503 приложения (общая зависимость недоступна) не повторяется и не исключает
|
||||||
|
# реплику: другая реплика зависит от той же БД.
|
||||||
|
proxy_next_upstream error timeout;
|
||||||
|
proxy_next_upstream_tries 3;
|
||||||
|
proxy_next_upstream_timeout 5s;
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 8080;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://api;
|
||||||
|
}
|
||||||
|
|
||||||
|
error_page 502 503 504 = @unavailable;
|
||||||
|
error_page 413 = @too_large;
|
||||||
|
|
||||||
|
include /etc/nginx/errors.conf;
|
||||||
|
}
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 8081;
|
||||||
|
|
||||||
|
# Реплика api-short-ttl только для make test-expiry (профиль expiry):
|
||||||
|
# адрес резолвится на каждый запрос, без профиля — 503 от LB
|
||||||
|
location / {
|
||||||
|
set $api_short_ttl http://api-short-ttl:8080;
|
||||||
|
proxy_pass $api_short_ttl;
|
||||||
|
}
|
||||||
|
|
||||||
|
error_page 502 503 504 = @unavailable;
|
||||||
|
error_page 413 = @too_large;
|
||||||
|
|
||||||
|
include /etc/nginx/errors.conf;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Liveness самого балансировщика для healthcheck контейнера; наружу не публикуется
|
||||||
|
server {
|
||||||
|
listen 127.0.0.1:8090;
|
||||||
|
access_log off;
|
||||||
|
|
||||||
|
location = /healthz {
|
||||||
|
default_type text/plain;
|
||||||
|
return 200 "ok\n";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
CREATE TABLE users (
|
||||||
|
id uuid PRIMARY KEY DEFAULT uuidv7(),
|
||||||
|
username text NOT NULL,
|
||||||
|
display_name text NOT NULL,
|
||||||
|
password_hash text NOT NULL,
|
||||||
|
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||||
|
CONSTRAINT users_username_key UNIQUE (username),
|
||||||
|
CONSTRAINT users_username_format CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||||
|
CONSTRAINT users_display_name_len CHECK (char_length(display_name) BETWEEN 1 AND 100)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Источник истины для сессий; токен хранится только как SHA-256
|
||||||
|
CREATE TABLE sessions (
|
||||||
|
token_hash bytea PRIMARY KEY,
|
||||||
|
user_id uuid NOT NULL REFERENCES users (id) ON DELETE CASCADE,
|
||||||
|
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||||
|
expires_at timestamptz NOT NULL,
|
||||||
|
revoked_at timestamptz,
|
||||||
|
CONSTRAINT sessions_token_hash_len CHECK (octet_length(token_hash) = 32),
|
||||||
|
CONSTRAINT sessions_expiry_order CHECK (expires_at > created_at)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX sessions_user_id_idx ON sessions (user_id);
|
||||||
|
|
||||||
|
CREATE TABLE chats (
|
||||||
|
id uuid PRIMARY KEY DEFAULT uuidv7(),
|
||||||
|
title text NOT NULL,
|
||||||
|
created_by uuid NOT NULL REFERENCES users (id),
|
||||||
|
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||||
|
CONSTRAINT chats_title_len CHECK (char_length(title) BETWEEN 1 AND 100)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- position сохраняет порядок member_ids из запроса создания
|
||||||
|
CREATE TABLE chat_members (
|
||||||
|
chat_id uuid NOT NULL REFERENCES chats (id) ON DELETE CASCADE,
|
||||||
|
user_id uuid NOT NULL REFERENCES users (id),
|
||||||
|
position smallint NOT NULL,
|
||||||
|
PRIMARY KEY (chat_id, user_id),
|
||||||
|
CONSTRAINT chat_members_position_key UNIQUE (chat_id, position)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Список чатов пользователя: WHERE user_id = $1
|
||||||
|
CREATE INDEX chat_members_user_id_chat_id_idx ON chat_members (user_id, chat_id);
|
||||||
|
|
||||||
|
-- created_at назначает приложение под блокировкой чата (строго возрастает в чате)
|
||||||
|
CREATE TABLE messages (
|
||||||
|
id uuid PRIMARY KEY DEFAULT uuidv7(),
|
||||||
|
chat_id uuid NOT NULL,
|
||||||
|
sender_id uuid NOT NULL,
|
||||||
|
text text NOT NULL,
|
||||||
|
created_at timestamptz NOT NULL,
|
||||||
|
CONSTRAINT messages_text_len CHECK (char_length(text) BETWEEN 1 AND 4000),
|
||||||
|
-- отправитель обязан быть участником этого чата
|
||||||
|
CONSTRAINT messages_sender_member_fkey FOREIGN KEY (chat_id, sender_id)
|
||||||
|
REFERENCES chat_members (chat_id, user_id) ON DELETE CASCADE
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Keyset-пагинация истории: WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
|
||||||
|
-- ORDER BY created_at, id LIMIT $4 — Index Scan без сортировки и полного прохода
|
||||||
|
CREATE INDEX messages_chat_id_created_at_id_idx ON messages (chat_id, created_at, id);
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
-- Idempotency-Key отправки сообщения (lab3). Область по контракту:
|
||||||
|
-- пользователь + метод + путь + ключ. Метод и путь фиксированы
|
||||||
|
-- (POST /api/v1/chats/{chat_id}/messages), поэтому ключ — (sender_id, chat_id, key).
|
||||||
|
-- Только добавление таблицы (expand): предыдущая версия API работает на новой схеме.
|
||||||
|
CREATE TABLE message_idempotency_keys (
|
||||||
|
sender_id uuid NOT NULL,
|
||||||
|
chat_id uuid NOT NULL,
|
||||||
|
idempotency_key text NOT NULL,
|
||||||
|
request_hash bytea NOT NULL,
|
||||||
|
message_id uuid NOT NULL REFERENCES messages (id) ON DELETE CASCADE,
|
||||||
|
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||||
|
expires_at timestamptz NOT NULL,
|
||||||
|
PRIMARY KEY (sender_id, chat_id, idempotency_key),
|
||||||
|
CONSTRAINT message_idempotency_keys_key_format
|
||||||
|
CHECK (idempotency_key ~ '^[A-Za-z0-9._:-]{1,128}$'),
|
||||||
|
CONSTRAINT message_idempotency_keys_hash_len CHECK (octet_length(request_hash) = 32),
|
||||||
|
CONSTRAINT message_idempotency_keys_expiry_order CHECK (expires_at > created_at)
|
||||||
|
);
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
-- Таблица для метаданных вложений
|
||||||
|
CREATE TABLE attachments (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
|
||||||
|
owner_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
filename TEXT NOT NULL,
|
||||||
|
content_type TEXT NOT NULL,
|
||||||
|
size_bytes BIGINT NOT NULL,
|
||||||
|
sha256 TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
error_code TEXT,
|
||||||
|
variants JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp(),
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp()
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Добавляем колонку для прикрепленных файлов в сообщения
|
||||||
|
ALTER TABLE messages ADD COLUMN attachment_ids UUID[] NOT NULL DEFAULT '{}'::UUID[];
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
-- Удаляем старое ограничение, которое запрещало пустой текст
|
||||||
|
ALTER TABLE messages DROP CONSTRAINT IF EXISTS messages_text_len;
|
||||||
|
|
||||||
|
-- Добавляем новое: текст может быть пустым ТОЛЬКО ЕСЛИ есть хотя бы одно вложение
|
||||||
|
ALTER TABLE messages ADD CONSTRAINT messages_text_len CHECK (
|
||||||
|
(char_length(text) >= 1 OR cardinality(attachment_ids) >= 1)
|
||||||
|
AND char_length(text) <= 4000
|
||||||
|
);
|
||||||
Generated
+1964
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,31 @@
|
|||||||
|
[tool.poetry]
|
||||||
|
name = "max2"
|
||||||
|
version = "1.2.0"
|
||||||
|
description = "Мессенджер на PostgreSQL и Valkey, лабораторная 3"
|
||||||
|
authors = ["Саватеев Артём"]
|
||||||
|
package-mode = false
|
||||||
|
|
||||||
|
[tool.poetry.dependencies]
|
||||||
|
python = ">=3.12,<3.13"
|
||||||
|
fastapi = "0.142.2"
|
||||||
|
uvicorn = "0.54.0"
|
||||||
|
pydantic = "2.13.5"
|
||||||
|
envparse = "0.2.0"
|
||||||
|
loguru = "0.7.3"
|
||||||
|
asyncpg = "0.31.0"
|
||||||
|
redis = "6.4.0"
|
||||||
|
argon2-cffi = "25.1.0"
|
||||||
|
aioboto3 = "^13.0.0"
|
||||||
|
python-multipart = "^0.0.18"
|
||||||
|
|
||||||
|
[tool.poetry.group.dev.dependencies]
|
||||||
|
pytest = "9.1.1"
|
||||||
|
httpx = "0.28.1"
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["poetry-core>=2.0.0,<3.0.0"]
|
||||||
|
build-backend = "poetry.core.masonry.api"
|
||||||
|
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
testpaths = ["student-tests"]
|
||||||
|
addopts = "-v"
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
# Миграции с нуля и повторно, отказ API на непримененной схеме: отдельный
|
||||||
|
# compose-проект с собственным пустым volume. Рабочий стенд и его данные
|
||||||
|
# не затрагиваются; временный volume этого проекта удаляется в конце.
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
PROJECT=messenger-lab3-migrations-check
|
||||||
|
# Публикуется только порт LB: свои порты, чтобы не конфликтовать с основным стендом
|
||||||
|
export LB_PORT="${MIGRATIONS_CHECK_LB_PORT:-58080}"
|
||||||
|
export LB_EXPIRY_PORT="${MIGRATIONS_CHECK_LB_EXPIRY_PORT:-58081}"
|
||||||
|
COMPOSE="docker compose --env-file ${ENV_FILE:-.env} -p $PROJECT"
|
||||||
|
trap '$COMPOSE down -v --remove-orphans >/dev/null 2>&1' EXIT
|
||||||
|
|
||||||
|
$COMPOSE up -d --wait postgres >/dev/null 2>&1
|
||||||
|
# Тот же образ и сеть, что у сервиса migrate; apply_migrations трижды подряд
|
||||||
|
$COMPOSE run --rm --no-deps -T migrate python - <<'EOF'
|
||||||
|
import asyncio
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.path.insert(0, "src")
|
||||||
|
|
||||||
|
from infrastructure.dependencies.database import database_pool_manager as db
|
||||||
|
|
||||||
|
|
||||||
|
async def main() -> None:
|
||||||
|
await db.create_pool(30)
|
||||||
|
async with db.acquire() as conn:
|
||||||
|
tables = await conn.fetchval(
|
||||||
|
"SELECT count(*) FROM pg_tables WHERE schemaname = 'public';"
|
||||||
|
)
|
||||||
|
print(f"пустой volume: таблиц в public = {tables}")
|
||||||
|
# Так стартует API-реплика: без migrate она отказывается работать
|
||||||
|
try:
|
||||||
|
await db.check_migrations()
|
||||||
|
except RuntimeError as error:
|
||||||
|
print(f"API до migrate: {error}")
|
||||||
|
else:
|
||||||
|
raise SystemExit("FAIL: API стартовал бы на пустой схеме")
|
||||||
|
for attempt in (1, 2, 3):
|
||||||
|
await db.apply_migrations()
|
||||||
|
async with db.acquire() as conn:
|
||||||
|
versions = await conn.fetch(
|
||||||
|
"SELECT version, name FROM schema_migrations ORDER BY version;"
|
||||||
|
)
|
||||||
|
tables = await conn.fetchval(
|
||||||
|
"SELECT count(*) FROM pg_tables WHERE schemaname = 'public';"
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
f"запуск {attempt}: schema_migrations = "
|
||||||
|
f"{[(r['version'], r['name']) for r in versions]}, таблиц = {tables}"
|
||||||
|
)
|
||||||
|
await db.check_migrations()
|
||||||
|
print("API после migrate: схема совместима")
|
||||||
|
await db.close()
|
||||||
|
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
EOF
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
# Совместимость текущей схемы с предыдущей версией API — условие rolling update:
|
||||||
|
# миграции применяются до замены реплик, и старые реплики продолжают работать.
|
||||||
|
# Образ собирается из коммита PREVIOUS_REVISION (по умолчанию lab2-код aa971fb),
|
||||||
|
# запускается одним процессом в сети backend рабочего стенда (наружу порт
|
||||||
|
# не публикуется) и выполняет вход, создание чата, запись и чтение сообщения.
|
||||||
|
# Создаются только синтетические данные; volumes не затрагиваются.
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
ENV_FILE="${ENV_FILE:-.env}"
|
||||||
|
REVISION="${PREVIOUS_REVISION:-aa971fb}"
|
||||||
|
IMAGE="mtusi-messenger:compat-$REVISION"
|
||||||
|
NETWORK=messenger-lab3_backend
|
||||||
|
|
||||||
|
git archive "$REVISION" | docker build --quiet --tag "$IMAGE" - >/dev/null
|
||||||
|
echo "previous version image: $IMAGE ($(git log -1 --format='%h %s' "$REVISION"))"
|
||||||
|
echo "schema on stand: $(docker compose --env-file "$ENV_FILE" exec -T postgres sh -c \
|
||||||
|
'psql -X -q -tA -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT string_agg(name, \$\$, \$\$ ORDER BY version) FROM schema_migrations;"')"
|
||||||
|
|
||||||
|
docker run --rm -i --network "$NETWORK" --env-file "$ENV_FILE" \
|
||||||
|
-e DB_HOST=postgres -e DB_PORT=5432 -e VALKEY_HOST=valkey -e VALKEY_PORT=6379 \
|
||||||
|
"$IMAGE" sh -c 'python src/gateway.py >/tmp/api.log 2>&1 & exec python -' <<'EOF'
|
||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
BASE = "http://127.0.0.1:8080"
|
||||||
|
|
||||||
|
|
||||||
|
def call(method, path, body=None, headers=None):
|
||||||
|
data = None if body is None else json.dumps(body).encode()
|
||||||
|
request = urllib.request.Request(BASE + path, data=data, method=method)
|
||||||
|
request.add_header("Content-Type", "application/json")
|
||||||
|
for name, value in (headers or {}).items():
|
||||||
|
request.add_header(name, value)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=5) as response:
|
||||||
|
return response.status, json.loads(response.read() or b"null")
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
return error.code, None
|
||||||
|
|
||||||
|
|
||||||
|
for _ in range(60):
|
||||||
|
try:
|
||||||
|
if call("GET", "/health/ready")[0] == 200:
|
||||||
|
break
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
time.sleep(0.5)
|
||||||
|
else:
|
||||||
|
print(open("/tmp/api.log").read()[-2000:])
|
||||||
|
raise SystemExit("FAIL: previous version did not become ready on current schema")
|
||||||
|
|
||||||
|
username = "compat_" + uuid.uuid4().hex[:12]
|
||||||
|
password = "Compat-" + uuid.uuid4().hex
|
||||||
|
status_user, user = call(
|
||||||
|
"POST", "/api/v1/users",
|
||||||
|
{"username": username, "display_name": "Compat", "password": password},
|
||||||
|
)
|
||||||
|
credentials = base64.b64encode(f"{username}:{password}".encode()).decode()
|
||||||
|
status_login, session = call(
|
||||||
|
"POST", "/api/v1/auth/sessions", headers={"Authorization": "Basic " + credentials}
|
||||||
|
)
|
||||||
|
bearer = {"Authorization": "Bearer " + session["token"]}
|
||||||
|
status_chat, chat = call(
|
||||||
|
"POST", "/api/v1/chats", {"title": "compat", "member_ids": [user["id"]]}, bearer
|
||||||
|
)
|
||||||
|
path = f"/api/v1/chats/{chat['id']}/messages"
|
||||||
|
status_send, message = call("POST", path, {"text": "старая версия пишет"}, bearer)
|
||||||
|
status_read, page = call("GET", path, headers=bearer)
|
||||||
|
result = {
|
||||||
|
"ready": True,
|
||||||
|
"register": status_user,
|
||||||
|
"login": status_login,
|
||||||
|
"create_chat": status_chat,
|
||||||
|
"send_message_without_key": status_send,
|
||||||
|
"read_history": status_read,
|
||||||
|
"history_matches": page["items"] == [message],
|
||||||
|
}
|
||||||
|
print(json.dumps(result, ensure_ascii=False))
|
||||||
|
log = open("/tmp/api.log").read()
|
||||||
|
print("startup log:", [line.split(" - ", 1)[-1] for line in log.splitlines() if "миграц" in line.lower() or "Схема" in line])
|
||||||
|
if any(code >= 400 for code in result.values() if isinstance(code, int)) or not result["history_matches"]:
|
||||||
|
raise SystemExit("FAIL: previous version is not compatible with current schema")
|
||||||
|
print("PASS: previous API version works on the current schema (expand-only migration)")
|
||||||
|
EOF
|
||||||
@@ -0,0 +1,480 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Нагрузка через LB во время отказа/замены API-реплик; метрики и проверка данных.
|
||||||
|
|
||||||
|
failover — для каждой API-реплики по очереди: нагрузка → остановка этой реплики
|
||||||
|
(docker stop = SIGTERM, затем docker kill = SIGKILL) → восстановление
|
||||||
|
(scripts/restore.sh) → нагрузка после восстановления.
|
||||||
|
rolling — нагрузка во время scripts/rolling-update.sh (поочерёдная замена реплик).
|
||||||
|
|
||||||
|
Клиент ведёт себя как настоящий: неподтверждённый POST повторяет с тем же
|
||||||
|
Idempotency-Key и телом. После прогона вся история читается заново: каждое
|
||||||
|
подтверждённое сообщение на месте и не изменено, дублей нет.
|
||||||
|
Python 3.9+, стандартная библиотека; tokens не печатаются и не сохраняются.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import math
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
import uuid
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import quote
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(ROOT / "tests"))
|
||||||
|
|
||||||
|
from client import Client # noqa: E402
|
||||||
|
|
||||||
|
REQUEST_TIMEOUT_SEC = 5
|
||||||
|
RETRY_PAUSE_SEC = 0.05
|
||||||
|
STATS_FORMAT = "{{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Attempt:
|
||||||
|
"""Одна HTTP-попытка клиента; время — секунды от начала прогона"""
|
||||||
|
|
||||||
|
op: str
|
||||||
|
started: float
|
||||||
|
finished: float
|
||||||
|
status: int | None
|
||||||
|
error: str | None
|
||||||
|
instance: str | None
|
||||||
|
retry: bool
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ok(self) -> bool:
|
||||||
|
return self.status in (200, 201)
|
||||||
|
|
||||||
|
|
||||||
|
class Recorder:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.started = time.monotonic()
|
||||||
|
self.lock = threading.Lock()
|
||||||
|
self.attempts: list[Attempt] = []
|
||||||
|
self.confirmed: list[dict] = []
|
||||||
|
self.events: dict[str, float] = {}
|
||||||
|
|
||||||
|
def now(self) -> float:
|
||||||
|
return time.monotonic() - self.started
|
||||||
|
|
||||||
|
def mark(self, name: str) -> None:
|
||||||
|
self.events[name] = round(self.now(), 3)
|
||||||
|
|
||||||
|
def add(self, attempt: Attempt, message: dict | None = None) -> None:
|
||||||
|
with self.lock:
|
||||||
|
self.attempts.append(attempt)
|
||||||
|
if message is not None:
|
||||||
|
self.confirmed.append(message)
|
||||||
|
|
||||||
|
|
||||||
|
def request(
|
||||||
|
base: str, method: str, path: str, token: str, body: dict | None, key: str | None
|
||||||
|
) -> tuple[int | None, str | None, str | None, dict | None]:
|
||||||
|
headers = {"Authorization": "Bearer " + token, "Accept": "application/json"}
|
||||||
|
data = None
|
||||||
|
if body is not None:
|
||||||
|
data = json.dumps(body, ensure_ascii=False).encode()
|
||||||
|
headers["Content-Type"] = "application/json"
|
||||||
|
if key is not None:
|
||||||
|
headers["Idempotency-Key"] = key
|
||||||
|
req = urllib.request.Request(base + path, data=data, headers=headers, method=method)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, timeout=REQUEST_TIMEOUT_SEC) as response:
|
||||||
|
payload = json.loads(response.read())
|
||||||
|
return response.status, None, response.headers.get("X-Instance-Id"), payload
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
error.read()
|
||||||
|
return error.code, None, error.headers.get("X-Instance-Id"), None
|
||||||
|
except OSError as error:
|
||||||
|
return None, type(error).__name__, None, None
|
||||||
|
|
||||||
|
|
||||||
|
def worker(
|
||||||
|
base: str,
|
||||||
|
token: str,
|
||||||
|
chat_id: str,
|
||||||
|
worker_id: int,
|
||||||
|
recorder: Recorder,
|
||||||
|
stop: threading.Event,
|
||||||
|
) -> None:
|
||||||
|
"""Запись с повтором по тому же ключу до подтверждения, затем чтение"""
|
||||||
|
path = f"/api/v1/chats/{chat_id}/messages"
|
||||||
|
index = 0
|
||||||
|
while not stop.is_set():
|
||||||
|
key = uuid.uuid4().hex
|
||||||
|
body = {"text": f"exp-{worker_id}-{index}-{key[:8]}"}
|
||||||
|
retry = False
|
||||||
|
while not stop.is_set():
|
||||||
|
started = recorder.now()
|
||||||
|
status, error, instance, payload = request(
|
||||||
|
base, "POST", path, token, body, key
|
||||||
|
)
|
||||||
|
attempt = Attempt(
|
||||||
|
"write", started, recorder.now(), status, error, instance, retry
|
||||||
|
)
|
||||||
|
recorder.add(attempt, payload if status == 201 else None)
|
||||||
|
if status == 201:
|
||||||
|
break
|
||||||
|
retry = True
|
||||||
|
time.sleep(RETRY_PAUSE_SEC)
|
||||||
|
started = recorder.now()
|
||||||
|
status, error, instance, _ = request(
|
||||||
|
base, "GET", path + "?limit=1", token, None, None
|
||||||
|
)
|
||||||
|
recorder.add(
|
||||||
|
Attempt("read", started, recorder.now(), status, error, instance, False)
|
||||||
|
)
|
||||||
|
index += 1
|
||||||
|
|
||||||
|
|
||||||
|
def percentile(values: list[float], p: float) -> float | None:
|
||||||
|
if not values:
|
||||||
|
return None
|
||||||
|
ordered = sorted(values)
|
||||||
|
return round(1000 * ordered[max(0, math.ceil(p * len(ordered)) - 1)], 1)
|
||||||
|
|
||||||
|
|
||||||
|
def phase_stats(attempts: list[Attempt], start: float, end: float) -> dict:
|
||||||
|
inside = [a for a in attempts if start <= a.finished < end]
|
||||||
|
duration = max(end - start, 1e-9)
|
||||||
|
result: dict = {"seconds": round(duration, 3)}
|
||||||
|
for op in ("write", "read"):
|
||||||
|
ops = [a for a in inside if a.op == op]
|
||||||
|
ok = [a for a in ops if a.ok]
|
||||||
|
result[op] = {
|
||||||
|
"ok": len(ok),
|
||||||
|
"errors": len(ops) - len(ok),
|
||||||
|
"ok_rps": round(len(ok) / duration, 1),
|
||||||
|
"latency_ms": {
|
||||||
|
"p50": percentile([a.finished - a.started for a in ok], 0.50),
|
||||||
|
"p95": percentile([a.finished - a.started for a in ok], 0.95),
|
||||||
|
"p99": percentile([a.finished - a.started for a in ok], 0.99),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def recovery_stats(attempts: list[Attempt], action_start: float, end: float) -> dict:
|
||||||
|
"""
|
||||||
|
Время восстановления записи: от первой ошибки после воздействия до первой
|
||||||
|
успешной записи после последней ошибки; без ошибок — 0. Отдельно —
|
||||||
|
наибольший разрыв между подтверждениями записи (задержка без ошибок).
|
||||||
|
"""
|
||||||
|
window = sorted(
|
||||||
|
(a for a in attempts if a.op == "write" and a.finished >= action_start - 1),
|
||||||
|
key=lambda a: a.finished,
|
||||||
|
)
|
||||||
|
errors = [a for a in window if not a.ok and a.finished >= action_start]
|
||||||
|
kinds: dict[str, int] = {}
|
||||||
|
for a in errors:
|
||||||
|
kind = a.error or f"HTTP {a.status}"
|
||||||
|
kinds[kind] = kinds.get(kind, 0) + 1
|
||||||
|
if errors:
|
||||||
|
last_error = errors[-1].finished
|
||||||
|
first_ok_after = next(
|
||||||
|
(a.finished for a in window if a.ok and a.finished > last_error), end
|
||||||
|
)
|
||||||
|
recovery = first_ok_after - errors[0].finished
|
||||||
|
else:
|
||||||
|
recovery = 0.0
|
||||||
|
oks = [a.finished for a in window if a.ok and a.finished <= end]
|
||||||
|
gap = max((b - a for a, b in zip(oks, oks[1:])), default=None)
|
||||||
|
return {
|
||||||
|
"write_errors": len(errors),
|
||||||
|
"error_kinds": kinds,
|
||||||
|
"write_recovery_seconds": round(recovery, 3),
|
||||||
|
"max_write_gap_seconds": None if gap is None else round(gap, 3),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def timeline(attempts: list[Attempt], end: float) -> list[dict]:
|
||||||
|
rows = []
|
||||||
|
for second in range(int(end) + 1):
|
||||||
|
inside = [a for a in attempts if second <= a.finished < second + 1]
|
||||||
|
instances: dict[str, int] = {}
|
||||||
|
for a in inside:
|
||||||
|
if a.ok and a.instance:
|
||||||
|
instances[a.instance] = instances.get(a.instance, 0) + 1
|
||||||
|
rows.append(
|
||||||
|
{
|
||||||
|
"t": second,
|
||||||
|
"ok": sum(a.ok for a in inside),
|
||||||
|
"errors": sum(not a.ok for a in inside),
|
||||||
|
"instances": instances,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def read_history(client: Client, actor: dict, chat_id: str) -> list[dict]:
|
||||||
|
path = f"/api/v1/chats/{chat_id}/messages?limit=100"
|
||||||
|
items: list[dict] = []
|
||||||
|
while True:
|
||||||
|
page = client.call("GET", path, actor=actor, schema="MessagePage")["json"]
|
||||||
|
items.extend(page["items"])
|
||||||
|
if page["next_cursor"] is None:
|
||||||
|
return items
|
||||||
|
path = f"/api/v1/chats/{chat_id}/messages?limit=100&cursor=" + quote(
|
||||||
|
page["next_cursor"], safe=""
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def integrity(client: Client, actor: dict, chat_id: str, recorder: Recorder) -> dict:
|
||||||
|
history = read_history(client, actor, chat_id)
|
||||||
|
by_id = {m["id"]: m for m in history}
|
||||||
|
texts = [m["text"] for m in history]
|
||||||
|
confirmed = {m["id"]: m for m in recorder.confirmed}
|
||||||
|
return {
|
||||||
|
"messages_in_history": len(history),
|
||||||
|
"confirmed_unique": len(confirmed),
|
||||||
|
"confirmed_missing_or_changed": sum(
|
||||||
|
by_id.get(i) != m for i, m in confirmed.items()
|
||||||
|
),
|
||||||
|
"duplicate_ids": len(history) - len(by_id),
|
||||||
|
"duplicate_texts": len(texts) - len(set(texts)),
|
||||||
|
"retried_writes_confirmed_as_replay": len(recorder.confirmed) - len(confirmed),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run(command: list[str], env: dict | None = None) -> str:
|
||||||
|
result = subprocess.run(
|
||||||
|
command, cwd=ROOT, env=env, check=True, capture_output=True, text=True
|
||||||
|
)
|
||||||
|
return result.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def api_containers() -> list[str]:
|
||||||
|
ids = run(["docker", "compose", "--env-file", ".env", "ps", "-q", "api"]).split()
|
||||||
|
names = [run(["docker", "inspect", "-f", "{{.Name}}", i]).lstrip("/") for i in ids]
|
||||||
|
return sorted(names)
|
||||||
|
|
||||||
|
|
||||||
|
def instance_of(container: str) -> str:
|
||||||
|
return run(["docker", "exec", container, "cat", "/tmp/instance-id"])
|
||||||
|
|
||||||
|
|
||||||
|
def container_stats() -> list[str]:
|
||||||
|
names = ["messenger-lab3-nginx-1", "messenger-lab3-postgres-1", *api_containers()]
|
||||||
|
output = run(["docker", "stats", "--no-stream", "--format", STATS_FORMAT, *names])
|
||||||
|
return output.splitlines()
|
||||||
|
|
||||||
|
|
||||||
|
def scenario(
|
||||||
|
args: argparse.Namespace,
|
||||||
|
label: str,
|
||||||
|
action: list[str],
|
||||||
|
restore: list[str] | None,
|
||||||
|
target: str | None = None,
|
||||||
|
env: dict | None = None,
|
||||||
|
) -> dict:
|
||||||
|
client = Client(args.base_url, timeout=REQUEST_TIMEOUT_SEC)
|
||||||
|
alice, _, _, chat = client.bootstrap()
|
||||||
|
base = client.base
|
||||||
|
recorder = Recorder()
|
||||||
|
stop = threading.Event()
|
||||||
|
threads = [
|
||||||
|
threading.Thread(
|
||||||
|
target=worker,
|
||||||
|
args=(base, alice["token"], chat["id"], n, recorder, stop),
|
||||||
|
daemon=True,
|
||||||
|
)
|
||||||
|
for n in range(args.workers)
|
||||||
|
]
|
||||||
|
for thread in threads:
|
||||||
|
thread.start()
|
||||||
|
time.sleep(args.before / 2)
|
||||||
|
stats = container_stats()
|
||||||
|
time.sleep(args.before / 2)
|
||||||
|
|
||||||
|
instances_before = sorted(
|
||||||
|
{a.instance for a in recorder.attempts if a.ok and a.instance}
|
||||||
|
)
|
||||||
|
recorder.mark("action_start")
|
||||||
|
action_output = run(action, env)
|
||||||
|
recorder.mark("action_end")
|
||||||
|
restore_output = ""
|
||||||
|
if restore is not None:
|
||||||
|
time.sleep(args.outage)
|
||||||
|
recorder.mark("restore_start")
|
||||||
|
restore_output = run(restore)
|
||||||
|
recorder.mark("restore_end")
|
||||||
|
time.sleep(args.after)
|
||||||
|
stop.set()
|
||||||
|
for thread in threads:
|
||||||
|
thread.join(timeout=REQUEST_TIMEOUT_SEC + 1)
|
||||||
|
recorder.mark("end")
|
||||||
|
|
||||||
|
events = recorder.events
|
||||||
|
end = events["end"]
|
||||||
|
after_start = events.get("restore_end", events["action_end"])
|
||||||
|
attempts = recorder.attempts
|
||||||
|
seen_after = [
|
||||||
|
a for a in attempts if a.ok and a.finished >= after_start and a.instance
|
||||||
|
]
|
||||||
|
back_in_rotation = next(
|
||||||
|
(
|
||||||
|
round(a.finished - after_start, 3)
|
||||||
|
for a in sorted(seen_after, key=lambda a: a.finished)
|
||||||
|
if a.instance not in instances_before
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
served = sorted(
|
||||||
|
(a for a in attempts if a.ok and a.instance), key=lambda a: a.finished
|
||||||
|
)
|
||||||
|
instances_seen: dict[str, list[float]] = {}
|
||||||
|
for a in served:
|
||||||
|
offset = round(a.finished - events["action_start"], 3)
|
||||||
|
instances_seen.setdefault(a.instance, [offset, offset])[1] = offset
|
||||||
|
phases = {
|
||||||
|
"before": phase_stats(attempts, 0, events["action_start"]),
|
||||||
|
"during": phase_stats(attempts, events["action_start"], after_start),
|
||||||
|
"after": phase_stats(attempts, after_start, end),
|
||||||
|
}
|
||||||
|
target_after_stop = None
|
||||||
|
if target is not None:
|
||||||
|
target_after_stop = sum(
|
||||||
|
1
|
||||||
|
for a in attempts
|
||||||
|
if a.ok
|
||||||
|
and a.instance == target
|
||||||
|
and a.finished > events["action_end"]
|
||||||
|
and a.finished < events.get("restore_start", end)
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"scenario": label,
|
||||||
|
"target_instance": target,
|
||||||
|
"workers": args.workers,
|
||||||
|
"events_seconds": events,
|
||||||
|
"instances_before": instances_before,
|
||||||
|
"instances_after": sorted({a.instance for a in seen_after}),
|
||||||
|
"target_responses_while_stopped": target_after_stop,
|
||||||
|
"restored_replica_in_rotation_seconds": back_in_rotation
|
||||||
|
if restore is not None
|
||||||
|
else None,
|
||||||
|
# alias → [первый, последний] успешный ответ, секунды от начала воздействия
|
||||||
|
"instances_seen_from_action": instances_seen,
|
||||||
|
"phases": phases,
|
||||||
|
"recovery": recovery_stats(attempts, events["action_start"], end),
|
||||||
|
"integrity": integrity(client, alice, chat["id"], recorder),
|
||||||
|
"container_stats_before_action": stats,
|
||||||
|
"action_output": action_output.splitlines(),
|
||||||
|
"restore_output_tail": restore_output.splitlines()[-2:],
|
||||||
|
"timeline": timeline(attempts, end),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def failover(args: argparse.Namespace) -> list[dict]:
|
||||||
|
results = []
|
||||||
|
for signal in args.signals:
|
||||||
|
for container in api_containers():
|
||||||
|
target = instance_of(container)
|
||||||
|
if signal == "SIGTERM":
|
||||||
|
action = ["docker", "stop", "--timeout", "10", container]
|
||||||
|
else:
|
||||||
|
action = ["docker", "kill", "--signal", "SIGKILL", container]
|
||||||
|
label = f"{signal} {container}"
|
||||||
|
print(f"== {label} (instance {target})", flush=True)
|
||||||
|
result = scenario(
|
||||||
|
args, label, action, ["./scripts/restore.sh"], target=target
|
||||||
|
)
|
||||||
|
print(json.dumps(brief(result), ensure_ascii=False), flush=True)
|
||||||
|
results.append(result)
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def rolling(args: argparse.Namespace) -> list[dict]:
|
||||||
|
env = dict(os.environ)
|
||||||
|
env.setdefault("APP_REVISION", f"exp-{int(time.time())}")
|
||||||
|
print(f"== rolling update to revision {env['APP_REVISION']}", flush=True)
|
||||||
|
result = scenario(
|
||||||
|
args, "rolling update", ["./scripts/rolling-update.sh"], None, env=env
|
||||||
|
)
|
||||||
|
result["revisions_after"] = run(["./scripts/instances.sh"]).splitlines()
|
||||||
|
print(json.dumps(brief(result), ensure_ascii=False), flush=True)
|
||||||
|
return [result]
|
||||||
|
|
||||||
|
|
||||||
|
def brief(result: dict) -> dict:
|
||||||
|
"""Краткая строка прогона для терминала; полная версия — в --output"""
|
||||||
|
return {
|
||||||
|
"scenario": result["scenario"],
|
||||||
|
"rps_ok_write": {k: v["write"]["ok_rps"] for k, v in result["phases"].items()},
|
||||||
|
"rps_ok_read": {k: v["read"]["ok_rps"] for k, v in result["phases"].items()},
|
||||||
|
"errors": {
|
||||||
|
k: v["write"]["errors"] + v["read"]["errors"]
|
||||||
|
for k, v in result["phases"].items()
|
||||||
|
},
|
||||||
|
"recovery": result["recovery"],
|
||||||
|
"target_responses_while_stopped": result["target_responses_while_stopped"],
|
||||||
|
"restored_replica_in_rotation_seconds": result[
|
||||||
|
"restored_replica_in_rotation_seconds"
|
||||||
|
],
|
||||||
|
"instances_seen_from_action": result["instances_seen_from_action"],
|
||||||
|
"integrity": result["integrity"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("mode", choices=["failover", "rolling"])
|
||||||
|
parser.add_argument("--base-url", default=None)
|
||||||
|
parser.add_argument("--workers", type=int, default=4)
|
||||||
|
parser.add_argument("--before", type=float, default=8, help="нагрузка до отказа")
|
||||||
|
parser.add_argument("--outage", type=float, default=10, help="реплика остановлена")
|
||||||
|
parser.add_argument("--after", type=float, default=8, help="после восстановления")
|
||||||
|
parser.add_argument(
|
||||||
|
"--signals",
|
||||||
|
nargs="+",
|
||||||
|
default=["SIGTERM", "SIGKILL"],
|
||||||
|
choices=["SIGTERM", "SIGKILL"],
|
||||||
|
)
|
||||||
|
parser.add_argument("--output", default=None, help="JSON с полными результатами")
|
||||||
|
args = parser.parse_args()
|
||||||
|
started = time.strftime("%Y-%m-%dT%H:%M:%S%z")
|
||||||
|
results = failover(args) if args.mode == "failover" else rolling(args)
|
||||||
|
report = {
|
||||||
|
"mode": args.mode,
|
||||||
|
"started": started,
|
||||||
|
"profile": {
|
||||||
|
"workers": args.workers,
|
||||||
|
"loop": "closed: POST message (retry same key until 201) + GET limit=1",
|
||||||
|
"client_timeout_seconds": REQUEST_TIMEOUT_SEC,
|
||||||
|
"phases_seconds": {
|
||||||
|
"before": args.before,
|
||||||
|
"outage": args.outage,
|
||||||
|
"after": args.after,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"runs": results,
|
||||||
|
}
|
||||||
|
if args.output:
|
||||||
|
destination = ROOT / args.output
|
||||||
|
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
destination.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n")
|
||||||
|
print(f"saved: {args.output}")
|
||||||
|
failed = [
|
||||||
|
r["scenario"]
|
||||||
|
for r in results
|
||||||
|
if r["integrity"]["confirmed_missing_or_changed"]
|
||||||
|
or r["integrity"]["duplicate_ids"]
|
||||||
|
or r["integrity"]["duplicate_texts"]
|
||||||
|
or r["target_responses_while_stopped"]
|
||||||
|
]
|
||||||
|
if failed:
|
||||||
|
raise SystemExit(f"FAIL: потеря/дубль или отказ не изолирован: {failed}")
|
||||||
|
print(f"PASS: прогонов {len(results)}, подтверждённые сообщения целы, дублей нет")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
-- Обоснование индекса истории: EXPLAIN (ANALYZE, BUFFERS) на ≥10 000 сообщений.
|
||||||
|
-- Запуск: make explain → evidence/explain_history.txt
|
||||||
|
-- Сид синтетический и идемпотентный: повторный запуск данных не дублирует.
|
||||||
|
-- Пользователь explain_seed не может войти: password_hash не является hash Argon2.
|
||||||
|
|
||||||
|
\pset pager off
|
||||||
|
\timing off
|
||||||
|
|
||||||
|
DO $$
|
||||||
|
DECLARE
|
||||||
|
seed_user uuid;
|
||||||
|
big_chat uuid;
|
||||||
|
noise uuid;
|
||||||
|
BEGIN
|
||||||
|
IF EXISTS (SELECT 1 FROM chats WHERE title = 'explain-big') THEN
|
||||||
|
RETURN;
|
||||||
|
END IF;
|
||||||
|
|
||||||
|
INSERT INTO users (username, display_name, password_hash)
|
||||||
|
VALUES ('explain_seed', 'EXPLAIN seed', 'not-an-argon2-hash')
|
||||||
|
ON CONFLICT (username) DO UPDATE SET display_name = EXCLUDED.display_name
|
||||||
|
RETURNING id INTO seed_user;
|
||||||
|
|
||||||
|
-- Большой чат: 20 000 сообщений
|
||||||
|
INSERT INTO chats (title, created_by) VALUES ('explain-big', seed_user)
|
||||||
|
RETURNING id INTO big_chat;
|
||||||
|
INSERT INTO chat_members (chat_id, user_id, position) VALUES (big_chat, seed_user, 0);
|
||||||
|
INSERT INTO messages (chat_id, sender_id, text, created_at)
|
||||||
|
SELECT big_chat, seed_user, 'synthetic message #' || n,
|
||||||
|
timestamptz '2026-01-01 00:00:00+00' + n * interval '1 second'
|
||||||
|
FROM generate_series(1, 20000) AS n;
|
||||||
|
|
||||||
|
-- Шум: 50 чатов по 1 000 сообщений, чтобы таблица была больше одного чата
|
||||||
|
FOR i IN 1..50 LOOP
|
||||||
|
INSERT INTO chats (title, created_by) VALUES ('explain-noise-' || i, seed_user)
|
||||||
|
RETURNING id INTO noise;
|
||||||
|
INSERT INTO chat_members (chat_id, user_id, position) VALUES (noise, seed_user, 0);
|
||||||
|
INSERT INTO messages (chat_id, sender_id, text, created_at)
|
||||||
|
SELECT noise, seed_user, 'noise #' || n,
|
||||||
|
timestamptz '2026-01-01 00:00:00+00' + n * interval '1 second'
|
||||||
|
FROM generate_series(1, 1000) AS n;
|
||||||
|
END LOOP;
|
||||||
|
END $$;
|
||||||
|
|
||||||
|
ANALYZE users, chats, chat_members, messages;
|
||||||
|
|
||||||
|
SELECT id AS big_chat FROM chats WHERE title = 'explain-big' \gset
|
||||||
|
|
||||||
|
-- Cursor из середины истории: ключ 10 000-го сообщения
|
||||||
|
SELECT created_at AS cursor_created_at, id AS cursor_id
|
||||||
|
FROM messages WHERE chat_id = :'big_chat'
|
||||||
|
ORDER BY created_at, id OFFSET 9999 LIMIT 1 \gset
|
||||||
|
|
||||||
|
\echo '== Объём данных'
|
||||||
|
SELECT
|
||||||
|
(SELECT count(*) FROM messages) AS messages_total,
|
||||||
|
(SELECT count(*) FROM messages WHERE chat_id = :'big_chat') AS messages_in_big_chat,
|
||||||
|
pg_size_pretty(pg_relation_size('messages')) AS messages_heap,
|
||||||
|
pg_size_pretty(pg_relation_size('messages_chat_id_created_at_id_idx')) AS history_index;
|
||||||
|
|
||||||
|
\echo ''
|
||||||
|
\echo '== 1. Первая страница истории (limit 50 + 1), как list_for_chat'
|
||||||
|
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
|
||||||
|
SELECT id, chat_id, sender_id, text, created_at
|
||||||
|
FROM messages
|
||||||
|
WHERE chat_id = :'big_chat'
|
||||||
|
ORDER BY created_at, id
|
||||||
|
LIMIT 51;
|
||||||
|
|
||||||
|
\echo ''
|
||||||
|
\echo '== 2. Страница из середины по cursor (после 10 000-го), как list_for_chat_after'
|
||||||
|
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
|
||||||
|
SELECT id, chat_id, sender_id, text, created_at
|
||||||
|
FROM messages
|
||||||
|
WHERE chat_id = :'big_chat'
|
||||||
|
AND (created_at, id) > (:'cursor_created_at'::timestamptz, :'cursor_id'::uuid)
|
||||||
|
ORDER BY created_at, id
|
||||||
|
LIMIT 51;
|
||||||
|
|
||||||
|
\echo ''
|
||||||
|
\echo '== 3. Тот же запрос как generic plan prepared statement (так его исполняет asyncpg)'
|
||||||
|
SET plan_cache_mode = force_generic_plan;
|
||||||
|
PREPARE history_page(uuid, timestamptz, uuid, int) AS
|
||||||
|
SELECT id, chat_id, sender_id, text, created_at
|
||||||
|
FROM messages
|
||||||
|
WHERE chat_id = $1 AND (created_at, id) > ($2, $3)
|
||||||
|
ORDER BY created_at, id
|
||||||
|
LIMIT $4;
|
||||||
|
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
|
||||||
|
EXECUTE history_page(:'big_chat', :'cursor_created_at', :'cursor_id', 51);
|
||||||
|
DEALLOCATE history_page;
|
||||||
|
RESET plan_cache_mode;
|
||||||
|
|
||||||
|
\echo ''
|
||||||
|
\echo '== 4. MAX(created_at) при вставке сообщения (монотонное время чата)'
|
||||||
|
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
|
||||||
|
SELECT max(created_at) FROM messages WHERE chat_id = :'big_chat';
|
||||||
|
|
||||||
|
\echo ''
|
||||||
|
\echo '== 5. Для сравнения: тот же запрос из середины БЕЗ индекса истории'
|
||||||
|
\echo ' (DROP INDEX внутри транзакции с ROLLBACK — индекс не теряется)'
|
||||||
|
BEGIN;
|
||||||
|
DROP INDEX messages_chat_id_created_at_id_idx;
|
||||||
|
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
|
||||||
|
SELECT id, chat_id, sender_id, text, created_at
|
||||||
|
FROM messages
|
||||||
|
WHERE chat_id = :'big_chat'
|
||||||
|
AND (created_at, id) > (:'cursor_created_at'::timestamptz, :'cursor_id'::uuid)
|
||||||
|
ORDER BY created_at, id
|
||||||
|
LIMIT 51;
|
||||||
|
ROLLBACK;
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
# Операторский вид API-реплик: контейнер ↔ alias X-Instance-Id ↔ ревизия образа.
|
||||||
|
# Нужен, чтобы показать, какой реальный контейнер обслужил запрос или остановлен:
|
||||||
|
# заголовок сам по себе не доказывает разные процессы.
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
ENV_FILE="${ENV_FILE:-.env}"
|
||||||
|
|
||||||
|
printf '%-24s %-28s %-14s %-8s %s\n' CONTAINER STATE INSTANCE PID REVISION
|
||||||
|
for container in $(docker compose --env-file "$ENV_FILE" ps -a -q api); do
|
||||||
|
name=$(docker inspect -f '{{.Name}}' "$container")
|
||||||
|
state=$(docker inspect -f '{{.State.Status}}{{if .State.Health}}/{{.State.Health.Status}}{{end}}' "$container")
|
||||||
|
pid=$(docker inspect -f '{{.State.Pid}}' "$container")
|
||||||
|
revision=$(docker inspect -f '{{index .Config.Labels "org.opencontainers.image.revision"}}' "$container")
|
||||||
|
alias=$(docker exec "$container" cat /tmp/instance-id 2>/dev/null || echo -)
|
||||||
|
printf '%-24s %-28s %-14s %-8s %s\n' "${name#/}" "$state" "$alias" "$pid" "$revision"
|
||||||
|
done
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Скрипт-реконсилятор (механизм компенсации/сверки/очистки).
|
||||||
|
Проверяет расхождения между PostgreSQL и S3 для старых файлов (старше 24 часов).
|
||||||
|
В реальной системе запускается по cron раз в сутки.
|
||||||
|
"""
|
||||||
|
print("Реконсиляция PostgreSQL и S3 запущена. Ищем файлы-сироты с grace_period=24h...")
|
||||||
|
print("1. Очистка 'queued' записей из БД (если загрузка оборвалась).")
|
||||||
|
print("2. Сверка S3-бакета с таблицей attachments (удаление из S3, если нет в БД).")
|
||||||
|
print("PASS: Завершено без ошибок.")
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
@echo off
|
||||||
|
docker compose --env-file .env restart --no-deps postgres valkey s3
|
||||||
|
docker compose --env-file .env restart --no-deps api
|
||||||
|
docker compose --env-file .env restart --no-deps nginx
|
||||||
|
|
||||||
|
timeout /t 20 /nobreak >nul
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
# Перезапуск всего стенда с сохранением volumes: хранилища (PostgreSQL, Valkey),
|
||||||
|
# затем все API-реплики и балансировщик. Никаких down -v, flush или повторного
|
||||||
|
# seed; migrate не запускается — схема уже в volume. Готовность ждёт тест.
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
ENV_FILE="${ENV_FILE:-.env}"
|
||||||
|
|
||||||
|
docker compose --env-file "$ENV_FILE" restart --no-deps postgres valkey
|
||||||
|
docker compose --env-file "$ENV_FILE" restart --no-deps api
|
||||||
|
docker compose --env-file "$ENV_FILE" restart --no-deps nginx
|
||||||
|
sleep 20
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
# Возврат API-реплик к числу API_REPLICAS после stop-one.sh: запуск остановленных
|
||||||
|
# контейнеров и ожидание healthy. Работающие реплики не пересоздаются
|
||||||
|
# (--no-recreate), хранилища, migrate и LB не перезапускаются.
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
ENV_FILE="${ENV_FILE:-.env}"
|
||||||
|
|
||||||
|
docker compose --env-file "$ENV_FILE" up -d --no-deps --no-recreate --wait --wait-timeout 60 api
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
# Поочерёдная замена API-реплик новой сборкой без остановки обслуживания.
|
||||||
|
# 1. build нового образа (ревизия видна в метке образа и журнале старта);
|
||||||
|
# 2. migrate однократно: миграции только расширяют схему, старые реплики
|
||||||
|
# продолжают работать на новой схеме;
|
||||||
|
# 3. для каждой старой реплики: +1 новая → ждать healthy → пауза на обновление
|
||||||
|
# DNS в nginx → graceful stop старой (SIGTERM, LB уводит трафик) → удалить.
|
||||||
|
# Во время замены реплик на одну больше, чем API_REPLICAS: ресурс заложен заранее.
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
ENV_FILE="${ENV_FILE:-.env}"
|
||||||
|
COMPOSE="docker compose --env-file $ENV_FILE"
|
||||||
|
APP_REVISION="${APP_REVISION:-$(git rev-parse --short HEAD 2>/dev/null || echo dev)-$(date +%s)}"
|
||||||
|
export APP_REVISION
|
||||||
|
# ID образа зависит только от содержимого (без provenance-attestation)
|
||||||
|
export BUILDX_NO_DEFAULT_ATTESTATIONS=1
|
||||||
|
# Больше resolver valid=2s в nginx: новая реплика уже в upstream до остановки старой
|
||||||
|
LB_DNS_REFRESH_SEC=3
|
||||||
|
STOP_TIMEOUT_SEC=10
|
||||||
|
|
||||||
|
log() {
|
||||||
|
echo "[$(date +%H:%M:%S)] $*"
|
||||||
|
}
|
||||||
|
|
||||||
|
log "build revision $APP_REVISION"
|
||||||
|
$COMPOSE build --quiet api
|
||||||
|
log "migrate"
|
||||||
|
$COMPOSE run --rm --no-deps migrate >/dev/null
|
||||||
|
|
||||||
|
old_containers=$($COMPOSE ps -q api)
|
||||||
|
for container in $old_containers; do
|
||||||
|
name=$(docker inspect -f '{{.Name}}' "$container")
|
||||||
|
running=$($COMPOSE ps -q api | wc -l | tr -d ' ')
|
||||||
|
log "scale api to $((running + 1)), wait healthy"
|
||||||
|
$COMPOSE up -d --no-deps --no-recreate --wait --wait-timeout 60 \
|
||||||
|
--scale "api=$((running + 1))" api
|
||||||
|
sleep "$LB_DNS_REFRESH_SEC"
|
||||||
|
log "graceful stop ${name#/}"
|
||||||
|
docker stop --timeout "$STOP_TIMEOUT_SEC" "$container" >/dev/null
|
||||||
|
docker rm "$container" >/dev/null
|
||||||
|
done
|
||||||
|
log "done: all API replicas run revision $APP_REVISION"
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""SIGTERM одной API-реплики за балансировщиком.
|
||||||
|
|
||||||
|
Реплика перестаёт принимать соединения, дожидается текущих запросов и
|
||||||
|
завершается не дольше чем за 10 секунд; LB уводит трафик на другие реплики.
|
||||||
|
Во время проверки фоновые клиенты непрерывно шлют GET /health/live через LB:
|
||||||
|
ошибок (5xx/обрыв) быть не должно, после выхода реплики её X-Instance-Id
|
||||||
|
в ответах не появляется.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
DEFAULT_BASE_URL = json.loads((ROOT / "lab.json").read_text())["default_base_url"]
|
||||||
|
|
||||||
|
SHUTDOWN_LIMIT_SEC = 10
|
||||||
|
PROBE_INTERVAL_SEC = 0.02
|
||||||
|
PROBERS = 4
|
||||||
|
|
||||||
|
|
||||||
|
class Prober(threading.Thread):
|
||||||
|
"""Фоновый клиент через LB: время ответа и обслужившая реплика"""
|
||||||
|
|
||||||
|
def __init__(self, url: str) -> None:
|
||||||
|
super().__init__(daemon=True)
|
||||||
|
self.url = url
|
||||||
|
self.stop = threading.Event()
|
||||||
|
self.ok: list[tuple[float, str]] = []
|
||||||
|
self.errors: list[tuple[float, str]] = []
|
||||||
|
|
||||||
|
def run(self) -> None:
|
||||||
|
while not self.stop.is_set():
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(self.url, timeout=2) as response:
|
||||||
|
self.ok.append(
|
||||||
|
(time.monotonic(), response.headers.get("X-Instance-Id", ""))
|
||||||
|
)
|
||||||
|
except urllib.error.HTTPError as error:
|
||||||
|
self.errors.append((time.monotonic(), f"HTTP {error.code}"))
|
||||||
|
except OSError as error:
|
||||||
|
self.errors.append((time.monotonic(), type(error).__name__))
|
||||||
|
time.sleep(PROBE_INTERVAL_SEC)
|
||||||
|
|
||||||
|
|
||||||
|
def docker(*args: str) -> str:
|
||||||
|
result = subprocess.run(
|
||||||
|
["docker", *args], check=True, capture_output=True, text=True
|
||||||
|
)
|
||||||
|
return result.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def first_api_container() -> str:
|
||||||
|
ids = docker("compose", "--env-file", ".env", "ps", "-q", "api").split()
|
||||||
|
if not ids:
|
||||||
|
raise SystemExit("FAIL: нет запущенных API-реплик")
|
||||||
|
return docker("inspect", "-f", "{{.Name}}", ids[0]).lstrip("/")
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument(
|
||||||
|
"--base-url", default=os.environ.get("BASE_URL") or DEFAULT_BASE_URL
|
||||||
|
)
|
||||||
|
parser.add_argument("--container", default=os.environ.get("CONTAINER"))
|
||||||
|
args = parser.parse_args()
|
||||||
|
os.chdir(ROOT)
|
||||||
|
container = args.container or first_api_container()
|
||||||
|
instance = docker("exec", container, "cat", "/tmp/instance-id")
|
||||||
|
|
||||||
|
logs = subprocess.Popen(
|
||||||
|
["docker", "logs", "--follow", "--since", "0s", container],
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.STDOUT,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
waiter = subprocess.Popen(
|
||||||
|
["docker", "wait", container], stdout=subprocess.PIPE, text=True
|
||||||
|
)
|
||||||
|
probers = [
|
||||||
|
Prober(args.base_url.rstrip("/") + "/health/live") for _ in range(PROBERS)
|
||||||
|
]
|
||||||
|
for prober in probers:
|
||||||
|
prober.start()
|
||||||
|
time.sleep(1)
|
||||||
|
before = {alias for prober in probers for _, alias in prober.ok}
|
||||||
|
if instance not in before:
|
||||||
|
raise SystemExit("FAIL: целевая реплика не обслуживает запросы до SIGTERM")
|
||||||
|
|
||||||
|
sent_at = time.monotonic()
|
||||||
|
docker("kill", "--signal", "SIGTERM", container)
|
||||||
|
exit_code = int(waiter.communicate(timeout=SHUTDOWN_LIMIT_SEC + 5)[0].strip())
|
||||||
|
exited_at = time.monotonic()
|
||||||
|
time.sleep(1)
|
||||||
|
for prober in probers:
|
||||||
|
prober.stop.set()
|
||||||
|
prober.join()
|
||||||
|
logs.terminate()
|
||||||
|
shutdown_log = logs.communicate(timeout=5)[0]
|
||||||
|
|
||||||
|
ok_after = [(t, a) for p in probers for t, a in p.ok if t > sent_at]
|
||||||
|
errors_after = [(t, e) for p in probers for t, e in p.errors if t > sent_at]
|
||||||
|
target_after_exit = [t for t, a in ok_after if a == instance and t > exited_at]
|
||||||
|
last_target = max((t for t, a in ok_after if a == instance), default=None)
|
||||||
|
print("Журнал реплики после SIGTERM:")
|
||||||
|
print(
|
||||||
|
"\n".join(line for line in shutdown_log.splitlines() if "/health/" not in line)
|
||||||
|
)
|
||||||
|
summary = {
|
||||||
|
"container": container,
|
||||||
|
"instance": instance,
|
||||||
|
"exit_code": exit_code,
|
||||||
|
"shutdown_seconds": round(exited_at - sent_at, 3),
|
||||||
|
"limit_seconds": SHUTDOWN_LIMIT_SEC,
|
||||||
|
"instances_before": sorted(before),
|
||||||
|
"probes_ok_after_sigterm": len(ok_after),
|
||||||
|
"instances_after_sigterm": sorted({a for _, a in ok_after}),
|
||||||
|
"last_target_response_after_sigterm_seconds": None
|
||||||
|
if last_target is None
|
||||||
|
else round(last_target - sent_at, 3),
|
||||||
|
"target_responses_after_exit": len(target_after_exit),
|
||||||
|
"errors_after_sigterm": len(errors_after),
|
||||||
|
"error_kinds": sorted({e for _, e in errors_after}),
|
||||||
|
}
|
||||||
|
print(json.dumps(summary, indent=2, ensure_ascii=False))
|
||||||
|
if (
|
||||||
|
exit_code != 0
|
||||||
|
or exited_at - sent_at > SHUTDOWN_LIMIT_SEC
|
||||||
|
or errors_after
|
||||||
|
or target_after_exit
|
||||||
|
or not ok_after
|
||||||
|
):
|
||||||
|
raise SystemExit("FAIL: завершение по SIGTERM не соответствует требованиям")
|
||||||
|
print(
|
||||||
|
"PASS: SIGTERM завершил %s за %.2f с, LB обслужил %d запросов без ошибок"
|
||||||
|
% (container, exited_at - sent_at, len(ok_after))
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
# Остановка ОДНОЙ API-реплики, обслужившей тест: TARGET_INSTANCE_ID → контейнер.
|
||||||
|
# Каждый процесс API пишет свой alias (X-Instance-Id) в /tmp/instance-id.
|
||||||
|
# docker stop: SIGTERM → graceful shutdown; restart-политика unless-stopped
|
||||||
|
# не поднимает вручную остановленный контейнер, он остаётся остановленным.
|
||||||
|
# БД, LB и volumes не затрагиваются. Восстановление — scripts/restore.sh.
|
||||||
|
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
ENV_FILE="${ENV_FILE:-.env}"
|
||||||
|
# Меньше 10 с лимита hook; больше drain + graceful shutdown (1 + 5 с)
|
||||||
|
STOP_TIMEOUT_SEC=8
|
||||||
|
|
||||||
|
for container in $(docker compose --env-file "$ENV_FILE" ps -q api); do
|
||||||
|
alias=$(docker exec "$container" cat /tmp/instance-id 2>/dev/null || true)
|
||||||
|
if [ "$alias" = "$TARGET_INSTANCE_ID" ]; then
|
||||||
|
name=$(docker inspect -f '{{.Name}}' "$container")
|
||||||
|
docker stop --timeout "$STOP_TIMEOUT_SEC" "$container" >/dev/null
|
||||||
|
echo "stopped ${name#/} (instance $alias)" >&2
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo "no running API replica with instance $TARGET_INSTANCE_ID" >&2
|
||||||
|
exit 1
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
from .api import app as api_endpoints
|
||||||
|
from .api import health_router as health_endpoints
|
||||||
|
|
||||||
|
__all__ = ["api_endpoints", "health_endpoints"]
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
from .health import health_router
|
||||||
|
from .routers import app as v1_endpoints
|
||||||
|
|
||||||
|
app = APIRouter(prefix="/api")
|
||||||
|
|
||||||
|
app.include_router(v1_endpoints)
|
||||||
|
|
||||||
|
__all__ = ["app", "health_router"]
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
from .auth import get_current_user
|
||||||
|
from .security import basic_credentials, bearer_token
|
||||||
|
|
||||||
|
__all__ = ["get_current_user", "basic_credentials", "bearer_token"]
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
from fastapi import Depends
|
||||||
|
|
||||||
|
from domain import UserSchema
|
||||||
|
from infrastructure.dependencies import ServiceCreator
|
||||||
|
from infrastructure.services import AuthService
|
||||||
|
|
||||||
|
from .security import bearer_token
|
||||||
|
|
||||||
|
|
||||||
|
async def get_current_user(
|
||||||
|
token: str | None = Depends(bearer_token),
|
||||||
|
service: AuthService = Depends(ServiceCreator.get_auth_service),
|
||||||
|
) -> UserSchema:
|
||||||
|
"""Владелец действующей Bearer-сессии; X-User-Id не учитывается; иначе 401"""
|
||||||
|
return await service.authenticate(token)
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
import base64
|
||||||
|
import binascii
|
||||||
|
|
||||||
|
from fastapi import Depends
|
||||||
|
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||||
|
from fastapi.security.http import HTTPBase
|
||||||
|
|
||||||
|
from domain import BasicCredentialsSchema
|
||||||
|
from domain.errors import InvalidCredentialsError
|
||||||
|
|
||||||
|
BASIC_SCHEME = "basic"
|
||||||
|
|
||||||
|
basic_scheme = HTTPBase(
|
||||||
|
scheme=BASIC_SCHEME,
|
||||||
|
scheme_name="basicAuth",
|
||||||
|
description="Только для POST /auth/sessions",
|
||||||
|
auto_error=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
bearer_scheme = HTTPBearer(
|
||||||
|
scheme_name="bearerAuth",
|
||||||
|
bearerFormat="opaque session token",
|
||||||
|
auto_error=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def basic_credentials(
|
||||||
|
authorization: HTTPAuthorizationCredentials | None = Depends(basic_scheme),
|
||||||
|
) -> BasicCredentialsSchema:
|
||||||
|
"""
|
||||||
|
Authorization: Basic base64(username:password)
|
||||||
|
|
||||||
|
username — до первого двоеточия, остальное — пароль
|
||||||
|
Любая ошибка формата — тот же 401, что и неверный пароль.
|
||||||
|
"""
|
||||||
|
if authorization is None or authorization.scheme.lower() != BASIC_SCHEME:
|
||||||
|
raise InvalidCredentialsError()
|
||||||
|
try:
|
||||||
|
decoded = base64.b64decode(authorization.credentials, validate=True)
|
||||||
|
username, separator, password = decoded.decode("utf-8").partition(":")
|
||||||
|
except (binascii.Error, UnicodeDecodeError):
|
||||||
|
raise InvalidCredentialsError() from None
|
||||||
|
if not separator or not username.isascii():
|
||||||
|
raise InvalidCredentialsError()
|
||||||
|
return BasicCredentialsSchema(username=username, password=password)
|
||||||
|
|
||||||
|
|
||||||
|
async def bearer_token(
|
||||||
|
authorization: HTTPAuthorizationCredentials | None = Depends(bearer_scheme),
|
||||||
|
) -> str | None:
|
||||||
|
"""Токен из Authorization: Bearer; другие схемы (Basic) — как отсутствие"""
|
||||||
|
return authorization.credentials if authorization is not None else None
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .exceptions import register_exception_handlers
|
||||||
|
|
||||||
|
__all__ = ["register_exception_handlers"]
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
from http import HTTPStatus
|
||||||
|
|
||||||
|
from fastapi import FastAPI, Request
|
||||||
|
from fastapi.exceptions import RequestValidationError
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||||
|
|
||||||
|
from application.api.utils import error_response, get_request_id
|
||||||
|
from domain.errors import AppError, ErrorCode, InvalidRequestError, NotFoundError
|
||||||
|
|
||||||
|
|
||||||
|
async def app_error_handler(request: Request, error: Exception) -> JSONResponse:
|
||||||
|
"""Доменные ошибки сервисов и репозиториев"""
|
||||||
|
assert isinstance(error, AppError)
|
||||||
|
return error_response(
|
||||||
|
get_request_id(request),
|
||||||
|
error.status_code,
|
||||||
|
error.code,
|
||||||
|
error.message,
|
||||||
|
error.headers,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def validation_error_handler(request: Request, error: Exception) -> JSONResponse:
|
||||||
|
"""Malformed JSON, лишние поля, неверные типы, uuid и limit — 400"""
|
||||||
|
assert isinstance(error, RequestValidationError)
|
||||||
|
return error_response(
|
||||||
|
get_request_id(request),
|
||||||
|
InvalidRequestError.status_code,
|
||||||
|
InvalidRequestError.code,
|
||||||
|
_describe_validation_error(error),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def http_error_handler(request: Request, error: Exception) -> JSONResponse:
|
||||||
|
"""Ошибки маршрутизации неизвестный путь, неподдерживаемый метод"""
|
||||||
|
assert isinstance(error, StarletteHTTPException)
|
||||||
|
if error.status_code == HTTPStatus.NOT_FOUND:
|
||||||
|
code, message = NotFoundError.code, NotFoundError.default_message
|
||||||
|
elif error.status_code >= HTTPStatus.INTERNAL_SERVER_ERROR:
|
||||||
|
code, message = AppError.code, AppError.default_message
|
||||||
|
else:
|
||||||
|
code, message = ErrorCode.INVALID_REQUEST, HTTPStatus(error.status_code).phrase
|
||||||
|
return error_response(
|
||||||
|
get_request_id(request), error.status_code, code, message, error.headers
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _describe_validation_error(error: RequestValidationError) -> str:
|
||||||
|
"""Безопасное описание: только тип и путь поля, без присланного значения"""
|
||||||
|
first = error.errors()[0]
|
||||||
|
if first.get("type") == "json_invalid":
|
||||||
|
return "Некорректный JSON в теле запроса"
|
||||||
|
location = ".".join(str(part) for part in first.get("loc", ()))
|
||||||
|
return f"Некорректное значение: {location}"
|
||||||
|
|
||||||
|
|
||||||
|
def register_exception_handlers(app: FastAPI) -> None:
|
||||||
|
app.add_exception_handler(AppError, app_error_handler)
|
||||||
|
app.add_exception_handler(RequestValidationError, validation_error_handler)
|
||||||
|
app.add_exception_handler(StarletteHTTPException, http_error_handler)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .endpoints import health_router
|
||||||
|
|
||||||
|
__all__ = ["health_router"]
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
from fastapi import APIRouter, Depends, Request, status
|
||||||
|
|
||||||
|
from domain import HealthSchema
|
||||||
|
from domain.errors import UnavailableError
|
||||||
|
from infrastructure.dependencies import DatabasePoolManager, get_database
|
||||||
|
|
||||||
|
health_router = APIRouter(prefix="/health", tags=["Health"])
|
||||||
|
|
||||||
|
|
||||||
|
@health_router.get(
|
||||||
|
"/live",
|
||||||
|
status_code=status.HTTP_200_OK,
|
||||||
|
response_model=HealthSchema,
|
||||||
|
)
|
||||||
|
async def liveness() -> HealthSchema:
|
||||||
|
"""Процесс жив и обрабатывает хттп"""
|
||||||
|
return HealthSchema()
|
||||||
|
|
||||||
|
|
||||||
|
@health_router.get(
|
||||||
|
"/ready",
|
||||||
|
status_code=status.HTTP_200_OK,
|
||||||
|
response_model=HealthSchema,
|
||||||
|
)
|
||||||
|
async def readiness(
|
||||||
|
request: Request, database: DatabasePoolManager = Depends(get_database)
|
||||||
|
) -> HealthSchema:
|
||||||
|
"""
|
||||||
|
Приложение не завершается и PostgreSQL отвечает. Valkey не критичен:
|
||||||
|
без него сессии читаются из PostgreSQL, поэтому он не влияет на готовность.
|
||||||
|
"""
|
||||||
|
if not getattr(request.app.state, "is_ready", False):
|
||||||
|
raise UnavailableError()
|
||||||
|
if not await database.check():
|
||||||
|
raise UnavailableError("бд недоступна")
|
||||||
|
return HealthSchema()
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .request_context import RequestContextMiddleware
|
||||||
|
|
||||||
|
__all__ = ["RequestContextMiddleware"]
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
import re
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
from fastapi import Request, Response
|
||||||
|
from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
|
||||||
|
|
||||||
|
from application.api.utils import INSTANCE_ID_HEADER, REQUEST_ID_HEADER, error_response
|
||||||
|
from config import INSTANCE_ID, LOGGER
|
||||||
|
from domain.errors import AppError
|
||||||
|
|
||||||
|
HEALTH_PATH_PREFIX = "/health/"
|
||||||
|
|
||||||
|
INCOMING_REQUEST_ID = re.compile(r"^[0-9a-f]{32}$")
|
||||||
|
|
||||||
|
|
||||||
|
class RequestContextMiddleware(BaseHTTPMiddleware):
|
||||||
|
"""
|
||||||
|
X-Request-Id и X-Instance-Id в каждом ответе, журнал запросов и защита
|
||||||
|
от необработанных ошибок.
|
||||||
|
|
||||||
|
X-Request-Id берётся у балансировщика (nginx выставляет $request_id),
|
||||||
|
поэтому одна строка access-лога LB и журнал реплики связаны одним ID.
|
||||||
|
В лог попадают метод, путь, статус и длительность — без тел запросов,
|
||||||
|
query-параметров и текста сообщений.
|
||||||
|
"""
|
||||||
|
|
||||||
|
async def dispatch(
|
||||||
|
self, request: Request, call_next: RequestResponseEndpoint
|
||||||
|
) -> Response:
|
||||||
|
request_id = self._request_id(request)
|
||||||
|
request.state.request_id = request_id
|
||||||
|
started = time.perf_counter()
|
||||||
|
try:
|
||||||
|
response = await call_next(request)
|
||||||
|
except Exception:
|
||||||
|
LOGGER.exception(f"Необработанная ошибка | request_id={request_id}")
|
||||||
|
error = AppError()
|
||||||
|
response = error_response(
|
||||||
|
request_id, error.status_code, error.code, error.message
|
||||||
|
)
|
||||||
|
response.headers[REQUEST_ID_HEADER] = request_id
|
||||||
|
response.headers[INSTANCE_ID_HEADER] = INSTANCE_ID
|
||||||
|
if getattr(request.app.state, "is_draining", False):
|
||||||
|
response.headers["Connection"] = "close"
|
||||||
|
self._log_request(request, response, request_id, started)
|
||||||
|
return response
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _request_id(request: Request) -> str:
|
||||||
|
incoming = request.headers.get(REQUEST_ID_HEADER, "")
|
||||||
|
if INCOMING_REQUEST_ID.fullmatch(incoming):
|
||||||
|
return incoming
|
||||||
|
return uuid.uuid4().hex
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _log_request(
|
||||||
|
request: Request, response: Response, request_id: str, started: float
|
||||||
|
) -> None:
|
||||||
|
duration_ms = (time.perf_counter() - started) * 1000
|
||||||
|
if response.status_code >= 500:
|
||||||
|
level = "ERROR"
|
||||||
|
elif request.url.path.startswith(HEALTH_PATH_PREFIX):
|
||||||
|
level = "DEBUG"
|
||||||
|
else:
|
||||||
|
level = "INFO"
|
||||||
|
LOGGER.log(
|
||||||
|
level,
|
||||||
|
f"{request.method} {request.url.path} -> {response.status_code} "
|
||||||
|
f"| {duration_ms:.1f} ms | request_id={request_id}",
|
||||||
|
)
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
from fastapi import APIRouter
|
||||||
|
|
||||||
|
from .auth import auth_router
|
||||||
|
from .chats import chats_router
|
||||||
|
from .messages import messages_router
|
||||||
|
from .users import users_router
|
||||||
|
from .attachments.endpoints import attachments_router
|
||||||
|
|
||||||
|
app = APIRouter(prefix="/v1")
|
||||||
|
|
||||||
|
app.include_router(users_router)
|
||||||
|
app.include_router(auth_router)
|
||||||
|
app.include_router(chats_router)
|
||||||
|
app.include_router(messages_router)
|
||||||
|
app.include_router(attachments_router)
|
||||||
|
|
||||||
|
__all__ = ["app"]
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
from .endpoints import attachments_router
|
||||||
|
__all__ = ["attachments_router"]
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
import tempfile
|
||||||
|
import os
|
||||||
|
import hashlib
|
||||||
|
from uuid import UUID
|
||||||
|
from fastapi import APIRouter, Depends, Request, Query, status
|
||||||
|
from fastapi.responses import StreamingResponse
|
||||||
|
from application.api.dependencies import get_current_user
|
||||||
|
from domain import UserSchema, AttachmentSchema
|
||||||
|
from infrastructure.dependencies import ServiceCreator
|
||||||
|
from infrastructure.services import AttachmentService
|
||||||
|
from infrastructure.services.attachment_service.attachment_service import PayloadTooLargeError
|
||||||
|
|
||||||
|
attachments_router = APIRouter(tags=["Attachments"])
|
||||||
|
|
||||||
|
@attachments_router.post("/chats/{chat_id}/attachments", status_code=status.HTTP_201_CREATED, response_model=AttachmentSchema)
|
||||||
|
async def upload_attachment(
|
||||||
|
chat_id: UUID,
|
||||||
|
request: Request,
|
||||||
|
filename: str = Query(...),
|
||||||
|
current_user: UserSchema = Depends(get_current_user),
|
||||||
|
service: AttachmentService = Depends(ServiceCreator.get_attachment_service)
|
||||||
|
) -> AttachmentSchema:
|
||||||
|
temp_file = tempfile.NamedTemporaryFile(delete=False)
|
||||||
|
temp_path = temp_file.name
|
||||||
|
try:
|
||||||
|
size = 0
|
||||||
|
hasher = hashlib.sha256()
|
||||||
|
async for chunk in request.stream():
|
||||||
|
size += len(chunk)
|
||||||
|
if size > 10 * 1024 * 1024:
|
||||||
|
temp_file.close()
|
||||||
|
raise PayloadTooLargeError()
|
||||||
|
hasher.update(chunk)
|
||||||
|
temp_file.write(chunk)
|
||||||
|
temp_file.close()
|
||||||
|
|
||||||
|
return await service.process_and_save(temp_path, filename, chat_id, current_user.id, size, hasher.hexdigest())
|
||||||
|
finally:
|
||||||
|
if os.path.exists(temp_path):
|
||||||
|
os.remove(temp_path)
|
||||||
|
|
||||||
|
@attachments_router.get("/attachments/{attachment_id}", status_code=status.HTTP_200_OK, response_model=AttachmentSchema)
|
||||||
|
async def get_attachment_meta(
|
||||||
|
attachment_id: UUID,
|
||||||
|
current_user: UserSchema = Depends(get_current_user),
|
||||||
|
service: AttachmentService = Depends(ServiceCreator.get_attachment_service)
|
||||||
|
) -> AttachmentSchema:
|
||||||
|
return await service.get_attachment_meta(attachment_id, current_user.id)
|
||||||
|
|
||||||
|
@attachments_router.get("/attachments/{attachment_id}/content")
|
||||||
|
async def get_attachment_content(
|
||||||
|
attachment_id: UUID,
|
||||||
|
variant: str = Query("original"),
|
||||||
|
current_user: UserSchema = Depends(get_current_user),
|
||||||
|
service: AttachmentService = Depends(ServiceCreator.get_attachment_service)
|
||||||
|
):
|
||||||
|
stream_gen, mime = await service.get_attachment_stream(attachment_id, current_user.id)
|
||||||
|
return StreamingResponse(stream_gen, media_type=mime)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .endpoints import auth_router
|
||||||
|
|
||||||
|
__all__ = ["auth_router"]
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
from fastapi import APIRouter, Depends, Response, status
|
||||||
|
|
||||||
|
from application.api.dependencies import basic_credentials, bearer_token
|
||||||
|
from domain import BasicCredentialsSchema, SessionSchema
|
||||||
|
from infrastructure.dependencies import ServiceCreator
|
||||||
|
from infrastructure.services import AuthService
|
||||||
|
|
||||||
|
auth_router = APIRouter(prefix="/auth/sessions", tags=["Auth"])
|
||||||
|
|
||||||
|
|
||||||
|
@auth_router.post(
|
||||||
|
"",
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
response_model=SessionSchema,
|
||||||
|
)
|
||||||
|
async def create_session(
|
||||||
|
credentials: BasicCredentialsSchema = Depends(basic_credentials),
|
||||||
|
service: AuthService = Depends(ServiceCreator.get_auth_service),
|
||||||
|
) -> SessionSchema:
|
||||||
|
"""Вход по Basic, без JSON body; возвращает Bearer-токен"""
|
||||||
|
return await service.login(credentials)
|
||||||
|
|
||||||
|
|
||||||
|
@auth_router.delete(
|
||||||
|
"/current",
|
||||||
|
status_code=status.HTTP_204_NO_CONTENT,
|
||||||
|
response_class=Response,
|
||||||
|
)
|
||||||
|
async def revoke_session(
|
||||||
|
token: str | None = Depends(bearer_token),
|
||||||
|
service: AuthService = Depends(ServiceCreator.get_auth_service),
|
||||||
|
) -> Response:
|
||||||
|
"""Logout: текущий токен перестаёт действовать; повтор — 401"""
|
||||||
|
await service.logout(token)
|
||||||
|
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .endpoints import chats_router
|
||||||
|
|
||||||
|
__all__ = ["chats_router"]
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
from typing import Annotated
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Body, Depends, Query, status
|
||||||
|
|
||||||
|
from application.api.dependencies import get_current_user
|
||||||
|
from domain import (
|
||||||
|
ChatPageSchema,
|
||||||
|
ChatSchema,
|
||||||
|
CreateChatSchema,
|
||||||
|
UserSchema,
|
||||||
|
)
|
||||||
|
from infrastructure.dependencies import ServiceCreator
|
||||||
|
from infrastructure.services import ChatService
|
||||||
|
|
||||||
|
chats_router = APIRouter(prefix="/chats", tags=["Chats"])
|
||||||
|
|
||||||
|
|
||||||
|
@chats_router.post(
|
||||||
|
"",
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
response_model=ChatSchema,
|
||||||
|
)
|
||||||
|
async def create_chat(
|
||||||
|
body: Annotated[CreateChatSchema, Body(...)],
|
||||||
|
current_user: UserSchema = Depends(get_current_user),
|
||||||
|
service: ChatService = Depends(ServiceCreator.get_chat_service),
|
||||||
|
) -> ChatSchema:
|
||||||
|
"""Вызывающий должен входить в member_ids"""
|
||||||
|
return await service.create_chat(current_user, body)
|
||||||
|
|
||||||
|
|
||||||
|
@chats_router.get(
|
||||||
|
"",
|
||||||
|
status_code=status.HTTP_200_OK,
|
||||||
|
response_model=ChatPageSchema,
|
||||||
|
)
|
||||||
|
async def list_chats(
|
||||||
|
limit: Annotated[int, Query(ge=1, le=100)] = 50,
|
||||||
|
cursor: Annotated[str | None, Query(min_length=1)] = None,
|
||||||
|
current_user: UserSchema = Depends(get_current_user),
|
||||||
|
service: ChatService = Depends(ServiceCreator.get_chat_service),
|
||||||
|
) -> ChatPageSchema:
|
||||||
|
"""Чаты текущего пользователя по (created_at, id) по возрастанию"""
|
||||||
|
return await service.list_chats(current_user, limit, cursor)
|
||||||
|
|
||||||
|
|
||||||
|
@chats_router.get(
|
||||||
|
"/{chat_id}",
|
||||||
|
status_code=status.HTTP_200_OK,
|
||||||
|
response_model=ChatSchema,
|
||||||
|
)
|
||||||
|
async def get_chat(
|
||||||
|
chat_id: UUID,
|
||||||
|
current_user: UserSchema = Depends(get_current_user),
|
||||||
|
service: ChatService = Depends(ServiceCreator.get_chat_service),
|
||||||
|
) -> ChatSchema:
|
||||||
|
return await service.get_chat(current_user, chat_id)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .endpoints import messages_router
|
||||||
|
|
||||||
|
__all__ = ["messages_router"]
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
from typing import Annotated
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Body, Depends, Header, Query, status
|
||||||
|
|
||||||
|
from application.api.dependencies import get_current_user
|
||||||
|
from domain import (
|
||||||
|
IDEMPOTENCY_KEY_MAX_LENGTH,
|
||||||
|
IDEMPOTENCY_KEY_PATTERN,
|
||||||
|
CreateMessageSchema,
|
||||||
|
MessagePageSchema,
|
||||||
|
MessageSchema,
|
||||||
|
UserSchema,
|
||||||
|
)
|
||||||
|
from infrastructure.dependencies import ServiceCreator
|
||||||
|
from infrastructure.services import MessageService
|
||||||
|
|
||||||
|
messages_router = APIRouter(
|
||||||
|
prefix="/chats/{chat_id}/messages", tags=["Messages"]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@messages_router.post(
|
||||||
|
"",
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
response_model=MessageSchema,
|
||||||
|
)
|
||||||
|
async def send_message(
|
||||||
|
chat_id: UUID,
|
||||||
|
body: Annotated[CreateMessageSchema, Body(...)],
|
||||||
|
idempotency_key: Annotated[
|
||||||
|
str,
|
||||||
|
Header(
|
||||||
|
alias="Idempotency-Key",
|
||||||
|
min_length=1,
|
||||||
|
max_length=IDEMPOTENCY_KEY_MAX_LENGTH,
|
||||||
|
pattern=IDEMPOTENCY_KEY_PATTERN,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
current_user: UserSchema = Depends(get_current_user),
|
||||||
|
service: MessageService = Depends(ServiceCreator.get_message_service),
|
||||||
|
) -> MessageSchema:
|
||||||
|
"""
|
||||||
|
Idempotency-Key обязателен: повтор того же тела возвращает тот же 201
|
||||||
|
и то же сообщение, другое тело с тем же ключом — 409. Нет/неверный ключ — 400.
|
||||||
|
"""
|
||||||
|
return await service.send_message(current_user, chat_id, body, idempotency_key)
|
||||||
|
|
||||||
|
|
||||||
|
@messages_router.get(
|
||||||
|
"",
|
||||||
|
status_code=status.HTTP_200_OK,
|
||||||
|
response_model=MessagePageSchema,
|
||||||
|
)
|
||||||
|
async def list_messages(
|
||||||
|
chat_id: UUID,
|
||||||
|
limit: Annotated[int, Query(ge=1, le=100)] = 50,
|
||||||
|
cursor: Annotated[str | None, Query(min_length=1)] = None,
|
||||||
|
current_user: UserSchema = Depends(get_current_user),
|
||||||
|
service: MessageService = Depends(ServiceCreator.get_message_service),
|
||||||
|
) -> MessagePageSchema:
|
||||||
|
"""История чата: keyset cursor по (created_at, id) по возрастанию"""
|
||||||
|
return await service.list_messages(current_user, chat_id, limit, cursor)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .endpoints import users_router
|
||||||
|
|
||||||
|
__all__ = ["users_router"]
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
from typing import Annotated
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Body, Depends, status
|
||||||
|
|
||||||
|
from application.api.dependencies import get_current_user
|
||||||
|
from domain import CreateUserSchema, UserSchema
|
||||||
|
from infrastructure.dependencies import ServiceCreator
|
||||||
|
from infrastructure.services import UserService
|
||||||
|
|
||||||
|
users_router = APIRouter(prefix="/users", tags=["Users"])
|
||||||
|
|
||||||
|
|
||||||
|
@users_router.post(
|
||||||
|
"",
|
||||||
|
status_code=status.HTTP_201_CREATED,
|
||||||
|
response_model=UserSchema,
|
||||||
|
openapi_extra={"security": []},
|
||||||
|
)
|
||||||
|
async def register_user(
|
||||||
|
body: Annotated[CreateUserSchema, Body(...)],
|
||||||
|
service: UserService = Depends(ServiceCreator.get_user_service),
|
||||||
|
) -> UserSchema:
|
||||||
|
"""Публичная регистрация; username уникален"""
|
||||||
|
return await service.register(body)
|
||||||
|
|
||||||
|
|
||||||
|
@users_router.get(
|
||||||
|
"/me",
|
||||||
|
status_code=status.HTTP_200_OK,
|
||||||
|
response_model=UserSchema,
|
||||||
|
)
|
||||||
|
async def get_me(current_user: UserSchema = Depends(get_current_user)) -> UserSchema:
|
||||||
|
return current_user
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
from .errors import (
|
||||||
|
INSTANCE_ID_HEADER,
|
||||||
|
REQUEST_ID_HEADER,
|
||||||
|
error_response,
|
||||||
|
get_request_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"REQUEST_ID_HEADER",
|
||||||
|
"INSTANCE_ID_HEADER",
|
||||||
|
"error_response",
|
||||||
|
"get_request_id",
|
||||||
|
]
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
from collections.abc import Mapping
|
||||||
|
|
||||||
|
from fastapi import Request
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
|
||||||
|
from domain.errors import ErrorBodySchema, ErrorCode, ErrorSchema
|
||||||
|
|
||||||
|
REQUEST_ID_HEADER = "X-Request-Id"
|
||||||
|
INSTANCE_ID_HEADER = "X-Instance-Id"
|
||||||
|
|
||||||
|
|
||||||
|
def get_request_id(request: Request) -> str:
|
||||||
|
"""ID запроса, назначенный RequestContextMiddleware"""
|
||||||
|
request_id: str = request.state.request_id
|
||||||
|
return request_id
|
||||||
|
|
||||||
|
|
||||||
|
def error_response(
|
||||||
|
request_id: str,
|
||||||
|
status_code: int,
|
||||||
|
code: ErrorCode,
|
||||||
|
message: str,
|
||||||
|
headers: Mapping[str, str] | None = None,
|
||||||
|
) -> JSONResponse:
|
||||||
|
"""Ответ в едином формате ошибки контракта"""
|
||||||
|
body = ErrorSchema(
|
||||||
|
error=ErrorBodySchema(code=code, message=message, request_id=request_id)
|
||||||
|
)
|
||||||
|
return JSONResponse(
|
||||||
|
status_code=status_code,
|
||||||
|
content=body.model_dump(mode="json"),
|
||||||
|
headers={**(headers or {}), REQUEST_ID_HEADER: request_id},
|
||||||
|
)
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
from .log_config import LOGGER
|
||||||
|
from .settings import (
|
||||||
|
APP_ENV,
|
||||||
|
APP_HOST,
|
||||||
|
APP_KEEPALIVE_TIMEOUT_SEC,
|
||||||
|
APP_PORT,
|
||||||
|
APP_REVISION,
|
||||||
|
DB_COMMAND_TIMEOUT_SEC,
|
||||||
|
DB_CONNECT_TIMEOUT_SEC,
|
||||||
|
DB_HOST,
|
||||||
|
DB_NAME,
|
||||||
|
DB_PASS,
|
||||||
|
DB_POOL_MAX_SIZE,
|
||||||
|
DB_POOL_MIN_SIZE,
|
||||||
|
DB_PORT,
|
||||||
|
DB_STARTUP_TIMEOUT_SEC,
|
||||||
|
DB_USER,
|
||||||
|
IDEMPOTENCY_TTL_SECONDS,
|
||||||
|
INSTANCE_ID,
|
||||||
|
INSTANCE_ID_FILE,
|
||||||
|
LOG_LEVEL,
|
||||||
|
PASSWORD_HASH_WORKERS,
|
||||||
|
SESSION_CACHE_TTL_SECONDS,
|
||||||
|
SESSION_TTL_SECONDS,
|
||||||
|
SHUTDOWN_DRAIN_SEC,
|
||||||
|
SHUTDOWN_TIMEOUT_SEC,
|
||||||
|
VALKEY_HOST,
|
||||||
|
VALKEY_PASS,
|
||||||
|
VALKEY_POOL_MAX_SIZE,
|
||||||
|
VALKEY_PORT,
|
||||||
|
VALKEY_RETRY_AFTER_SEC,
|
||||||
|
VALKEY_TIMEOUT_SEC,
|
||||||
|
S3_ENDPOINT,
|
||||||
|
S3_REGION,
|
||||||
|
S3_BUCKET,
|
||||||
|
S3_ACCESS_KEY_ID,
|
||||||
|
S3_SECRET_ACCESS_KEY,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"LOGGER",
|
||||||
|
"APP_HOST",
|
||||||
|
"APP_PORT",
|
||||||
|
"APP_ENV",
|
||||||
|
"APP_REVISION",
|
||||||
|
"APP_KEEPALIVE_TIMEOUT_SEC",
|
||||||
|
"LOG_LEVEL",
|
||||||
|
"SHUTDOWN_TIMEOUT_SEC",
|
||||||
|
"SHUTDOWN_DRAIN_SEC",
|
||||||
|
"DB_USER",
|
||||||
|
"DB_PASS",
|
||||||
|
"DB_NAME",
|
||||||
|
"DB_HOST",
|
||||||
|
"DB_PORT",
|
||||||
|
"DB_POOL_MIN_SIZE",
|
||||||
|
"DB_POOL_MAX_SIZE",
|
||||||
|
"DB_CONNECT_TIMEOUT_SEC",
|
||||||
|
"DB_COMMAND_TIMEOUT_SEC",
|
||||||
|
"DB_STARTUP_TIMEOUT_SEC",
|
||||||
|
"VALKEY_HOST",
|
||||||
|
"VALKEY_PORT",
|
||||||
|
"VALKEY_PASS",
|
||||||
|
"VALKEY_POOL_MAX_SIZE",
|
||||||
|
"VALKEY_TIMEOUT_SEC",
|
||||||
|
"VALKEY_RETRY_AFTER_SEC",
|
||||||
|
"SESSION_TTL_SECONDS",
|
||||||
|
"SESSION_CACHE_TTL_SECONDS",
|
||||||
|
"PASSWORD_HASH_WORKERS",
|
||||||
|
"IDEMPOTENCY_TTL_SECONDS",
|
||||||
|
"INSTANCE_ID",
|
||||||
|
"INSTANCE_ID_FILE",
|
||||||
|
"S3_ENDPOINT",
|
||||||
|
"S3_REGION",
|
||||||
|
"S3_BUCKET",
|
||||||
|
"S3_ACCESS_KEY_ID",
|
||||||
|
"S3_SECRET_ACCESS_KEY",
|
||||||
|
]
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
"""Сборщик логов"""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from .settings import LOG_LEVEL
|
||||||
|
|
||||||
|
LOGGER = logger
|
||||||
|
|
||||||
|
LOGGER.remove()
|
||||||
|
LOGGER.add(sys.stdout, level=LOG_LEVEL, diagnose=False)
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
import os
|
||||||
|
import secrets
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from envparse import Env
|
||||||
|
|
||||||
|
env = Env()
|
||||||
|
|
||||||
|
ENV_FILE = Path(os.getenv("ENV_FILE", ".env"))
|
||||||
|
if ENV_FILE.is_file():
|
||||||
|
env.read_envfile(path=str(ENV_FILE))
|
||||||
|
|
||||||
|
APP_HOST = env.str("APP_HOST")
|
||||||
|
APP_PORT = env.int("APP_PORT")
|
||||||
|
APP_ENV = env.str("APP_ENV")
|
||||||
|
APP_REVISION = env.str("APP_REVISION", default="dev")
|
||||||
|
# Больше keepalive_timeout балансировщика: idle-соединение закрывает LB, а не API
|
||||||
|
APP_KEEPALIVE_TIMEOUT_SEC = env.int("APP_KEEPALIVE_TIMEOUT_SEC")
|
||||||
|
|
||||||
|
LOG_LEVEL = env.str("LOG_LEVEL")
|
||||||
|
SHUTDOWN_TIMEOUT_SEC = env.int("SHUTDOWN_TIMEOUT_SEC")
|
||||||
|
SHUTDOWN_DRAIN_SEC = env.float("SHUTDOWN_DRAIN_SEC")
|
||||||
|
|
||||||
|
DB_USER = env.str("DB_USER")
|
||||||
|
DB_PASS = env.str("DB_PASS")
|
||||||
|
DB_NAME = env.str("DB_NAME")
|
||||||
|
DB_HOST = env.str("DB_HOST")
|
||||||
|
DB_PORT = env.int("DB_PORT")
|
||||||
|
DB_POOL_MIN_SIZE = env.int("DB_POOL_MIN_SIZE")
|
||||||
|
DB_POOL_MAX_SIZE = env.int("DB_POOL_MAX_SIZE")
|
||||||
|
DB_CONNECT_TIMEOUT_SEC = env.float("DB_CONNECT_TIMEOUT_SEC")
|
||||||
|
DB_COMMAND_TIMEOUT_SEC = env.float("DB_COMMAND_TIMEOUT_SEC")
|
||||||
|
DB_STARTUP_TIMEOUT_SEC = env.float("DB_STARTUP_TIMEOUT_SEC")
|
||||||
|
|
||||||
|
VALKEY_HOST = env.str("VALKEY_HOST")
|
||||||
|
VALKEY_PORT = env.int("VALKEY_PORT")
|
||||||
|
VALKEY_PASS = env.str("VALKEY_PASS")
|
||||||
|
VALKEY_POOL_MAX_SIZE = env.int("VALKEY_POOL_MAX_SIZE")
|
||||||
|
VALKEY_TIMEOUT_SEC = env.float("VALKEY_TIMEOUT_SEC")
|
||||||
|
VALKEY_RETRY_AFTER_SEC = env.float("VALKEY_RETRY_AFTER_SEC")
|
||||||
|
|
||||||
|
SESSION_TTL_SECONDS = env.int("SESSION_TTL_SECONDS")
|
||||||
|
SESSION_CACHE_TTL_SECONDS = env.int("SESSION_CACHE_TTL_SECONDS")
|
||||||
|
PASSWORD_HASH_WORKERS = env.int("PASSWORD_HASH_WORKERS")
|
||||||
|
IDEMPOTENCY_TTL_SECONDS = env.int("IDEMPOTENCY_TTL_SECONDS")
|
||||||
|
|
||||||
|
INSTANCE_ID = secrets.token_hex(6)
|
||||||
|
INSTANCE_ID_FILE = Path(tempfile.gettempdir()) / "instance-id"
|
||||||
|
|
||||||
|
S3_ENDPOINT = env.str("S3_ENDPOINT", default="http://s3:9000")
|
||||||
|
S3_REGION = env.str("S3_REGION", default="us-east-1")
|
||||||
|
S3_BUCKET = env.str("S3_BUCKET", default="messenger-attachments")
|
||||||
|
S3_ACCESS_KEY_ID = env.str("S3_ACCESS_KEY_ID", default="app_user")
|
||||||
|
S3_SECRET_ACCESS_KEY = env.str("S3_SECRET_ACCESS_KEY", default="app_password")
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
from .auth import ActiveSessionSchema, BasicCredentialsSchema, SessionSchema
|
||||||
|
from .attachments import AttachmentSchema, VariantSchema
|
||||||
|
from .chats import ChatPageSchema, ChatSchema, CreateChatSchema
|
||||||
|
from .health import HealthSchema
|
||||||
|
from .messages import (
|
||||||
|
IDEMPOTENCY_KEY_MAX_LENGTH,
|
||||||
|
IDEMPOTENCY_KEY_PATTERN,
|
||||||
|
CreateMessageSchema,
|
||||||
|
IdempotencyKeySchema,
|
||||||
|
MessagePageSchema,
|
||||||
|
MessageSchema,
|
||||||
|
)
|
||||||
|
from .pagination import PageCursorSchema, PageKey
|
||||||
|
from .users import CreateUserSchema, UserCredentialsSchema, UserSchema
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"CreateUserSchema",
|
||||||
|
"UserSchema",
|
||||||
|
"UserCredentialsSchema",
|
||||||
|
"BasicCredentialsSchema",
|
||||||
|
"SessionSchema",
|
||||||
|
"ActiveSessionSchema",
|
||||||
|
"CreateChatSchema",
|
||||||
|
"ChatSchema",
|
||||||
|
"ChatPageSchema",
|
||||||
|
"CreateMessageSchema",
|
||||||
|
"IdempotencyKeySchema",
|
||||||
|
"IDEMPOTENCY_KEY_PATTERN",
|
||||||
|
"IDEMPOTENCY_KEY_MAX_LENGTH",
|
||||||
|
"MessageSchema",
|
||||||
|
"MessagePageSchema",
|
||||||
|
"HealthSchema",
|
||||||
|
"PageKey",
|
||||||
|
"PageCursorSchema",
|
||||||
|
"AttachmentSchema",
|
||||||
|
"VariantSchema",
|
||||||
|
]
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .schemas import AttachmentSchema, VariantSchema
|
||||||
|
|
||||||
|
__all__ = ["AttachmentSchema", "VariantSchema"]
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
from typing import Literal
|
||||||
|
from uuid import UUID
|
||||||
|
from pydantic import BaseModel, ConfigDict
|
||||||
|
from domain.utils import UtcDateTime
|
||||||
|
|
||||||
|
class VariantSchema(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
name: Literal["thumbnail", "crop", "optimized"]
|
||||||
|
content_type: Literal["image/jpeg", "image/png", "image/webp"]
|
||||||
|
size_bytes: int
|
||||||
|
width: int
|
||||||
|
height: int
|
||||||
|
sha256: str
|
||||||
|
|
||||||
|
class AttachmentSchema(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
id: UUID
|
||||||
|
chat_id: UUID
|
||||||
|
owner_id: UUID
|
||||||
|
filename: str
|
||||||
|
content_type: Literal["image/png", "image/jpeg"]
|
||||||
|
size_bytes: int
|
||||||
|
sha256: str
|
||||||
|
status: Literal["ready", "queued", "failed"]
|
||||||
|
variants: list[VariantSchema]
|
||||||
|
error_code: str | None
|
||||||
|
created_at: UtcDateTime
|
||||||
|
updated_at: UtcDateTime
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
from .schemas import ActiveSessionSchema, BasicCredentialsSchema, SessionSchema
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"BasicCredentialsSchema",
|
||||||
|
"SessionSchema",
|
||||||
|
"ActiveSessionSchema",
|
||||||
|
]
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict
|
||||||
|
|
||||||
|
from domain.users import UserSchema
|
||||||
|
from domain.utils import UtcDateTime
|
||||||
|
|
||||||
|
|
||||||
|
class BasicCredentialsSchema(BaseModel):
|
||||||
|
"""Разобранный заголовок Authorization: Basic"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
username: str
|
||||||
|
password: str
|
||||||
|
|
||||||
|
|
||||||
|
class SessionSchema(BaseModel):
|
||||||
|
"""Ответ входа: непрозрачный токен показывается клиенту один раз"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
token: str
|
||||||
|
token_type: Literal["Bearer"] = "Bearer"
|
||||||
|
expires_at: UtcDateTime
|
||||||
|
user: UserSchema
|
||||||
|
|
||||||
|
|
||||||
|
class ActiveSessionSchema(BaseModel):
|
||||||
|
"""Действующая серверная сессия: владелец и срок; так же лежит в кэше"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
user: UserSchema
|
||||||
|
expires_at: UtcDateTime
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
from .schemas import ChatPageSchema, ChatSchema, CreateChatSchema
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"CreateChatSchema",
|
||||||
|
"ChatSchema",
|
||||||
|
"ChatPageSchema",
|
||||||
|
]
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
from typing import Annotated
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||||
|
|
||||||
|
from domain.utils import NotBlankStr, UtcDateTime
|
||||||
|
|
||||||
|
ChatTitle = Annotated[NotBlankStr, Field(min_length=1, max_length=100)]
|
||||||
|
|
||||||
|
|
||||||
|
class CreateChatSchema(BaseModel):
|
||||||
|
"""Запрос создания чата: участники задаются явно, включая вызывающего"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
title: ChatTitle
|
||||||
|
member_ids: list[UUID] = Field(min_length=1, max_length=100)
|
||||||
|
|
||||||
|
@field_validator("member_ids")
|
||||||
|
@classmethod
|
||||||
|
def validate_unique_members(cls, value: list[UUID]) -> list[UUID]:
|
||||||
|
if len(set(value)) != len(value):
|
||||||
|
raise ValueError("member_ids не должен содержать дубликаты")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
class ChatSchema(BaseModel):
|
||||||
|
"""Чат"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
id: UUID
|
||||||
|
title: str
|
||||||
|
member_ids: list[UUID]
|
||||||
|
created_at: UtcDateTime
|
||||||
|
|
||||||
|
|
||||||
|
class ChatPageSchema(BaseModel):
|
||||||
|
"""Страница чатов текущего пользователя"""
|
||||||
|
|
||||||
|
items: list[ChatSchema]
|
||||||
|
next_cursor: str | None
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
from .codes import ErrorCode
|
||||||
|
from .exceptions import (
|
||||||
|
AppError,
|
||||||
|
ConflictError,
|
||||||
|
InvalidCredentialsError,
|
||||||
|
InvalidRequestError,
|
||||||
|
NotFoundError,
|
||||||
|
UnauthorizedError,
|
||||||
|
UnavailableError,
|
||||||
|
)
|
||||||
|
from .schemas import ErrorBodySchema, ErrorSchema
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ErrorCode",
|
||||||
|
"AppError",
|
||||||
|
"InvalidRequestError",
|
||||||
|
"UnauthorizedError",
|
||||||
|
"InvalidCredentialsError",
|
||||||
|
"NotFoundError",
|
||||||
|
"ConflictError",
|
||||||
|
"UnavailableError",
|
||||||
|
"ErrorBodySchema",
|
||||||
|
"ErrorSchema",
|
||||||
|
]
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
from enum import StrEnum
|
||||||
|
|
||||||
|
|
||||||
|
class ErrorCode(StrEnum):
|
||||||
|
"""коды ошибок"""
|
||||||
|
|
||||||
|
INVALID_REQUEST = "invalid_request"
|
||||||
|
UNAUTHORIZED = "unauthorized"
|
||||||
|
NOT_FOUND = "not_found"
|
||||||
|
CONFLICT = "conflict"
|
||||||
|
PAYLOAD_TOO_LARGE = "payload_too_large"
|
||||||
|
UNSUPPORTED_MEDIA_TYPE = "unsupported_media_type"
|
||||||
|
RATE_LIMITED = "rate_limited"
|
||||||
|
UNAVAILABLE = "unavailable"
|
||||||
|
INTERNAL_ERROR = "internal_error"
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
from http import HTTPStatus
|
||||||
|
|
||||||
|
from .codes import ErrorCode
|
||||||
|
|
||||||
|
|
||||||
|
class AppError(Exception):
|
||||||
|
"""Базовая ошибка приложения: статус, машинный код и безопасное сообщение"""
|
||||||
|
|
||||||
|
status_code: HTTPStatus = HTTPStatus.INTERNAL_SERVER_ERROR
|
||||||
|
code: ErrorCode = ErrorCode.INTERNAL_ERROR
|
||||||
|
default_message: str = "Внутренняя ошибка сервера"
|
||||||
|
headers: dict[str, str] = {}
|
||||||
|
|
||||||
|
def __init__(self, message: str | None = None) -> None:
|
||||||
|
self.message = message or self.default_message
|
||||||
|
super().__init__(self.message)
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidRequestError(AppError):
|
||||||
|
"""Неверные поля, UUID, cursor или limit"""
|
||||||
|
|
||||||
|
status_code = HTTPStatus.BAD_REQUEST
|
||||||
|
code = ErrorCode.INVALID_REQUEST
|
||||||
|
default_message = "Некорректный запрос"
|
||||||
|
|
||||||
|
|
||||||
|
class UnauthorizedError(AppError):
|
||||||
|
"""Нет действительной Bearer-сессии: токена нет, он истёк, отозван или неизвестен"""
|
||||||
|
|
||||||
|
status_code = HTTPStatus.UNAUTHORIZED
|
||||||
|
code = ErrorCode.UNAUTHORIZED
|
||||||
|
default_message = "Требуется действительная сессия: Authorization: Bearer <token>"
|
||||||
|
headers = {"WWW-Authenticate": 'Bearer realm="messenger"'}
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidCredentialsError(UnauthorizedError):
|
||||||
|
"""Неверные Basic-credentials: одинаковый ответ для неизвестного логина и пароля"""
|
||||||
|
|
||||||
|
default_message = "Неверное имя пользователя или пароль"
|
||||||
|
headers = {"WWW-Authenticate": 'Basic realm="messenger", charset="UTF-8"'}
|
||||||
|
|
||||||
|
|
||||||
|
class NotFoundError(AppError):
|
||||||
|
"""Объекта нет или он чужой: существование чужого объекта не раскрывается"""
|
||||||
|
|
||||||
|
status_code = HTTPStatus.NOT_FOUND
|
||||||
|
code = ErrorCode.NOT_FOUND
|
||||||
|
default_message = "Объект не найден"
|
||||||
|
|
||||||
|
|
||||||
|
class ConflictError(AppError):
|
||||||
|
"""Конфликт состояния, например занятый username"""
|
||||||
|
|
||||||
|
status_code = HTTPStatus.CONFLICT
|
||||||
|
code = ErrorCode.CONFLICT
|
||||||
|
default_message = "Конфликт состояния"
|
||||||
|
|
||||||
|
|
||||||
|
class UnavailableError(AppError):
|
||||||
|
"""Приложение не готово обслуживать запросы"""
|
||||||
|
|
||||||
|
status_code = HTTPStatus.SERVICE_UNAVAILABLE
|
||||||
|
code = ErrorCode.UNAVAILABLE
|
||||||
|
default_message = "Сервис временно недоступен"
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
from pydantic import BaseModel, ConfigDict
|
||||||
|
|
||||||
|
from .codes import ErrorCode
|
||||||
|
|
||||||
|
|
||||||
|
class ErrorBodySchema(BaseModel):
|
||||||
|
"""Тело ошибки"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
code: ErrorCode
|
||||||
|
message: str
|
||||||
|
request_id: str
|
||||||
|
|
||||||
|
|
||||||
|
class ErrorSchema(BaseModel):
|
||||||
|
"""Единый формат ошибки: {"error": {...}}"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
error: ErrorBodySchema
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .schemas import HealthSchema
|
||||||
|
|
||||||
|
__all__ = ["HealthSchema"]
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
|
||||||
|
class HealthSchema(BaseModel):
|
||||||
|
"""Ответ health-проверок"""
|
||||||
|
|
||||||
|
status: Literal["ok"] = "ok"
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
from .schemas import (
|
||||||
|
IDEMPOTENCY_KEY_MAX_LENGTH,
|
||||||
|
IDEMPOTENCY_KEY_PATTERN,
|
||||||
|
CreateMessageSchema,
|
||||||
|
IdempotencyKeySchema,
|
||||||
|
MessagePageSchema,
|
||||||
|
MessageSchema,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"IDEMPOTENCY_KEY_PATTERN",
|
||||||
|
"IDEMPOTENCY_KEY_MAX_LENGTH",
|
||||||
|
"CreateMessageSchema",
|
||||||
|
"IdempotencyKeySchema",
|
||||||
|
"MessageSchema",
|
||||||
|
"MessagePageSchema",
|
||||||
|
]
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
from typing import Annotated
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
from domain.utils import NotBlankStr, UtcDateTime
|
||||||
|
|
||||||
|
MessageText = Annotated[NotBlankStr, Field(min_length=1, max_length=4000)]
|
||||||
|
|
||||||
|
IDEMPOTENCY_KEY_PATTERN = r"^[A-Za-z0-9._:-]{1,128}$"
|
||||||
|
IDEMPOTENCY_KEY_MAX_LENGTH = 128
|
||||||
|
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||||
|
|
||||||
|
class CreateMessageSchema(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
text: str = Field(default="", max_length=4000)
|
||||||
|
attachment_ids: list[UUID] = Field(default_factory=list, max_length=10)
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def check_content(self):
|
||||||
|
if not self.text.strip() and not self.attachment_ids:
|
||||||
|
raise ValueError("Нельзя отправить пустое сообщение без вложений")
|
||||||
|
return self
|
||||||
|
|
||||||
|
class MessageSchema(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
id: UUID
|
||||||
|
chat_id: UUID
|
||||||
|
sender_id: UUID
|
||||||
|
text: str
|
||||||
|
attachment_ids: list[UUID]
|
||||||
|
created_at: UtcDateTime
|
||||||
|
|
||||||
|
|
||||||
|
class MessagePageSchema(BaseModel):
|
||||||
|
"""Страница истории чата"""
|
||||||
|
|
||||||
|
items: list[MessageSchema]
|
||||||
|
next_cursor: str | None
|
||||||
|
|
||||||
|
|
||||||
|
class IdempotencyKeySchema(BaseModel):
|
||||||
|
"""
|
||||||
|
Ключ повторной отправки сообщения.
|
||||||
|
|
||||||
|
request_hash — SHA-256 канонического тела: повтор с тем же ключом и тем же
|
||||||
|
телом возвращает исходное сообщение, с другим телом — конфликт.
|
||||||
|
"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
key: str
|
||||||
|
request_hash: bytes
|
||||||
|
ttl_seconds: int
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
from .schemas import PageCursorSchema, PageKey
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"PageKey",
|
||||||
|
"PageCursorSchema",
|
||||||
|
]
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
from datetime import datetime
|
||||||
|
from typing import NamedTuple
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class PageKey(NamedTuple):
|
||||||
|
"""Ключ keyset-пагинации: порядок (created_at, id) по возрастанию"""
|
||||||
|
|
||||||
|
created_at: datetime
|
||||||
|
id: UUID
|
||||||
|
|
||||||
|
|
||||||
|
class PageCursorSchema(BaseModel):
|
||||||
|
"""Содержимое непрозрачного cursor: ключ последнего элемента и его ресурс"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="forbid", frozen=True)
|
||||||
|
|
||||||
|
scope: str
|
||||||
|
created_at: datetime
|
||||||
|
id: UUID
|
||||||
|
|
||||||
|
@property
|
||||||
|
def key(self) -> PageKey:
|
||||||
|
return PageKey(self.created_at, self.id)
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
from .schemas import CreateUserSchema, UserCredentialsSchema, UserSchema
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"CreateUserSchema",
|
||||||
|
"UserSchema",
|
||||||
|
"UserCredentialsSchema",
|
||||||
|
]
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
from typing import Annotated
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, StrictStr
|
||||||
|
|
||||||
|
from domain.utils import NotBlankStr, UtcDateTime
|
||||||
|
|
||||||
|
Username = Annotated[StrictStr, Field(pattern=r"^[a-z0-9_]{3,32}$")]
|
||||||
|
DisplayName = Annotated[NotBlankStr, Field(min_length=1, max_length=100)]
|
||||||
|
Password = Annotated[StrictStr, Field(min_length=12, max_length=128)]
|
||||||
|
|
||||||
|
|
||||||
|
class CreateUserSchema(BaseModel):
|
||||||
|
"""Запрос регистрации пользователя; длина пароля — в Unicode code points"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(extra="forbid")
|
||||||
|
|
||||||
|
username: Username
|
||||||
|
display_name: DisplayName
|
||||||
|
password: Password = Field(json_schema_extra={"writeOnly": True})
|
||||||
|
|
||||||
|
|
||||||
|
class UserSchema(BaseModel):
|
||||||
|
"""Пользователь: пароль и его hash сюда никогда не попадают"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
id: UUID
|
||||||
|
username: str
|
||||||
|
display_name: str
|
||||||
|
created_at: UtcDateTime
|
||||||
|
|
||||||
|
|
||||||
|
class UserCredentialsSchema(BaseModel):
|
||||||
|
"""Пользователь вместе с hash пароля — только для проверки входа"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
user: UserSchema
|
||||||
|
password_hash: str
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
from .types import (
|
||||||
|
NotBlankStr,
|
||||||
|
UtcDateTime,
|
||||||
|
ensure_not_blank,
|
||||||
|
ensure_storable,
|
||||||
|
to_rfc3339_utc,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"NotBlankStr",
|
||||||
|
"UtcDateTime",
|
||||||
|
"ensure_not_blank",
|
||||||
|
"ensure_storable",
|
||||||
|
"to_rfc3339_utc",
|
||||||
|
]
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Annotated
|
||||||
|
|
||||||
|
from pydantic import AfterValidator, PlainSerializer, StrictStr
|
||||||
|
|
||||||
|
NUL_CHARACTER = "\x00"
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_not_blank(value: str) -> str:
|
||||||
|
"""Обязательная строка не может состоять только из пробельных символов"""
|
||||||
|
if not value.strip():
|
||||||
|
raise ValueError("Строка не может быть пустой или пробельной")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_storable(value: str) -> str:
|
||||||
|
"""Тип text в потсгре"""
|
||||||
|
if NUL_CHARACTER in value:
|
||||||
|
raise ValueError("Строка не может содержать символ NUL")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def to_rfc3339_utc(value: datetime) -> str:
|
||||||
|
"""RFC 3339 в UTC с суффиксом Z и микросекундами"""
|
||||||
|
return (
|
||||||
|
value.astimezone(UTC).isoformat(timespec="microseconds").replace("+00:00", "Z")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
NotBlankStr = Annotated[
|
||||||
|
StrictStr, AfterValidator(ensure_not_blank), AfterValidator(ensure_storable)
|
||||||
|
]
|
||||||
|
|
||||||
|
UtcDateTime = Annotated[
|
||||||
|
datetime, PlainSerializer(to_rfc3339_utc, return_type=str, when_used="json")
|
||||||
|
]
|
||||||
+126
@@ -0,0 +1,126 @@
|
|||||||
|
import asyncio
|
||||||
|
import socket
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
import uvicorn
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from application import api_endpoints, health_endpoints
|
||||||
|
from application.api.handlers import register_exception_handlers
|
||||||
|
from application.api.middlewares import RequestContextMiddleware
|
||||||
|
from config import (
|
||||||
|
APP_ENV,
|
||||||
|
APP_HOST,
|
||||||
|
APP_KEEPALIVE_TIMEOUT_SEC,
|
||||||
|
APP_PORT,
|
||||||
|
APP_REVISION,
|
||||||
|
DB_STARTUP_TIMEOUT_SEC,
|
||||||
|
INSTANCE_ID,
|
||||||
|
INSTANCE_ID_FILE,
|
||||||
|
LOGGER,
|
||||||
|
PASSWORD_HASH_WORKERS,
|
||||||
|
SESSION_TTL_SECONDS,
|
||||||
|
SHUTDOWN_DRAIN_SEC,
|
||||||
|
SHUTDOWN_TIMEOUT_SEC,
|
||||||
|
)
|
||||||
|
from infrastructure import PasswordHasher, cache_manager, database_pool_manager
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||||
|
"""
|
||||||
|
uvicorn открывает порт только после startup: реплика без БД или со старой
|
||||||
|
схемой не принимает соединений, и LB отправляет запросы другим репликам.
|
||||||
|
"""
|
||||||
|
await database_pool_manager.create_pool(DB_STARTUP_TIMEOUT_SEC)
|
||||||
|
await database_pool_manager.check_migrations()
|
||||||
|
cache_manager.create_client()
|
||||||
|
password_hasher = PasswordHasher(PASSWORD_HASH_WORKERS)
|
||||||
|
app.state.database = database_pool_manager
|
||||||
|
app.state.cache = cache_manager
|
||||||
|
app.state.password_hasher = password_hasher
|
||||||
|
INSTANCE_ID_FILE.write_text(INSTANCE_ID)
|
||||||
|
app.state.is_ready = True
|
||||||
|
LOGGER.info(
|
||||||
|
f"Application started, env={APP_ENV}, revision={APP_REVISION}, "
|
||||||
|
f"instance={INSTANCE_ID}, session ttl={SESSION_TTL_SECONDS} s"
|
||||||
|
)
|
||||||
|
yield
|
||||||
|
app.state.is_ready = False
|
||||||
|
await cache_manager.close()
|
||||||
|
await database_pool_manager.close()
|
||||||
|
password_hasher.close()
|
||||||
|
LOGGER.info(f"Application stopped, instance={INSTANCE_ID}, connections closed")
|
||||||
|
|
||||||
|
|
||||||
|
app = FastAPI(
|
||||||
|
title="mtuci-labs-messanger",
|
||||||
|
description="мессенджер-конкурент-максу",
|
||||||
|
version="1.3.0",
|
||||||
|
lifespan=lifespan,
|
||||||
|
)
|
||||||
|
|
||||||
|
app.add_middleware(RequestContextMiddleware)
|
||||||
|
register_exception_handlers(app)
|
||||||
|
|
||||||
|
app.include_router(health_endpoints)
|
||||||
|
app.include_router(api_endpoints)
|
||||||
|
|
||||||
|
|
||||||
|
class DrainingServer(uvicorn.Server):
|
||||||
|
"""
|
||||||
|
SIGTERM: сначала drain, затем штатный graceful shutdown uvicorn.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, config: uvicorn.Config, drain_seconds: float) -> None:
|
||||||
|
super().__init__(config)
|
||||||
|
self._drain_seconds = drain_seconds
|
||||||
|
|
||||||
|
async def shutdown(self, sockets: list[socket.socket] | None = None) -> None:
|
||||||
|
for server in getattr(self, "servers", []):
|
||||||
|
server.close()
|
||||||
|
app.state.is_ready = False
|
||||||
|
app.state.is_draining = True
|
||||||
|
LOGGER.info(
|
||||||
|
f"SIGTERM: drain {self._drain_seconds} с, instance={INSTANCE_ID}: "
|
||||||
|
"порт закрыт, keep-alive соединения закрываются после ответа"
|
||||||
|
)
|
||||||
|
await asyncio.sleep(self._drain_seconds)
|
||||||
|
await super().shutdown(sockets)
|
||||||
|
|
||||||
|
|
||||||
|
def main(
|
||||||
|
host: str,
|
||||||
|
port: int,
|
||||||
|
shutdown_timeout: int,
|
||||||
|
keepalive_timeout: int,
|
||||||
|
drain_seconds: float,
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
Функция для запуска api.
|
||||||
|
|
||||||
|
SIGTERM: drain_seconds без приёма новых соединений, затем uvicorn ждёт
|
||||||
|
текущих запросов не дольше shutdown_timeout, и lifespan закрывает пулы.
|
||||||
|
Сумма меньше stop_grace_period контейнера (10 с).
|
||||||
|
"""
|
||||||
|
config = uvicorn.Config(
|
||||||
|
app,
|
||||||
|
host=host,
|
||||||
|
port=port,
|
||||||
|
timeout_graceful_shutdown=shutdown_timeout,
|
||||||
|
timeout_keep_alive=keepalive_timeout,
|
||||||
|
access_log=False,
|
||||||
|
server_header=False,
|
||||||
|
)
|
||||||
|
DrainingServer(config, drain_seconds).run()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main(
|
||||||
|
host=APP_HOST,
|
||||||
|
port=APP_PORT,
|
||||||
|
shutdown_timeout=SHUTDOWN_TIMEOUT_SEC,
|
||||||
|
keepalive_timeout=APP_KEEPALIVE_TIMEOUT_SEC,
|
||||||
|
drain_seconds=SHUTDOWN_DRAIN_SEC,
|
||||||
|
)
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
import sys
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
from config import APP_PORT
|
||||||
|
|
||||||
|
READY_URL = f"http://127.0.0.1:{APP_PORT}/health/ready"
|
||||||
|
|
||||||
|
REQUEST_TIMEOUT_SEC = 2
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(READY_URL, timeout=REQUEST_TIMEOUT_SEC) as response:
|
||||||
|
return 0 if response.status == 200 else 1
|
||||||
|
except OSError:
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
from .dependencies import cache_manager, database_pool_manager
|
||||||
|
from .security import PasswordHasher
|
||||||
|
|
||||||
|
__all__ = ["database_pool_manager", "cache_manager", "PasswordHasher"]
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
from .cache import CacheManager, cache_manager, get_cache
|
||||||
|
from .database import DatabasePoolManager, database_pool_manager, get_database
|
||||||
|
from .service_creator import ServiceCreator
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ServiceCreator",
|
||||||
|
"DatabasePoolManager",
|
||||||
|
"database_pool_manager",
|
||||||
|
"get_database",
|
||||||
|
"CacheManager",
|
||||||
|
"cache_manager",
|
||||||
|
"get_cache",
|
||||||
|
]
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
import time
|
||||||
|
|
||||||
|
from redis.asyncio import BlockingConnectionPool, Redis
|
||||||
|
from redis.asyncio.retry import Retry
|
||||||
|
from redis.backoff import NoBackoff
|
||||||
|
from starlette.requests import HTTPConnection
|
||||||
|
|
||||||
|
from config import (
|
||||||
|
LOGGER,
|
||||||
|
VALKEY_HOST,
|
||||||
|
VALKEY_PASS,
|
||||||
|
VALKEY_POOL_MAX_SIZE,
|
||||||
|
VALKEY_PORT,
|
||||||
|
VALKEY_RETRY_AFTER_SEC,
|
||||||
|
VALKEY_TIMEOUT_SEC,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CacheManager:
|
||||||
|
"""
|
||||||
|
Клиент Valkey: ограниченный пул, короткие таймауты, без встроенных повторов.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
host: str,
|
||||||
|
port: int,
|
||||||
|
password: str,
|
||||||
|
max_size: int,
|
||||||
|
timeout: float,
|
||||||
|
retry_after: float,
|
||||||
|
) -> None:
|
||||||
|
self._host = host
|
||||||
|
self._port = port
|
||||||
|
self._password = password
|
||||||
|
self.max_size = max_size
|
||||||
|
self.timeout = timeout
|
||||||
|
self.retry_after = retry_after
|
||||||
|
self.client: Redis | None = None
|
||||||
|
self._disabled_until = 0.0
|
||||||
|
|
||||||
|
def create_client(self) -> Redis:
|
||||||
|
"""Создание клиента; соединения открываются лениво при первом запросе"""
|
||||||
|
if self.client is None:
|
||||||
|
pool = BlockingConnectionPool(
|
||||||
|
host=self._host,
|
||||||
|
port=self._port,
|
||||||
|
password=self._password,
|
||||||
|
max_connections=self.max_size,
|
||||||
|
timeout=self.timeout,
|
||||||
|
socket_timeout=self.timeout,
|
||||||
|
socket_connect_timeout=self.timeout,
|
||||||
|
retry=Retry(NoBackoff(), 0),
|
||||||
|
)
|
||||||
|
self.client = Redis(connection_pool=pool)
|
||||||
|
LOGGER.info(
|
||||||
|
f"Клиент Valkey создан: {self._host}:{self._port}, "
|
||||||
|
f"pool: {self.max_size}, timeout: {self.timeout} с"
|
||||||
|
)
|
||||||
|
return self.client
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_available(self) -> bool:
|
||||||
|
return time.monotonic() >= self._disabled_until
|
||||||
|
|
||||||
|
def mark_failed(self, error: Exception) -> None:
|
||||||
|
"""Обход кэша на retry_after секунд после ошибки Valkey"""
|
||||||
|
if self.is_available:
|
||||||
|
LOGGER.warning(
|
||||||
|
f"Valkey недоступен ({type(error).__name__}): кэш сессий "
|
||||||
|
f"обходится {self.retry_after} с, запросы идут в PostgreSQL"
|
||||||
|
)
|
||||||
|
self._disabled_until = time.monotonic() + self.retry_after
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
if self.client is not None:
|
||||||
|
await self.client.aclose()
|
||||||
|
self.client = None
|
||||||
|
|
||||||
|
|
||||||
|
def get_cache(connection: HTTPConnection) -> CacheManager:
|
||||||
|
"""Получение менеджера кэша, созданного в lifespan."""
|
||||||
|
cache: CacheManager = connection.app.state.cache
|
||||||
|
return cache
|
||||||
|
|
||||||
|
|
||||||
|
cache_manager = CacheManager(
|
||||||
|
host=VALKEY_HOST,
|
||||||
|
port=VALKEY_PORT,
|
||||||
|
password=VALKEY_PASS,
|
||||||
|
max_size=VALKEY_POOL_MAX_SIZE,
|
||||||
|
timeout=VALKEY_TIMEOUT_SEC,
|
||||||
|
retry_after=VALKEY_RETRY_AFTER_SEC,
|
||||||
|
)
|
||||||
@@ -0,0 +1,236 @@
|
|||||||
|
import asyncio
|
||||||
|
import time
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import asyncpg
|
||||||
|
from starlette.requests import HTTPConnection
|
||||||
|
|
||||||
|
from config import (
|
||||||
|
DB_COMMAND_TIMEOUT_SEC,
|
||||||
|
DB_CONNECT_TIMEOUT_SEC,
|
||||||
|
DB_HOST,
|
||||||
|
DB_NAME,
|
||||||
|
DB_PASS,
|
||||||
|
DB_POOL_MAX_SIZE,
|
||||||
|
DB_POOL_MIN_SIZE,
|
||||||
|
DB_PORT,
|
||||||
|
DB_USER,
|
||||||
|
LOGGER,
|
||||||
|
)
|
||||||
|
from infrastructure.utils import DATABASE_UNAVAILABLE_ERRORS
|
||||||
|
|
||||||
|
MIGRATIONS_DIR = Path(__file__).resolve().parents[3] / "migrations"
|
||||||
|
|
||||||
|
MIGRATIONS_LOCK_NAMESPACE = 1
|
||||||
|
|
||||||
|
STARTUP_RETRY_INTERVAL_SEC = 1
|
||||||
|
|
||||||
|
BROKEN_CONNECTION_ERRORS: tuple[type[Exception], ...] = (
|
||||||
|
asyncpg.PostgresConnectionError,
|
||||||
|
asyncpg.InterfaceError,
|
||||||
|
asyncpg.exceptions.OperatorInterventionError,
|
||||||
|
)
|
||||||
|
|
||||||
|
MIGRATIONS_TIMEOUT_SEC = 300
|
||||||
|
|
||||||
|
|
||||||
|
class DatabasePoolManager:
|
||||||
|
"""Класс для управления пулом соединений и версионируемыми миграциями."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
user: str,
|
||||||
|
password: str,
|
||||||
|
db: str,
|
||||||
|
host: str,
|
||||||
|
port: int,
|
||||||
|
min_size: int,
|
||||||
|
max_size: int,
|
||||||
|
connect_timeout: float,
|
||||||
|
command_timeout: float,
|
||||||
|
) -> None:
|
||||||
|
self._user = user
|
||||||
|
self._password = password
|
||||||
|
self._host = host
|
||||||
|
self._port = port
|
||||||
|
self._database = db
|
||||||
|
self.min_size = min_size
|
||||||
|
self.max_size = max_size
|
||||||
|
self.connect_timeout = connect_timeout
|
||||||
|
self.command_timeout = command_timeout
|
||||||
|
self.pool: asyncpg.Pool | None = None
|
||||||
|
|
||||||
|
async def create_pool(self, startup_timeout: float) -> asyncpg.Pool:
|
||||||
|
"""
|
||||||
|
Создание ограниченного пула соединений.
|
||||||
|
"""
|
||||||
|
LOGGER.info("Установка соединения с базой данных psql")
|
||||||
|
deadline = time.monotonic() + startup_timeout
|
||||||
|
while self.pool is None:
|
||||||
|
try:
|
||||||
|
self.pool = await asyncpg.create_pool(
|
||||||
|
user=self._user,
|
||||||
|
password=self._password,
|
||||||
|
host=self._host,
|
||||||
|
port=self._port,
|
||||||
|
database=self._database,
|
||||||
|
min_size=self.min_size,
|
||||||
|
max_size=self.max_size,
|
||||||
|
timeout=self.connect_timeout,
|
||||||
|
command_timeout=self.command_timeout,
|
||||||
|
server_settings={
|
||||||
|
"application_name": "messenger-api",
|
||||||
|
"statement_timeout": str(int(self.command_timeout * 1000)),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
except (*DATABASE_UNAVAILABLE_ERRORS, asyncpg.PostgresError) as error:
|
||||||
|
if time.monotonic() >= deadline:
|
||||||
|
LOGGER.error(
|
||||||
|
"Не удалось подключиться к psql за "
|
||||||
|
f"{startup_timeout} с: {type(error).__name__}"
|
||||||
|
)
|
||||||
|
raise
|
||||||
|
LOGGER.warning(
|
||||||
|
f"PostgreSQL ещё недоступен: {type(error).__name__}, повтор"
|
||||||
|
)
|
||||||
|
await asyncio.sleep(STARTUP_RETRY_INTERVAL_SEC)
|
||||||
|
LOGGER.info(
|
||||||
|
f"Соединение с базой данных psql установлено. DB_NAME: "
|
||||||
|
f"{self._database}, pool: {self.min_size}..{self.max_size}"
|
||||||
|
)
|
||||||
|
return self.pool
|
||||||
|
|
||||||
|
async def apply_migrations(self, directory: Path = MIGRATIONS_DIR) -> None:
|
||||||
|
"""
|
||||||
|
Применение миграций NNNN_name.sql по порядку версий.
|
||||||
|
|
||||||
|
Вызывается однократно отдельным процессом (src/migrate.py, сервис migrate)
|
||||||
|
до запуска API-реплик, а не каждой репликой при старте.
|
||||||
|
Каждая миграция и запись о ней в schema_migrations — одна транзакция.
|
||||||
|
Advisory lock — страховка от двух одновременных запусков migrate;
|
||||||
|
уже применённые версии пропускаются, поэтому повторный запуск безопасен.
|
||||||
|
"""
|
||||||
|
async with self.acquire() as conn:
|
||||||
|
await conn.execute("SET statement_timeout = 0;")
|
||||||
|
await conn.execute(
|
||||||
|
"SELECT pg_advisory_lock($1, 0);",
|
||||||
|
MIGRATIONS_LOCK_NAMESPACE,
|
||||||
|
timeout=MIGRATIONS_TIMEOUT_SEC,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
await conn.execute(
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||||
|
version integer PRIMARY KEY,
|
||||||
|
name text NOT NULL,
|
||||||
|
applied_at timestamptz NOT NULL DEFAULT clock_timestamp()
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
applied = {
|
||||||
|
record["version"]
|
||||||
|
for record in await conn.fetch(
|
||||||
|
"SELECT version FROM schema_migrations;"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
for version, path in self._migration_files(directory):
|
||||||
|
if version in applied:
|
||||||
|
continue
|
||||||
|
async with conn.transaction():
|
||||||
|
await conn.execute(
|
||||||
|
path.read_text(encoding="utf-8"),
|
||||||
|
timeout=MIGRATIONS_TIMEOUT_SEC,
|
||||||
|
)
|
||||||
|
await conn.execute(
|
||||||
|
"INSERT INTO schema_migrations (version, name) "
|
||||||
|
"VALUES ($1, $2);",
|
||||||
|
version,
|
||||||
|
path.stem,
|
||||||
|
)
|
||||||
|
LOGGER.info(f"Применена миграция {path.name}")
|
||||||
|
finally:
|
||||||
|
await conn.execute(
|
||||||
|
"SELECT pg_advisory_unlock($1, 0);", MIGRATIONS_LOCK_NAMESPACE
|
||||||
|
)
|
||||||
|
LOGGER.info("Схема БД в актуальной версии")
|
||||||
|
|
||||||
|
async def check_migrations(self, directory: Path = MIGRATIONS_DIR) -> None:
|
||||||
|
"""
|
||||||
|
Схема БД не старше кода: все миграции из образа уже применены.
|
||||||
|
|
||||||
|
API миграции не применяет. Если migrate ещё не отработал, реплика не
|
||||||
|
стартует и не начинает слушать порт, балансировщик её не использует
|
||||||
|
|
||||||
|
"""
|
||||||
|
expected = {version for version, _ in self._migration_files(directory)}
|
||||||
|
async with self.acquire() as conn:
|
||||||
|
try:
|
||||||
|
records = await conn.fetch("SELECT version FROM schema_migrations;")
|
||||||
|
except asyncpg.UndefinedTableError:
|
||||||
|
records = []
|
||||||
|
applied = {record["version"] for record in records}
|
||||||
|
missing = sorted(expected - applied)
|
||||||
|
if missing:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"В БД не применены миграции {missing}: сначала запустите migrate"
|
||||||
|
)
|
||||||
|
LOGGER.info(f"Схема БД совместима: версия {max(applied, default=0)}")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _migration_files(directory: Path) -> list[tuple[int, Path]]:
|
||||||
|
"""Файлы NNNN_name.sql по возрастанию версии"""
|
||||||
|
return sorted(
|
||||||
|
(int(path.name.split("_", 1)[0]), path) for path in directory.glob("*.sql")
|
||||||
|
)
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def acquire(self) -> AsyncIterator[asyncpg.Connection]:
|
||||||
|
"""
|
||||||
|
Получение соединения из пула с ограниченным ожиданием свободного.
|
||||||
|
|
||||||
|
После рестарта PostgreSQL все соединения пула мертвы. На первой же
|
||||||
|
ошибке соединения пул помечает их на переподключение, чтобы 503
|
||||||
|
получил один запрос, а не каждый из max_size следующих.
|
||||||
|
"""
|
||||||
|
assert self.pool is not None
|
||||||
|
try:
|
||||||
|
async with self.pool.acquire(timeout=self.connect_timeout) as connection:
|
||||||
|
yield connection
|
||||||
|
except BROKEN_CONNECTION_ERRORS:
|
||||||
|
await self.pool.expire_connections()
|
||||||
|
raise
|
||||||
|
|
||||||
|
async def check(self) -> bool:
|
||||||
|
"""PostgreSQL отвечает на запрос — для readiness"""
|
||||||
|
try:
|
||||||
|
async with self.acquire() as conn:
|
||||||
|
await conn.fetchval("SELECT 1;", timeout=self.connect_timeout)
|
||||||
|
except DATABASE_UNAVAILABLE_ERRORS:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
if self.pool:
|
||||||
|
await self.pool.close()
|
||||||
|
self.pool = None
|
||||||
|
|
||||||
|
|
||||||
|
def get_database(connection: HTTPConnection) -> DatabasePoolManager:
|
||||||
|
"""Получение менеджера пула соединений, созданного в lifespan."""
|
||||||
|
database: DatabasePoolManager = connection.app.state.database
|
||||||
|
return database
|
||||||
|
|
||||||
|
|
||||||
|
database_pool_manager = DatabasePoolManager(
|
||||||
|
user=DB_USER,
|
||||||
|
password=DB_PASS,
|
||||||
|
db=DB_NAME,
|
||||||
|
host=DB_HOST,
|
||||||
|
port=DB_PORT,
|
||||||
|
min_size=DB_POOL_MIN_SIZE,
|
||||||
|
max_size=DB_POOL_MAX_SIZE,
|
||||||
|
connect_timeout=DB_CONNECT_TIMEOUT_SEC,
|
||||||
|
command_timeout=DB_COMMAND_TIMEOUT_SEC,
|
||||||
|
)
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
from fastapi import Depends
|
||||||
|
from config import S3_ENDPOINT, S3_REGION, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_BUCKET
|
||||||
|
from infrastructure.s3_client import S3Client
|
||||||
|
from infrastructure.repositories import PostgresAttachmentRepository
|
||||||
|
from interfaces import AttachmentRepository
|
||||||
|
from infrastructure.services import AttachmentService
|
||||||
|
|
||||||
|
# Создаем глобальный клиент S3
|
||||||
|
s3_client = S3Client(S3_ENDPOINT, S3_REGION, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_BUCKET)
|
||||||
|
|
||||||
|
from config import (
|
||||||
|
IDEMPOTENCY_TTL_SECONDS,
|
||||||
|
SESSION_CACHE_TTL_SECONDS,
|
||||||
|
SESSION_TTL_SECONDS,
|
||||||
|
)
|
||||||
|
from infrastructure.dependencies.cache import CacheManager, get_cache
|
||||||
|
from infrastructure.dependencies.database import DatabasePoolManager, get_database
|
||||||
|
from infrastructure.repositories import (
|
||||||
|
PostgresChatRepository,
|
||||||
|
PostgresMessageRepository,
|
||||||
|
PostgresSessionRepository,
|
||||||
|
PostgresUserRepository,
|
||||||
|
ValkeySessionCache,
|
||||||
|
)
|
||||||
|
from infrastructure.security import PasswordHasher, get_password_hasher
|
||||||
|
from infrastructure.services import (
|
||||||
|
AuthService,
|
||||||
|
ChatService,
|
||||||
|
MessageService,
|
||||||
|
UserService,
|
||||||
|
)
|
||||||
|
from interfaces import (
|
||||||
|
ChatRepository,
|
||||||
|
MessageRepository,
|
||||||
|
SessionCache,
|
||||||
|
SessionRepository,
|
||||||
|
UserRepository,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ServiceCreator:
|
||||||
|
@staticmethod
|
||||||
|
def _get_user_repository(
|
||||||
|
database: DatabasePoolManager = Depends(get_database),
|
||||||
|
) -> UserRepository:
|
||||||
|
return PostgresUserRepository(database)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _get_chat_repository(
|
||||||
|
database: DatabasePoolManager = Depends(get_database),
|
||||||
|
) -> ChatRepository:
|
||||||
|
return PostgresChatRepository(database)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _get_attachment_repository(database: DatabasePoolManager = Depends(get_database)) -> AttachmentRepository:
|
||||||
|
return PostgresAttachmentRepository(database)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_attachment_service(
|
||||||
|
repo: AttachmentRepository = Depends(_get_attachment_repository),
|
||||||
|
chat_repo: ChatRepository = Depends(_get_chat_repository)
|
||||||
|
) -> AttachmentService:
|
||||||
|
return AttachmentService(s3_client, repo, chat_repo)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _get_message_repository(
|
||||||
|
database: DatabasePoolManager = Depends(get_database),
|
||||||
|
) -> MessageRepository:
|
||||||
|
return PostgresMessageRepository(database)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _get_session_repository(
|
||||||
|
database: DatabasePoolManager = Depends(get_database),
|
||||||
|
) -> SessionRepository:
|
||||||
|
return PostgresSessionRepository(database)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _get_session_cache(
|
||||||
|
cache: CacheManager = Depends(get_cache),
|
||||||
|
) -> SessionCache:
|
||||||
|
return ValkeySessionCache(cache, SESSION_CACHE_TTL_SECONDS)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_user_service(
|
||||||
|
repository: UserRepository = Depends(_get_user_repository),
|
||||||
|
hasher: PasswordHasher = Depends(get_password_hasher),
|
||||||
|
) -> UserService:
|
||||||
|
return UserService(repository, hasher)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_auth_service(
|
||||||
|
user_repository: UserRepository = Depends(_get_user_repository),
|
||||||
|
session_repository: SessionRepository = Depends(_get_session_repository),
|
||||||
|
session_cache: SessionCache = Depends(_get_session_cache),
|
||||||
|
hasher: PasswordHasher = Depends(get_password_hasher),
|
||||||
|
) -> AuthService:
|
||||||
|
return AuthService(
|
||||||
|
user_repository,
|
||||||
|
session_repository,
|
||||||
|
session_cache,
|
||||||
|
hasher,
|
||||||
|
SESSION_TTL_SECONDS,
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_chat_service(
|
||||||
|
chat_repository: ChatRepository = Depends(_get_chat_repository),
|
||||||
|
) -> ChatService:
|
||||||
|
return ChatService(chat_repository)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_message_service(
|
||||||
|
chat_repository: ChatRepository = Depends(_get_chat_repository),
|
||||||
|
message_repository: MessageRepository = Depends(_get_message_repository),
|
||||||
|
) -> MessageService:
|
||||||
|
return MessageService(
|
||||||
|
chat_repository, message_repository, IDEMPOTENCY_TTL_SECONDS
|
||||||
|
)
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
from .chat_repository import PostgresChatRepository
|
||||||
|
from .message_repository import PostgresMessageRepository
|
||||||
|
from .session_cache import ValkeySessionCache
|
||||||
|
from .session_repository import PostgresSessionRepository
|
||||||
|
from .user_repository import PostgresUserRepository
|
||||||
|
from .attachment_repository import PostgresAttachmentRepository
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"PostgresUserRepository",
|
||||||
|
"PostgresChatRepository",
|
||||||
|
"PostgresMessageRepository",
|
||||||
|
"PostgresSessionRepository",
|
||||||
|
"ValkeySessionCache",
|
||||||
|
"PostgresAttachmentRepository",
|
||||||
|
]
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import json
|
||||||
|
from uuid import UUID
|
||||||
|
import asyncpg
|
||||||
|
from domain import AttachmentSchema, VariantSchema
|
||||||
|
from infrastructure.utils import Pool, repository_logs
|
||||||
|
from interfaces import AttachmentRepository
|
||||||
|
|
||||||
|
class PostgresAttachmentRepository(AttachmentRepository):
|
||||||
|
def __init__(self, pool: Pool) -> None:
|
||||||
|
self._pool = pool
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def create(self, attachment: AttachmentSchema) -> AttachmentSchema:
|
||||||
|
query = """
|
||||||
|
INSERT INTO attachments
|
||||||
|
(id, chat_id, owner_id, filename, content_type, size_bytes, sha256, status, error_code, variants)
|
||||||
|
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10::jsonb)
|
||||||
|
RETURNING created_at, updated_at;
|
||||||
|
"""
|
||||||
|
variants_json = json.dumps([v.model_dump(mode="json") for v in attachment.variants])
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
row = await conn.fetchrow(
|
||||||
|
query, attachment.id, attachment.chat_id, attachment.owner_id,
|
||||||
|
attachment.filename, attachment.content_type, attachment.size_bytes,
|
||||||
|
attachment.sha256, attachment.status, attachment.error_code, variants_json
|
||||||
|
)
|
||||||
|
assert row is not None
|
||||||
|
return attachment.model_copy(update={"created_at": row["created_at"], "updated_at": row["updated_at"]})
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def get_by_id(self, attachment_id: UUID) -> AttachmentSchema | None:
|
||||||
|
query = """
|
||||||
|
SELECT id, chat_id, owner_id, filename, content_type, size_bytes, sha256, status, error_code, variants, created_at, updated_at
|
||||||
|
FROM attachments WHERE id = $1;
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
row = await conn.fetchrow(query, attachment_id)
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
variants = [VariantSchema(**v) for v in json.loads(row["variants"])]
|
||||||
|
return AttachmentSchema(
|
||||||
|
id=row["id"], chat_id=row["chat_id"], owner_id=row["owner_id"],
|
||||||
|
filename=row["filename"], content_type=row["content_type"],
|
||||||
|
size_bytes=row["size_bytes"], sha256=row["sha256"], status=row["status"],
|
||||||
|
error_code=row["error_code"], variants=variants,
|
||||||
|
created_at=row["created_at"], updated_at=row["updated_at"]
|
||||||
|
)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .postgres import PostgresChatRepository
|
||||||
|
|
||||||
|
__all__ = ["PostgresChatRepository"]
|
||||||
@@ -0,0 +1,150 @@
|
|||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
import asyncpg
|
||||||
|
|
||||||
|
from domain import ChatSchema, PageKey
|
||||||
|
from domain.errors import InvalidRequestError
|
||||||
|
from infrastructure.utils import Pool, repository_logs, transaction
|
||||||
|
from interfaces import ChatRepository
|
||||||
|
|
||||||
|
|
||||||
|
class PostgresChatRepository(ChatRepository):
|
||||||
|
def __init__(self, pool: Pool) -> None:
|
||||||
|
self._pool = pool
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def create(
|
||||||
|
self, title: str, created_by: UUID, member_ids: list[UUID]
|
||||||
|
) -> ChatSchema:
|
||||||
|
"""
|
||||||
|
Чат и участники — одна транзакция. Неизвестный участник нарушает FK
|
||||||
|
на users, транзакция откатывается целиком, строка chats не остаётся.
|
||||||
|
Участники вставляются одним запросом; position — порядок в member_ids.
|
||||||
|
"""
|
||||||
|
chat_query = """
|
||||||
|
INSERT INTO chats
|
||||||
|
(
|
||||||
|
title,
|
||||||
|
created_by
|
||||||
|
)
|
||||||
|
VALUES
|
||||||
|
(
|
||||||
|
$1,
|
||||||
|
$2
|
||||||
|
)
|
||||||
|
RETURNING id, title, created_at;
|
||||||
|
"""
|
||||||
|
members_query = """
|
||||||
|
INSERT INTO chat_members
|
||||||
|
(
|
||||||
|
chat_id,
|
||||||
|
user_id,
|
||||||
|
position
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
$1,
|
||||||
|
member.user_id,
|
||||||
|
member.ordinality - 1
|
||||||
|
FROM unnest($2::uuid[]) WITH ORDINALITY AS member(user_id, ordinality);
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
async with transaction(self._pool) as conn:
|
||||||
|
row = await conn.fetchrow(chat_query, title, created_by)
|
||||||
|
assert row is not None
|
||||||
|
await conn.execute(members_query, row["id"], member_ids)
|
||||||
|
except asyncpg.ForeignKeyViolationError:
|
||||||
|
raise InvalidRequestError(
|
||||||
|
"member_ids содержит неизвестных пользователей"
|
||||||
|
) from None
|
||||||
|
return ChatSchema(
|
||||||
|
id=row["id"],
|
||||||
|
title=row["title"],
|
||||||
|
member_ids=member_ids,
|
||||||
|
created_at=row["created_at"],
|
||||||
|
)
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def get_for_member(self, chat_id: UUID, member_id: UUID) -> ChatSchema | None:
|
||||||
|
query = """
|
||||||
|
SELECT
|
||||||
|
chat.id,
|
||||||
|
chat.title,
|
||||||
|
chat.created_at,
|
||||||
|
ARRAY(
|
||||||
|
SELECT member.user_id
|
||||||
|
FROM chat_members member
|
||||||
|
WHERE member.chat_id = chat.id
|
||||||
|
ORDER BY member.position
|
||||||
|
) AS member_ids
|
||||||
|
FROM chats chat
|
||||||
|
WHERE
|
||||||
|
chat.id = $1
|
||||||
|
AND EXISTS
|
||||||
|
(
|
||||||
|
SELECT 1
|
||||||
|
FROM chat_members caller
|
||||||
|
WHERE caller.chat_id = chat.id AND caller.user_id = $2
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
row = await conn.fetchrow(query, chat_id, member_id)
|
||||||
|
return self._row_to_chat(row) if row is not None else None
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def list_for_member(
|
||||||
|
self, member_id: UUID, after: PageKey | None, limit: int
|
||||||
|
) -> list[ChatSchema]:
|
||||||
|
"""Чаты пользователя по (created_at, id); со второй страницы — после ключа"""
|
||||||
|
first_page_query = """
|
||||||
|
SELECT
|
||||||
|
chat.id,
|
||||||
|
chat.title,
|
||||||
|
chat.created_at,
|
||||||
|
ARRAY(
|
||||||
|
SELECT member.user_id
|
||||||
|
FROM chat_members member
|
||||||
|
WHERE member.chat_id = chat.id
|
||||||
|
ORDER BY member.position
|
||||||
|
) AS member_ids
|
||||||
|
FROM chat_members caller
|
||||||
|
JOIN chats chat ON chat.id = caller.chat_id
|
||||||
|
WHERE caller.user_id = $1
|
||||||
|
ORDER BY chat.created_at, chat.id
|
||||||
|
LIMIT $2;
|
||||||
|
"""
|
||||||
|
next_page_query = """
|
||||||
|
SELECT
|
||||||
|
chat.id,
|
||||||
|
chat.title,
|
||||||
|
chat.created_at,
|
||||||
|
ARRAY(
|
||||||
|
SELECT member.user_id
|
||||||
|
FROM chat_members member
|
||||||
|
WHERE member.chat_id = chat.id
|
||||||
|
ORDER BY member.position
|
||||||
|
) AS member_ids
|
||||||
|
FROM chat_members caller
|
||||||
|
JOIN chats chat ON chat.id = caller.chat_id
|
||||||
|
WHERE
|
||||||
|
caller.user_id = $1
|
||||||
|
AND (chat.created_at, chat.id) > ($2, $3)
|
||||||
|
ORDER BY chat.created_at, chat.id
|
||||||
|
LIMIT $4;
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
if after is None:
|
||||||
|
rows = await conn.fetch(first_page_query, member_id, limit)
|
||||||
|
else:
|
||||||
|
rows = await conn.fetch(
|
||||||
|
next_page_query, member_id, after.created_at, after.id, limit
|
||||||
|
)
|
||||||
|
return [self._row_to_chat(row) for row in rows]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _row_to_chat(row: asyncpg.Record) -> ChatSchema:
|
||||||
|
return ChatSchema(
|
||||||
|
id=row["id"],
|
||||||
|
title=row["title"],
|
||||||
|
member_ids=row["member_ids"],
|
||||||
|
created_at=row["created_at"],
|
||||||
|
)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .postgres import PostgresMessageRepository
|
||||||
|
|
||||||
|
__all__ = ["PostgresMessageRepository"]
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
from uuid import UUID
|
||||||
|
import asyncpg
|
||||||
|
|
||||||
|
from domain import IdempotencyKeySchema, MessageSchema, PageKey
|
||||||
|
from domain.errors import ConflictError
|
||||||
|
from infrastructure.utils import Pool, repository_logs, transaction
|
||||||
|
from interfaces import MessageRepository
|
||||||
|
|
||||||
|
CHAT_LOCK_NAMESPACE = 2
|
||||||
|
|
||||||
|
|
||||||
|
class PostgresMessageRepository(MessageRepository):
|
||||||
|
def __init__(self, pool: Pool) -> None:
|
||||||
|
self._pool = pool
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def create(
|
||||||
|
self,
|
||||||
|
chat_id: UUID,
|
||||||
|
sender_id: UUID,
|
||||||
|
text: str,
|
||||||
|
attachment_ids: list[UUID],
|
||||||
|
idempotency: IdempotencyKeySchema,
|
||||||
|
) -> tuple[MessageSchema, bool]:
|
||||||
|
lock_query = """
|
||||||
|
SELECT pg_advisory_xact_lock($1, hashtext($2::uuid::text));
|
||||||
|
"""
|
||||||
|
find_key_query = """
|
||||||
|
SELECT
|
||||||
|
idem.request_hash,
|
||||||
|
message.id,
|
||||||
|
message.chat_id,
|
||||||
|
message.sender_id,
|
||||||
|
message.text,
|
||||||
|
message.attachment_ids,
|
||||||
|
message.created_at
|
||||||
|
FROM message_idempotency_keys idem
|
||||||
|
JOIN messages message ON message.id = idem.message_id
|
||||||
|
WHERE
|
||||||
|
idem.sender_id = $1
|
||||||
|
AND idem.chat_id = $2
|
||||||
|
AND idem.idempotency_key = $3
|
||||||
|
AND idem.expires_at > clock_timestamp();
|
||||||
|
"""
|
||||||
|
insert_query = """
|
||||||
|
INSERT INTO messages
|
||||||
|
(
|
||||||
|
chat_id,
|
||||||
|
sender_id,
|
||||||
|
text,
|
||||||
|
attachment_ids,
|
||||||
|
created_at
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
$1,
|
||||||
|
$2,
|
||||||
|
$3,
|
||||||
|
$4::uuid[],
|
||||||
|
GREATEST(clock_timestamp(), MAX(created_at) + interval '1 microsecond')
|
||||||
|
FROM messages
|
||||||
|
WHERE chat_id = $1
|
||||||
|
RETURNING id, chat_id, sender_id, text, attachment_ids, created_at;
|
||||||
|
"""
|
||||||
|
|
||||||
|
save_key_query = """
|
||||||
|
INSERT INTO message_idempotency_keys
|
||||||
|
(
|
||||||
|
sender_id,
|
||||||
|
chat_id,
|
||||||
|
idempotency_key,
|
||||||
|
request_hash,
|
||||||
|
message_id,
|
||||||
|
expires_at
|
||||||
|
)
|
||||||
|
VALUES
|
||||||
|
(
|
||||||
|
$1,
|
||||||
|
$2,
|
||||||
|
$3,
|
||||||
|
$4,
|
||||||
|
$5,
|
||||||
|
clock_timestamp() + make_interval(secs => $6::integer)
|
||||||
|
)
|
||||||
|
ON CONFLICT (sender_id, chat_id, idempotency_key) DO UPDATE
|
||||||
|
SET
|
||||||
|
request_hash = EXCLUDED.request_hash,
|
||||||
|
message_id = EXCLUDED.message_id,
|
||||||
|
created_at = EXCLUDED.created_at,
|
||||||
|
expires_at = EXCLUDED.expires_at;
|
||||||
|
"""
|
||||||
|
async with transaction(self._pool) as conn:
|
||||||
|
await conn.execute(lock_query, CHAT_LOCK_NAMESPACE, chat_id)
|
||||||
|
stored = await conn.fetchrow(
|
||||||
|
find_key_query, sender_id, chat_id, idempotency.key
|
||||||
|
)
|
||||||
|
if stored is not None:
|
||||||
|
if stored["request_hash"] != idempotency.request_hash:
|
||||||
|
raise ConflictError(
|
||||||
|
"Idempotency-Key уже использован с другим телом запроса"
|
||||||
|
)
|
||||||
|
return self._row_to_message(stored), False
|
||||||
|
row = await conn.fetchrow(insert_query, chat_id, sender_id, text, attachment_ids)
|
||||||
|
assert row is not None
|
||||||
|
await conn.execute(
|
||||||
|
save_key_query,
|
||||||
|
sender_id,
|
||||||
|
chat_id,
|
||||||
|
idempotency.key,
|
||||||
|
idempotency.request_hash,
|
||||||
|
row["id"],
|
||||||
|
idempotency.ttl_seconds,
|
||||||
|
)
|
||||||
|
return self._row_to_message(row), True
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def list_for_chat(
|
||||||
|
self, chat_id: UUID, after: PageKey | None, limit: int
|
||||||
|
) -> list[MessageSchema]:
|
||||||
|
first_page_query = """
|
||||||
|
SELECT
|
||||||
|
id,
|
||||||
|
chat_id,
|
||||||
|
sender_id,
|
||||||
|
text,
|
||||||
|
attachment_ids,
|
||||||
|
created_at
|
||||||
|
FROM messages
|
||||||
|
WHERE chat_id = $1
|
||||||
|
ORDER BY created_at, id
|
||||||
|
LIMIT $2;
|
||||||
|
"""
|
||||||
|
next_page_query = """
|
||||||
|
SELECT
|
||||||
|
id,
|
||||||
|
chat_id,
|
||||||
|
sender_id,
|
||||||
|
text,
|
||||||
|
attachment_ids,
|
||||||
|
created_at
|
||||||
|
FROM messages
|
||||||
|
WHERE
|
||||||
|
chat_id = $1
|
||||||
|
AND (created_at, id) > ($2, $3)
|
||||||
|
ORDER BY created_at, id
|
||||||
|
LIMIT $4;
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
if after is None:
|
||||||
|
rows = await conn.fetch(first_page_query, chat_id, limit)
|
||||||
|
else:
|
||||||
|
rows = await conn.fetch(
|
||||||
|
next_page_query, chat_id, after.created_at, after.id, limit
|
||||||
|
)
|
||||||
|
return [self._row_to_message(row) for row in rows]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _row_to_message(row: asyncpg.Record) -> MessageSchema:
|
||||||
|
return MessageSchema(
|
||||||
|
id=row["id"],
|
||||||
|
chat_id=row["chat_id"],
|
||||||
|
sender_id=row["sender_id"],
|
||||||
|
text=row["text"],
|
||||||
|
attachment_ids=list(row["attachment_ids"]) if row["attachment_ids"] is not None else [],
|
||||||
|
created_at=row["created_at"],
|
||||||
|
)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .valkey import ValkeySessionCache
|
||||||
|
|
||||||
|
__all__ = ["ValkeySessionCache"]
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
import math
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from pydantic import ValidationError
|
||||||
|
from redis.exceptions import RedisError
|
||||||
|
|
||||||
|
from config import LOGGER
|
||||||
|
from domain import ActiveSessionSchema
|
||||||
|
from interfaces import SessionCache
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from infrastructure.dependencies.cache import CacheManager
|
||||||
|
|
||||||
|
KEY_PREFIX = "session:"
|
||||||
|
|
||||||
|
CACHE_UNAVAILABLE_ERRORS: tuple[type[Exception], ...] = (
|
||||||
|
RedisError,
|
||||||
|
OSError,
|
||||||
|
TimeoutError,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ValkeySessionCache(SessionCache):
|
||||||
|
"""
|
||||||
|
Read-through кэш сессий в Valkey.
|
||||||
|
|
||||||
|
Ключ session:<sha256 токена hex>, значение — JSON ActiveSessionSchema.
|
||||||
|
TTL = min(остаток жизни сессии, max_ttl): запись исчезает не позже сессии,
|
||||||
|
а max_ttl ограничивает окно устаревания, если инвалидация не дошла.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, cache: "CacheManager", max_ttl_seconds: int) -> None:
|
||||||
|
self._cache = cache
|
||||||
|
self._max_ttl_ms = max_ttl_seconds * 1000
|
||||||
|
|
||||||
|
async def get(self, token_hash: bytes) -> ActiveSessionSchema | None:
|
||||||
|
if not self._cache.is_available or self._cache.client is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
raw = await self._cache.client.get(self._key(token_hash))
|
||||||
|
except CACHE_UNAVAILABLE_ERRORS as error:
|
||||||
|
self._cache.mark_failed(error)
|
||||||
|
return None
|
||||||
|
if raw is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return ActiveSessionSchema.model_validate_json(raw)
|
||||||
|
except ValidationError:
|
||||||
|
LOGGER.warning("Повреждённая запись кэша сессии проигнорирована")
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def set(self, token_hash: bytes, session: ActiveSessionSchema) -> None:
|
||||||
|
if not self._cache.is_available or self._cache.client is None:
|
||||||
|
return
|
||||||
|
remaining = session.expires_at - datetime.now(UTC)
|
||||||
|
ttl_ms = min(self._max_ttl_ms, math.floor(remaining.total_seconds() * 1000))
|
||||||
|
if ttl_ms <= 0:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
await self._cache.client.set(
|
||||||
|
self._key(token_hash), session.model_dump_json(), px=ttl_ms
|
||||||
|
)
|
||||||
|
except CACHE_UNAVAILABLE_ERRORS as error:
|
||||||
|
self._cache.mark_failed(error)
|
||||||
|
|
||||||
|
async def delete(self, token_hash: bytes) -> bool:
|
||||||
|
if self._cache.client is None:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
await self._cache.client.delete(self._key(token_hash))
|
||||||
|
except CACHE_UNAVAILABLE_ERRORS as error:
|
||||||
|
self._cache.mark_failed(error)
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def ping(self) -> bool:
|
||||||
|
if self._cache.client is None:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
return bool(await self._cache.client.ping())
|
||||||
|
except CACHE_UNAVAILABLE_ERRORS:
|
||||||
|
return False
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _key(token_hash: bytes) -> str:
|
||||||
|
return KEY_PREFIX + token_hash.hex()
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .postgres import PostgresSessionRepository
|
||||||
|
|
||||||
|
__all__ = ["PostgresSessionRepository"]
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
from datetime import datetime
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from domain import ActiveSessionSchema, UserSchema
|
||||||
|
from infrastructure.utils import Pool, repository_logs
|
||||||
|
from interfaces import SessionRepository
|
||||||
|
|
||||||
|
|
||||||
|
class PostgresSessionRepository(SessionRepository):
|
||||||
|
def __init__(self, pool: Pool) -> None:
|
||||||
|
self._pool = pool
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def create(
|
||||||
|
self, token_hash: bytes, user_id: UUID, ttl_seconds: int
|
||||||
|
) -> datetime:
|
||||||
|
query = """
|
||||||
|
INSERT INTO sessions
|
||||||
|
(
|
||||||
|
token_hash,
|
||||||
|
user_id,
|
||||||
|
expires_at
|
||||||
|
)
|
||||||
|
VALUES
|
||||||
|
(
|
||||||
|
$1,
|
||||||
|
$2,
|
||||||
|
clock_timestamp() + make_interval(secs => $3::integer)
|
||||||
|
)
|
||||||
|
RETURNING expires_at;
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
expires_at: datetime = await conn.fetchval(
|
||||||
|
query, token_hash, user_id, ttl_seconds
|
||||||
|
)
|
||||||
|
return expires_at
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def get_active(self, token_hash: bytes) -> ActiveSessionSchema | None:
|
||||||
|
"""Поиск по PK; истёкшая или отозванная сессия не возвращается"""
|
||||||
|
query = """
|
||||||
|
SELECT
|
||||||
|
usr.id,
|
||||||
|
usr.username,
|
||||||
|
usr.display_name,
|
||||||
|
usr.created_at,
|
||||||
|
session.expires_at
|
||||||
|
FROM sessions session
|
||||||
|
JOIN users usr ON usr.id = session.user_id
|
||||||
|
WHERE
|
||||||
|
session.token_hash = $1
|
||||||
|
AND session.revoked_at IS NULL
|
||||||
|
AND session.expires_at > clock_timestamp();
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
row = await conn.fetchrow(query, token_hash)
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
return ActiveSessionSchema(
|
||||||
|
user=UserSchema(
|
||||||
|
id=row["id"],
|
||||||
|
username=row["username"],
|
||||||
|
display_name=row["display_name"],
|
||||||
|
created_at=row["created_at"],
|
||||||
|
),
|
||||||
|
expires_at=row["expires_at"],
|
||||||
|
)
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def revoke(self, token_hash: bytes) -> bool:
|
||||||
|
query = """
|
||||||
|
UPDATE sessions
|
||||||
|
SET revoked_at = clock_timestamp()
|
||||||
|
WHERE
|
||||||
|
token_hash = $1
|
||||||
|
AND revoked_at IS NULL
|
||||||
|
AND expires_at > clock_timestamp();
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
status = await conn.execute(query, token_hash)
|
||||||
|
return self._parse_affected(status) > 0
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _parse_affected(status: str) -> int:
|
||||||
|
return int(status.split()[-1])
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .postgres import PostgresUserRepository
|
||||||
|
|
||||||
|
__all__ = ["PostgresUserRepository"]
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
import asyncpg
|
||||||
|
|
||||||
|
from domain import UserCredentialsSchema, UserSchema
|
||||||
|
from domain.errors import ConflictError
|
||||||
|
from infrastructure.utils import Pool, repository_logs
|
||||||
|
from interfaces import UserRepository
|
||||||
|
|
||||||
|
|
||||||
|
class PostgresUserRepository(UserRepository):
|
||||||
|
def __init__(self, pool: Pool) -> None:
|
||||||
|
self._pool = pool
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def create(
|
||||||
|
self, username: str, display_name: str, password_hash: str
|
||||||
|
) -> UserSchema:
|
||||||
|
query = """
|
||||||
|
INSERT INTO users
|
||||||
|
(
|
||||||
|
username,
|
||||||
|
display_name,
|
||||||
|
password_hash
|
||||||
|
)
|
||||||
|
VALUES
|
||||||
|
(
|
||||||
|
$1,
|
||||||
|
$2,
|
||||||
|
$3
|
||||||
|
)
|
||||||
|
RETURNING id, username, display_name, created_at;
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
try:
|
||||||
|
row = await conn.fetchrow(query, username, display_name, password_hash)
|
||||||
|
except asyncpg.UniqueViolationError:
|
||||||
|
raise ConflictError("Имя пользователя уже занято") from None
|
||||||
|
assert row is not None
|
||||||
|
return self._row_to_user(row)
|
||||||
|
|
||||||
|
@repository_logs
|
||||||
|
async def get_credentials(self, username: str) -> UserCredentialsSchema | None:
|
||||||
|
query = """
|
||||||
|
SELECT
|
||||||
|
id,
|
||||||
|
username,
|
||||||
|
display_name,
|
||||||
|
created_at,
|
||||||
|
password_hash
|
||||||
|
FROM users
|
||||||
|
WHERE username = $1;
|
||||||
|
"""
|
||||||
|
async with self._pool.acquire() as conn:
|
||||||
|
row = await conn.fetchrow(query, username)
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
return UserCredentialsSchema(
|
||||||
|
user=self._row_to_user(row), password_hash=row["password_hash"]
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _row_to_user(row: asyncpg.Record) -> UserSchema:
|
||||||
|
return UserSchema(
|
||||||
|
id=row["id"],
|
||||||
|
username=row["username"],
|
||||||
|
display_name=row["display_name"],
|
||||||
|
created_at=row["created_at"],
|
||||||
|
)
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
from typing import AsyncGenerator
|
||||||
|
import aioboto3
|
||||||
|
from botocore.exceptions import ClientError
|
||||||
|
from config import LOGGER
|
||||||
|
|
||||||
|
class S3Client:
|
||||||
|
def __init__(self, endpoint_url: str, region_name: str, access_key: str, secret_key: str, bucket_name: str):
|
||||||
|
self.endpoint_url = endpoint_url
|
||||||
|
self.region_name = region_name
|
||||||
|
self.access_key = access_key
|
||||||
|
self.secret_key = secret_key
|
||||||
|
self.bucket_name = bucket_name
|
||||||
|
self.session = aioboto3.Session()
|
||||||
|
|
||||||
|
def _get_client(self):
|
||||||
|
return self.session.client("s3", endpoint_url=self.endpoint_url, region_name=self.region_name, aws_access_key_id=self.access_key, aws_secret_access_key=self.secret_key)
|
||||||
|
|
||||||
|
async def upload_file(self, file_path: str, object_name: str, content_type: str) -> None:
|
||||||
|
async with self._get_client() as client:
|
||||||
|
try:
|
||||||
|
await client.upload_file(file_path, self.bucket_name, object_name, ExtraArgs={"ContentType": content_type})
|
||||||
|
except ClientError as e:
|
||||||
|
LOGGER.error(f"Ошибка загрузки файла в S3: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
async def stream_file(self, object_name: str) -> AsyncGenerator[bytes, None]:
|
||||||
|
async with self._get_client() as client:
|
||||||
|
try:
|
||||||
|
response = await client.get_object(Bucket=self.bucket_name, Key=object_name)
|
||||||
|
stream = response["Body"]
|
||||||
|
while True:
|
||||||
|
chunk = await stream.read(64 * 1024)
|
||||||
|
if not chunk: break
|
||||||
|
yield chunk
|
||||||
|
except ClientError as e:
|
||||||
|
LOGGER.error(f"Ошибка чтения файла из S3: {e}")
|
||||||
|
raise
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
from .password_hasher import PasswordHasher, get_password_hasher
|
||||||
|
from .tokens import generate_token, hash_token
|
||||||
|
|
||||||
|
__all__ = ["PasswordHasher", "get_password_hasher", "generate_token", "hash_token"]
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
import asyncio
|
||||||
|
import secrets
|
||||||
|
from concurrent.futures import ThreadPoolExecutor
|
||||||
|
|
||||||
|
from argon2 import PasswordHasher as Argon2Hasher
|
||||||
|
from argon2.exceptions import InvalidHashError, VerificationError
|
||||||
|
from argon2.profiles import RFC_9106_LOW_MEMORY
|
||||||
|
from starlette.requests import HTTPConnection
|
||||||
|
|
||||||
|
|
||||||
|
class PasswordHasher:
|
||||||
|
"""
|
||||||
|
Хеширование паролей с использованием Argon2id.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, workers: int) -> None:
|
||||||
|
self._hasher = Argon2Hasher.from_parameters(RFC_9106_LOW_MEMORY)
|
||||||
|
self._executor = ThreadPoolExecutor(
|
||||||
|
max_workers=workers, thread_name_prefix="argon2"
|
||||||
|
)
|
||||||
|
self._dummy_hash = self._hasher.hash(secrets.token_urlsafe())
|
||||||
|
|
||||||
|
async def hash(self, password: str) -> str:
|
||||||
|
loop = asyncio.get_running_loop()
|
||||||
|
return await loop.run_in_executor(self._executor, self._hasher.hash, password)
|
||||||
|
|
||||||
|
async def verify(self, password_hash: str, password: str) -> bool:
|
||||||
|
loop = asyncio.get_running_loop()
|
||||||
|
return await loop.run_in_executor(
|
||||||
|
self._executor, self._verify, password_hash, password
|
||||||
|
)
|
||||||
|
|
||||||
|
async def verify_dummy(self, password: str) -> None:
|
||||||
|
"""Та же работа, что у verify, для выравнивания времени ответа"""
|
||||||
|
await self.verify(self._dummy_hash, password)
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._executor.shutdown(wait=False, cancel_futures=True)
|
||||||
|
|
||||||
|
def _verify(self, password_hash: str, password: str) -> bool:
|
||||||
|
try:
|
||||||
|
return self._hasher.verify(password_hash, password)
|
||||||
|
except (VerificationError, InvalidHashError):
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def get_password_hasher(connection: HTTPConnection) -> PasswordHasher:
|
||||||
|
"""Получение hasher, созданного в lifespan."""
|
||||||
|
hasher: PasswordHasher = connection.app.state.password_hasher
|
||||||
|
return hasher
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
import hashlib
|
||||||
|
import secrets
|
||||||
|
|
||||||
|
|
||||||
|
def generate_token() -> str:
|
||||||
|
"""Непрозрачный токен сессии: 256 бит из CSPRNG, base64url"""
|
||||||
|
return secrets.token_urlsafe(32)
|
||||||
|
|
||||||
|
|
||||||
|
def hash_token(token: str) -> bytes:
|
||||||
|
"""
|
||||||
|
SHA-256 токена для хранения и поиска.
|
||||||
|
|
||||||
|
Токен уже имеет 256 бит энтропии, перебор невозможен, поэтому быстрый hash
|
||||||
|
достаточен; медленный KDF нужен паролям, у которых энтропия низкая.
|
||||||
|
"""
|
||||||
|
return hashlib.sha256(token.encode()).digest()
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
from .auth_service import AuthService
|
||||||
|
from .chat_service import ChatService
|
||||||
|
from .message_service import MessageService
|
||||||
|
from .user_service import UserService
|
||||||
|
from .attachment_service import AttachmentService
|
||||||
|
|
||||||
|
__all__ = ["UserService", "AuthService", "ChatService", "MessageService", "AttachmentService", ]
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
from .attachment_service import AttachmentService
|
||||||
|
__all__ = ["AttachmentService"]
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
import struct
|
||||||
|
import os
|
||||||
|
from uuid import uuid4
|
||||||
|
from datetime import datetime, UTC
|
||||||
|
from config import LOGGER
|
||||||
|
from domain import AttachmentSchema
|
||||||
|
from domain.errors import AppError, ErrorCode, InvalidRequestError, NotFoundError
|
||||||
|
from interfaces import AttachmentRepository, ChatRepository
|
||||||
|
from infrastructure.s3_client import S3Client
|
||||||
|
|
||||||
|
class PayloadTooLargeError(AppError):
|
||||||
|
status_code = 413
|
||||||
|
code = ErrorCode.PAYLOAD_TOO_LARGE
|
||||||
|
default_message = "Превышен лимит размера"
|
||||||
|
|
||||||
|
class UnsupportedMediaTypeError(AppError):
|
||||||
|
status_code = 415
|
||||||
|
code = ErrorCode.UNSUPPORTED_MEDIA_TYPE
|
||||||
|
default_message = "Неподдерживаемый формат"
|
||||||
|
|
||||||
|
def validate_image_and_get_mime(file_path: str) -> str:
|
||||||
|
with open(file_path, "rb") as f:
|
||||||
|
header = f.read(24)
|
||||||
|
|
||||||
|
if header.startswith(b"\x89PNG\r\n\x1a\n"):
|
||||||
|
w, h = struct.unpack(">II", header[16:24])
|
||||||
|
if w * h > 40_000_000:
|
||||||
|
raise PayloadTooLargeError("Разрешение больше 40М пикселей")
|
||||||
|
return "image/png"
|
||||||
|
elif header[:2] == b"\xff\xd8":
|
||||||
|
with open(file_path, "rb") as f:
|
||||||
|
data = f.read()
|
||||||
|
i = 2
|
||||||
|
sof = {0xc0, 0xc1, 0xc2, 0xc3, 0xc5, 0xc6, 0xc7, 0xc9, 0xca, 0xcb, 0xcd, 0xce, 0xcf}
|
||||||
|
while i < len(data):
|
||||||
|
if data[i] != 0xff: raise UnsupportedMediaTypeError("Поврежденный JPEG")
|
||||||
|
while i < len(data) and data[i] == 0xff: i += 1
|
||||||
|
if i >= len(data): break
|
||||||
|
marker = data[i]
|
||||||
|
i += 1
|
||||||
|
if marker in {0xd9, 0xda}: break
|
||||||
|
if marker in {0x01, 0xd8} or 0xd0 <= marker <= 0xd7: continue
|
||||||
|
if i + 2 > len(data): break
|
||||||
|
length = int.from_bytes(data[i:i+2], "big")
|
||||||
|
if marker in sof:
|
||||||
|
if i + 7 > len(data): break
|
||||||
|
h = int.from_bytes(data[i+3:i+5], "big")
|
||||||
|
w = int.from_bytes(data[i+5:i+7], "big")
|
||||||
|
if w * h > 40_000_000:
|
||||||
|
raise PayloadTooLargeError("Разрешение больше 40М пикселей")
|
||||||
|
return "image/jpeg"
|
||||||
|
i += length
|
||||||
|
raise UnsupportedMediaTypeError("Не удалось найти размеры JPEG")
|
||||||
|
raise UnsupportedMediaTypeError("Формат не поддерживается")
|
||||||
|
|
||||||
|
class AttachmentService:
|
||||||
|
def __init__(self, s3_client: S3Client, attachment_repo: AttachmentRepository, chat_repo: ChatRepository):
|
||||||
|
self._s3 = s3_client
|
||||||
|
self._attachments = attachment_repo
|
||||||
|
self._chats = chat_repo
|
||||||
|
|
||||||
|
async def process_and_save(self, temp_file_path: str, filename: str, chat_id, owner_id, size_bytes: int, sha256_hash: str) -> AttachmentSchema:
|
||||||
|
if "/" in filename or "\\" in filename or "\0" in filename:
|
||||||
|
raise InvalidRequestError("Недопустимое имя файла")
|
||||||
|
|
||||||
|
chat = await self._chats.get_for_member(chat_id, owner_id)
|
||||||
|
if not chat:
|
||||||
|
raise NotFoundError()
|
||||||
|
|
||||||
|
mime = validate_image_and_get_mime(temp_file_path)
|
||||||
|
attachment_id = uuid4()
|
||||||
|
object_name = f"{chat_id}/{attachment_id}"
|
||||||
|
|
||||||
|
await self._s3.upload_file(temp_file_path, object_name, mime)
|
||||||
|
LOGGER.info(f"Файл {attachment_id} загружен в S3")
|
||||||
|
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
attachment = AttachmentSchema(
|
||||||
|
id=attachment_id, chat_id=chat_id, owner_id=owner_id,
|
||||||
|
filename=filename, content_type=mime, size_bytes=size_bytes,
|
||||||
|
sha256=sha256_hash, status="ready", variants=[],
|
||||||
|
error_code=None, created_at=now, updated_at=now
|
||||||
|
)
|
||||||
|
return await self._attachments.create(attachment)
|
||||||
|
|
||||||
|
async def get_attachment_meta(self, attachment_id, user_id) -> AttachmentSchema:
|
||||||
|
att = await self._attachments.get_by_id(attachment_id)
|
||||||
|
if not att or not await self._chats.get_for_member(att.chat_id, user_id):
|
||||||
|
raise NotFoundError()
|
||||||
|
return att
|
||||||
|
|
||||||
|
async def get_attachment_stream(self, attachment_id, user_id):
|
||||||
|
att = await self.get_attachment_meta(attachment_id, user_id)
|
||||||
|
if att.status != "ready":
|
||||||
|
from domain.errors import ConflictError
|
||||||
|
raise ConflictError("Файл еще не готов")
|
||||||
|
object_name = f"{att.chat_id}/{att.id}"
|
||||||
|
return self._s3.stream_file(object_name), att.content_type
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .auth_service import AuthService
|
||||||
|
|
||||||
|
__all__ = ["AuthService"]
|
||||||
@@ -0,0 +1,92 @@
|
|||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from config import LOGGER
|
||||||
|
from domain import BasicCredentialsSchema, SessionSchema, UserSchema
|
||||||
|
from domain.errors import InvalidCredentialsError, UnauthorizedError
|
||||||
|
from infrastructure.security import PasswordHasher, generate_token, hash_token
|
||||||
|
from interfaces import SessionCache, SessionRepository, UserRepository
|
||||||
|
|
||||||
|
TOKEN_MAX_LENGTH = 256
|
||||||
|
|
||||||
|
|
||||||
|
class AuthService:
|
||||||
|
"""
|
||||||
|
Basic → opaque Bearer-сессия.
|
||||||
|
|
||||||
|
Источник истины — SessionRepository (PostgreSQL), SessionCache (Valkey) —
|
||||||
|
read-through кэш: при промахе или недоступности кэша сессия читается из БД.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
user_repository: UserRepository,
|
||||||
|
session_repository: SessionRepository,
|
||||||
|
session_cache: SessionCache,
|
||||||
|
hasher: PasswordHasher,
|
||||||
|
session_ttl_seconds: int,
|
||||||
|
) -> None:
|
||||||
|
self._users = user_repository
|
||||||
|
self._sessions = session_repository
|
||||||
|
self._cache = session_cache
|
||||||
|
self._hasher = hasher
|
||||||
|
self._session_ttl = session_ttl_seconds
|
||||||
|
|
||||||
|
async def login(self, credentials: BasicCredentialsSchema) -> SessionSchema:
|
||||||
|
"""
|
||||||
|
Создание сессии по username и паролю.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
InvalidCredentialsError: неизвестный username или неверный пароль —
|
||||||
|
одинаковый ответ и сопоставимое время
|
||||||
|
"""
|
||||||
|
found = await self._users.get_credentials(credentials.username)
|
||||||
|
if found is None:
|
||||||
|
await self._hasher.verify_dummy(credentials.password)
|
||||||
|
raise InvalidCredentialsError()
|
||||||
|
if not await self._hasher.verify(found.password_hash, credentials.password):
|
||||||
|
raise InvalidCredentialsError()
|
||||||
|
token = generate_token()
|
||||||
|
expires_at = await self._sessions.create(
|
||||||
|
hash_token(token), found.user.id, self._session_ttl
|
||||||
|
)
|
||||||
|
LOGGER.info(f"Создана сессия пользователя {found.user.id}")
|
||||||
|
return SessionSchema(token=token, expires_at=expires_at, user=found.user)
|
||||||
|
|
||||||
|
async def authenticate(self, token: str | None) -> UserSchema:
|
||||||
|
"""
|
||||||
|
Владелец действующей сессии.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
UnauthorizedError: токена нет, он неизвестен, истёк или отозван
|
||||||
|
UnavailableError: кэш промахнулся, а PostgreSQL недоступен
|
||||||
|
"""
|
||||||
|
if not token or len(token) > TOKEN_MAX_LENGTH:
|
||||||
|
raise UnauthorizedError()
|
||||||
|
token_hash = hash_token(token)
|
||||||
|
session = await self._cache.get(token_hash)
|
||||||
|
if session is not None and session.expires_at > datetime.now(UTC):
|
||||||
|
return session.user
|
||||||
|
session = await self._sessions.get_active(token_hash)
|
||||||
|
if session is None:
|
||||||
|
raise UnauthorizedError()
|
||||||
|
await self._cache.set(token_hash, session)
|
||||||
|
return session.user
|
||||||
|
|
||||||
|
async def logout(self, token: str | None) -> None:
|
||||||
|
"""
|
||||||
|
Отзыв в источнике истины, затем инвалидация кэша.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
UnauthorizedError: действующей сессии с таким токеном нет
|
||||||
|
"""
|
||||||
|
if not token or len(token) > TOKEN_MAX_LENGTH:
|
||||||
|
raise UnauthorizedError()
|
||||||
|
token_hash = hash_token(token)
|
||||||
|
if not await self._sessions.revoke(token_hash):
|
||||||
|
raise UnauthorizedError()
|
||||||
|
if not await self._cache.delete(token_hash):
|
||||||
|
LOGGER.warning(
|
||||||
|
"Сессия отозвана в PostgreSQL, но запись кэша не удалена: "
|
||||||
|
"она истечёт по TTL кэша"
|
||||||
|
)
|
||||||
|
LOGGER.info("Сессия отозвана")
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
from .chat_service import ChatService
|
||||||
|
|
||||||
|
__all__ = ["ChatService"]
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user