Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9c5c004625 | ||
|
|
749109aaf4 | ||
|
|
d475bf29ed | ||
|
|
3571422836 |
+22
@@ -0,0 +1,22 @@
|
|||||||
|
FROM python:3.11.7-slim-bookworm AS builder
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY requirements.txt .
|
||||||
|
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
|
||||||
|
|
||||||
|
FROM python:3.11.7-slim-bookworm AS runner
|
||||||
|
|
||||||
|
RUN useradd -u 10001 -m appuser
|
||||||
|
WORKDIR /home/appuser
|
||||||
|
|
||||||
|
COPY --from=builder /install /usr/local
|
||||||
|
COPY app.py worker.py ./
|
||||||
|
|
||||||
|
USER appuser
|
||||||
|
EXPOSE 8080
|
||||||
|
|
||||||
|
HEALTHCHECK --interval=5s --timeout=3s --retries=3 \
|
||||||
|
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8080/health/ready')"
|
||||||
|
|
||||||
|
ENV PYTHONUNBUFFERED=1
|
||||||
|
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080", "--timeout-graceful-shutdown", "10"]
|
||||||
@@ -1,37 +1,73 @@
|
|||||||
# Отчёт — лабораторная 5
|
# Отчёт — лабораторная 5
|
||||||
|
|
||||||
- Автор, группа: TODO
|
- Автор, группа: Бондаренко Мария, БПИ2302
|
||||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
- Целевая оценка: 4
|
||||||
- Commit/PR: TODO после публикации
|
|
||||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
|
||||||
|
|
||||||
## Запуск из чистого clone
|
|
||||||
|
|
||||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
|
||||||
|
|
||||||
## Архитектура и решения
|
## Архитектура и решения
|
||||||
|
|
||||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab4, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
Система построена по микросервисной/событийно-ориентированной схеме с разделением синхронного HTTP API и асинхронного фонового обработчика медиафайлов:
|
||||||
|
|
||||||
## Доказательства критериев
|
1. API-сервисы (api-1, api-2):
|
||||||
|
- Реализованы на базе FastAPI, балансируются через Nginx
|
||||||
|
- При загрузке изображения выполняют первичную валидацию сигнатур (PNG/JPEG), проверку размера и разрешения
|
||||||
|
- Сохраняют исходный файл во временную зону объектного хранилища S3
|
||||||
|
- В рамках единой транзакции PostgreSQL фиксируют метаданные вложения в таблице attachments и записывают событие в таблицу outbox
|
||||||
|
2. Транзакционный Outbox:
|
||||||
|
- Фоновый поток в API-инстансах вычитывает неопубликованные записи, отправляет задачи в стрим брокера attachment_jobs и проставляет published_at. Это исключает потерю задач при сбое между COMMIT в БД и публикацией в брокер
|
||||||
|
3. Брокер сообщений:
|
||||||
|
- Запущен с включенной персистентностью
|
||||||
|
- Используются Consumer Groups для балансировки нагрузки между воркерами и подтверждения обработки
|
||||||
|
4. Объектное хранилище:
|
||||||
|
- Хранит временные файлы в staging/ и финальные артефакты в attachments/{chat_id}/{attachment_id}/{variant}
|
||||||
|
5. Фоновый обработчик:
|
||||||
|
- Многопоточный воркер
|
||||||
|
- Вычитывает задачи пачками и обрабатывает их идемпотентно с блокировкой строки в БД
|
||||||
|
|
||||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
## Пайплайн обработки изображений и варианты (Variants)
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| Публичный smoke | make test | TODO | TODO |
|
|
||||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
|
||||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
|
||||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
|
||||||
|
|
||||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
Перед генерацией производных форматов исходное изображение проходит нормализацию:
|
||||||
|
- EXIF-ориентация: применяется ImageOps.exif_transpose(img) для корректного поворота фотографий с мобильных устройств перед ресайзом и кропом
|
||||||
|
- Цветовое пространство: изображения с альфа-каналом (RGBA, LA, P с прозрачностью) накладываются на белый фон (255, 255, 255) и приводятся к RGB
|
||||||
|
- Защита от Decompression Bomb: установлено ограничение Image.MAX_IMAGE_PIXELS = 40000000
|
||||||
|
|
||||||
## Эксперименты и отказы
|
## Спецификация генерируемых вариантов (Grade 4):
|
||||||
|
1. thumbnail (превью для списка/чата):
|
||||||
|
- Формат и качество: image/jpeg, quality=85, optimize=True.
|
||||||
|
- Геометрия: пропорциональное масштабирование с вписыванием в квадрат 256256 пикселей (без апскейла, если оригинал меньше)
|
||||||
|
2. optimized (оптимизированный просмотр):
|
||||||
|
- Формат и качество: image/webp, quality=85.
|
||||||
|
- Геометрия: пропорциональное масштабирование с вписыванием в рамку 12801280 пикселей (без апскейла)
|
||||||
|
3. crop (квадратная миниатюра/аватар):
|
||||||
|
- Формат и качество: image/jpeg, quality=85, optimize=True.
|
||||||
|
- Геометрия: центральное квадратное кадрирование с последующим приведением к точному размеру 128128 пикселей
|
||||||
|
|
||||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
|
||||||
|
|
||||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
## Отказоустойчивость, ретраи и восстановление
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| TODO | TODO | TODO | TODO |
|
|
||||||
|
|
||||||
## Ограничения и следующие шаги
|
Для обеспечения надежности уровня на оценку 4 реализованы следующие механизмы:
|
||||||
|
|
||||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
- Разделение ошибок (Permanent / Transient):
|
||||||
|
- Постоянные ошибки сразу переводят вложение в терминальный статус failed с соответствующим error_code и подтверждаются в брокере без бесполезных повторов
|
||||||
|
- Временные ошибки обрабатываются с повторами до 5 раз
|
||||||
|
- Exponential Backoff + Jitter:
|
||||||
|
- Перед постановкой следующей попытки ыполняется задержка для предотвращения лавинообразной нагрузки на S3/БД
|
||||||
|
- Dead Letter Queue (DLQ) и ручной Replay:
|
||||||
|
- При исчерпании попыток вложение получает статус failed, а сообщение перекладывается в отдельный стрим attachment_jobs_dlq
|
||||||
|
- Реализован эндпоинт POST /api/v1/attachments/{attachment_id}/replay, который сбрасывает счетчик попыток, переводит статус обратно в queued и транзакционно создает новую задачу в outbox
|
||||||
|
- Защита от зависаний:
|
||||||
|
- Каждая задача ограничена по времени выполнения (30 секунд).
|
||||||
|
- Фоновый цикл воркера каждые 15 секунд вызывает XAUTOCLAIM, перехватывая зависшие сообщения от упавших экземпляров воркеров
|
||||||
|
- Очистка временного хранилища:
|
||||||
|
- Воркер периодически (раз в 10 минут) сканирует префикс staging/ в S3 и удаляет осиротевшие объекты старше 1 часа.
|
||||||
|
|
||||||
|
## Результаты проверки и артефакты
|
||||||
|
|
||||||
|
1. Контрактные и функциональные тесты:
|
||||||
|
- scripts/check_contract.py — подтверждено соответствие спецификации OpenAPI, наличие заголовков X-Request-Id и X-Instance-Id
|
||||||
|
- tests/smoke.py (evidence/smoke.txt) — пройден полный публичный сценарий
|
||||||
|
2. Демонстрация асинхронности и переживания рестартов:
|
||||||
|
- При остановленном контейнере worker (demo-01c-worker-stopped.txt) загружен файл sample.png (9becbd32-e413-417a-8762-79bcc195e296), получен ответ 202 Accepted со статусом queued (demo-02-uppload-202.json).
|
||||||
|
- Выполнены рестарты API-реплик (demo-04-api-restarted.txt) и брокера Redis (demo-08-after-broker-restart.json).
|
||||||
|
- После запуска воркера (demo-05-worker-started.txt) задача успешно обработана из персистентной очереди, статус перешел в ready (demo-06-ready.json), а вариант thumbnail корректно отдан клиенту (demo-07-thumbnail.jpg).
|
||||||
|
3. Артефакты расширенного пайплайна:
|
||||||
|
- Сформированы и верифицированы все три варианта (g4-variants.txt, g4-thunbnail.jpg, g4-crop.jpg) с корректными MIME-типами (image/jpeg, image/webp) и размерами
|
||||||
+218
@@ -0,0 +1,218 @@
|
|||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
image: postgres:15.5-alpine3.19
|
||||||
|
environment:
|
||||||
|
POSTGRES_DB: messenger
|
||||||
|
POSTGRES_USER: postgres
|
||||||
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
|
||||||
|
volumes:
|
||||||
|
- pgdata:/var/lib/postgresql/data
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '0.50'
|
||||||
|
memory: 512M
|
||||||
|
reservations:
|
||||||
|
cpus: '0.10'
|
||||||
|
memory: 128M
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U postgres -d messenger"]
|
||||||
|
interval: 3s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 5
|
||||||
|
|
||||||
|
redis:
|
||||||
|
image: redis:7.2.4-alpine
|
||||||
|
command: redis-server --appendonly yes --appendfsync everysec --loglevel warning
|
||||||
|
volumes:
|
||||||
|
- redisdata:/data
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '0.25'
|
||||||
|
memory: 256M
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "redis-cli", "ping"]
|
||||||
|
interval: 3s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 5
|
||||||
|
|
||||||
|
migrate:
|
||||||
|
image: postgres:15.5-alpine3.19
|
||||||
|
environment:
|
||||||
|
PGPASSWORD: ${POSTGRES_PASSWORD}
|
||||||
|
volumes:
|
||||||
|
- ./migrations:/migrations
|
||||||
|
command: >
|
||||||
|
sh -c "until pg_isready -h postgres -U postgres; do sleep 1; done;
|
||||||
|
psql -h postgres -U postgres -d messenger -f /migrations/init.sql"
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
restart: "no"
|
||||||
|
|
||||||
|
s3:
|
||||||
|
image: chrislusf/seaweedfs:latest
|
||||||
|
command: ["server", "-s3", "-dir=/data", "-s3.port=8333"]
|
||||||
|
environment:
|
||||||
|
- AWS_ACCESS_KEY_ID=appuser
|
||||||
|
- AWS_SECRET_ACCESS_KEY=appsecretpassword
|
||||||
|
volumes:
|
||||||
|
- s3data:/data
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '0.50'
|
||||||
|
memory: 512M
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1:8333/healthz || exit 1"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 5
|
||||||
|
start_period: 10s
|
||||||
|
|
||||||
|
s3-init:
|
||||||
|
image: python:3.11.7-slim-bookworm
|
||||||
|
volumes:
|
||||||
|
- ./scripts:/scripts
|
||||||
|
- ./requirements.txt:/requirements.txt
|
||||||
|
environment:
|
||||||
|
- S3_ENDPOINT=http://s3:8333
|
||||||
|
- S3_BUCKET=messenger-attachments
|
||||||
|
- S3_ROOT_ACCESS_KEY=appuser
|
||||||
|
- S3_ROOT_SECRET_KEY=appsecretpassword
|
||||||
|
command: >
|
||||||
|
sh -c "pip install --no-cache-dir -r /requirements.txt &&
|
||||||
|
python /scripts/bootstrap_s3.py"
|
||||||
|
depends_on:
|
||||||
|
s3:
|
||||||
|
condition: service_healthy
|
||||||
|
restart: "no"
|
||||||
|
|
||||||
|
api-1:
|
||||||
|
build: .
|
||||||
|
stop_grace_period: 15s
|
||||||
|
environment:
|
||||||
|
- DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432/messenger
|
||||||
|
- REDIS_URL=redis://redis:6379/0
|
||||||
|
- SESSION_TTL_SECONDS=3600
|
||||||
|
- INSTANCE_ID=api-replica-1
|
||||||
|
- SECRET_HMAC_KEY=${SECRET_HMAC_KEY}
|
||||||
|
- S3_ENDPOINT=http://s3:8333
|
||||||
|
- S3_BUCKET=messenger-attachments
|
||||||
|
- S3_ACCESS_KEY_ID=appuser
|
||||||
|
- S3_SECRET_ACCESS_KEY=appsecretpassword
|
||||||
|
- BROKER_STREAM=attachment_jobs
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '0.50'
|
||||||
|
memory: 256M
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
migrate:
|
||||||
|
condition: service_completed_successfully
|
||||||
|
s3-init:
|
||||||
|
condition: service_completed_successfully
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8080/health/ready')"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 3
|
||||||
|
|
||||||
|
api-2:
|
||||||
|
build: .
|
||||||
|
stop_grace_period: 15s
|
||||||
|
environment:
|
||||||
|
- DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432/messenger
|
||||||
|
- REDIS_URL=redis://redis:6379/0
|
||||||
|
- SESSION_TTL_SECONDS=3600
|
||||||
|
- INSTANCE_ID=api-replica-2
|
||||||
|
- SECRET_HMAC_KEY=${SECRET_HMAC_KEY}
|
||||||
|
- S3_ENDPOINT=http://s3:8333
|
||||||
|
- S3_BUCKET=messenger-attachments
|
||||||
|
- S3_ACCESS_KEY_ID=appuser
|
||||||
|
- S3_SECRET_ACCESS_KEY=appsecretpassword
|
||||||
|
- BROKER_STREAM=attachment_jobs
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '0.50'
|
||||||
|
memory: 256M
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
migrate:
|
||||||
|
condition: service_completed_successfully
|
||||||
|
s3-init:
|
||||||
|
condition: service_completed_successfully
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8080/health/ready')"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 3
|
||||||
|
|
||||||
|
worker:
|
||||||
|
build: .
|
||||||
|
command: ["python", "worker.py"]
|
||||||
|
stop_grace_period: 15s
|
||||||
|
environment:
|
||||||
|
- DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432/messenger
|
||||||
|
- REDIS_URL=redis://redis:6379/0
|
||||||
|
- S3_ENDPOINT=http://s3:8333
|
||||||
|
- S3_BUCKET=messenger-attachments
|
||||||
|
- S3_ACCESS_KEY_ID=appuser
|
||||||
|
- S3_SECRET_ACCESS_KEY=appsecretpassword
|
||||||
|
- BROKER_STREAM=attachment_jobs
|
||||||
|
- BROKER_GROUP=attachment_workers
|
||||||
|
- JOB_MAX_ATTEMPTS=5
|
||||||
|
- MAX_IMAGE_PIXELS=40000000
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '0.50'
|
||||||
|
memory: 256M
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
migrate:
|
||||||
|
condition: service_completed_successfully
|
||||||
|
s3-init:
|
||||||
|
condition: service_completed_successfully
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "python", "-c", "import os, redis; redis.Redis.from_url(os.environ['REDIS_URL']).ping()"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 3
|
||||||
|
|
||||||
|
nginx:
|
||||||
|
image: nginx:1.25.4-alpine
|
||||||
|
ports:
|
||||||
|
- "8080:8080"
|
||||||
|
volumes:
|
||||||
|
- ./nginx.conf:/etc/nginx/nginx.conf:ro
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: '0.25'
|
||||||
|
memory: 128M
|
||||||
|
depends_on:
|
||||||
|
- api-1
|
||||||
|
- api-2
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "nginx", "-t"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 3
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
pgdata:
|
||||||
|
redisdata:
|
||||||
|
s3data:
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
{
|
||||||
|
"id": "fdeb71d4-ee4f-47a6-8968-ffc7740e5242",
|
||||||
|
"username": "demo_75866",
|
||||||
|
"display_name": "Demo User",
|
||||||
|
"created_at": "2026-11-16T13:43:38.059472Z"
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"token": "grSsHp1gJpFLckF14UT44obEonSGQRvP878JhLyCaNzM6PGBH1LwrdgZIFF8Zu8U",
|
||||||
|
"token_type": "Bearer",
|
||||||
|
"expires_at": "2026-11-16T14:43:38.850596Z",
|
||||||
|
"user": {
|
||||||
|
"id": "fdeb71d4-ee4f-47a6-8968-ffc7740e5242",
|
||||||
|
"username": "demo_75866",
|
||||||
|
"display_name": "Demo User",
|
||||||
|
"created_at": "2026-11-16T13:43:38.059472Z"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"id": "5db90ff5-9604-4853-b002-db4144868bcb",
|
||||||
|
"title": "Demo chat",
|
||||||
|
"member_ids": [
|
||||||
|
"fdeb71d4-ee4f-47a6-8968-ffc7740e5242"
|
||||||
|
],
|
||||||
|
"created_at": "2026-11-16T13:43:39.240968Z"
|
||||||
|
}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||||
|
mtusi2026lab5-api-1-1 mtusi2026lab5-api-1 "uvicorn app:app --hтАж" api-1 30 minutes ago Up 6 minutes (healthy) 8080/tcp
|
||||||
|
mtusi2026lab5-api-2-1 mtusi2026lab5-api-2 "uvicorn app:app --hтАж" api-2 30 minutes ago Up 6 minutes (healthy) 8080/tcp
|
||||||
|
mtusi2026lab5-migrate-1 postgres:15.5-alpine3.19 "docker-entrypoint.sтАж" migrate 30 minutes ago Exited (0) 5 minutes ago
|
||||||
|
mtusi2026lab5-nginx-1 nginx:1.25.4-alpine "/docker-entrypoint.тАж" nginx 30 minutes ago Up 25 minutes (healthy) 0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp
|
||||||
|
mtusi2026lab5-postgres-1 postgres:15.5-alpine3.19 "docker-entrypoint.sтАж" postgres 30 minutes ago Up 30 minutes (healthy) 5432/tcp
|
||||||
|
mtusi2026lab5-redis-1 redis:7.2.4-alpine "docker-entrypoint.sтАж" redis 30 minutes ago Up 4 minutes (healthy) 6379/tcp
|
||||||
|
mtusi2026lab5-s3-1 chrislusf/seaweedfs:latest "/entrypoint.sh servтАж" s3 30 minutes ago Up 30 minutes (healthy) 7333/tcp, 8080/tcp, 8333/tcp, 8888/tcp, 9333/tcp, 18080/tcp, 18888/tcp, 19333/tcp
|
||||||
|
mtusi2026lab5-s3-init-1 python:3.11.7-slim-bookworm "sh -c 'pip install тАж" s3-init 30 minutes ago Exited (0) 5 minutes ago
|
||||||
|
mtusi2026lab5-worker-1 mtusi2026lab5-worker "python worker.py" worker 30 minutes ago Exited (0) Less than a second ago
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
{"id":"9becbd32-e413-417a-8762-79bcc195e296","chat_id":"5db90ff5-9604-4853-b002-db4144868bcb","owner_id":"fdeb71d4-ee4f-47a6-8968-ffc7740e5242","filename":"sample.png","content_type":"image/png","size_bytes":111817,"sha256":"99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75","status":"queued","variants":[],"error_code":null,"created_at":"2026-11-16T13:43:41.734766Z","updated_at":"2026-11-16T13:43:41.734766Z"}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
{
|
||||||
|
"id": "74dc2e91-a901-4067-8993-da9d2cc4ca55",
|
||||||
|
"chat_id": "5db90ff5-9604-4853-b002-db4144868bcb",
|
||||||
|
"sender_id": "fdeb71d4-ee4f-47a6-8968-ffc7740e5242",
|
||||||
|
"text": "",
|
||||||
|
"created_at": "2026-11-16T13:43:42.100558Z",
|
||||||
|
"attachment_ids": [
|
||||||
|
"9becbd32-e413-417a-8762-79bcc195e296"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||||
|
mtusi2026lab5-api-1-1 mtusi2026lab5-api-1 "uvicorn app:app --hтАж" api-1 30 minutes ago Up Less than a second (health: starting) 8080/tcp
|
||||||
|
mtusi2026lab5-api-2-1 mtusi2026lab5-api-2 "uvicorn app:app --hтАж" api-2 30 minutes ago Up Less than a second (health: starting) 8080/tcp
|
||||||
|
mtusi2026lab5-nginx-1 nginx:1.25.4-alpine "/docker-entrypoint.тАж" nginx 30 minutes ago Up 25 minutes (healthy) 0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp
|
||||||
|
mtusi2026lab5-postgres-1 postgres:15.5-alpine3.19 "docker-entrypoint.sтАж" postgres 30 minutes ago Up 30 minutes (healthy) 5432/tcp
|
||||||
|
mtusi2026lab5-redis-1 redis:7.2.4-alpine "docker-entrypoint.sтАж" redis 30 minutes ago Up 4 minutes (healthy) 6379/tcp
|
||||||
|
mtusi2026lab5-s3-1 chrislusf/seaweedfs:latest "/entrypoint.sh servтАж" s3 30 minutes ago Up 30 minutes (healthy) 7333/tcp, 8080/tcp, 8333/tcp, 8888/tcp, 9333/tcp, 18080/tcp, 18888/tcp, 19333/tcp
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||||
|
mtusi2026lab5-api-1-1 mtusi2026lab5-api-1 "uvicorn app:app --hтАж" api-1 30 minutes ago Up 10 seconds (healthy) 8080/tcp
|
||||||
|
mtusi2026lab5-api-2-1 mtusi2026lab5-api-2 "uvicorn app:app --hтАж" api-2 30 minutes ago Up 9 seconds (healthy) 8080/tcp
|
||||||
|
mtusi2026lab5-migrate-1 postgres:15.5-alpine3.19 "docker-entrypoint.sтАж" migrate 30 minutes ago Exited (0) 7 seconds ago
|
||||||
|
mtusi2026lab5-nginx-1 nginx:1.25.4-alpine "/docker-entrypoint.тАж" nginx 30 minutes ago Up 26 minutes (healthy) 0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp
|
||||||
|
mtusi2026lab5-postgres-1 postgres:15.5-alpine3.19 "docker-entrypoint.sтАж" postgres 30 minutes ago Up 30 minutes (healthy) 5432/tcp
|
||||||
|
mtusi2026lab5-redis-1 redis:7.2.4-alpine "docker-entrypoint.sтАж" redis 30 minutes ago Up 4 minutes (healthy) 6379/tcp
|
||||||
|
mtusi2026lab5-s3-1 chrislusf/seaweedfs:latest "/entrypoint.sh servтАж" s3 30 minutes ago Up 30 minutes (healthy) 7333/tcp, 8080/tcp, 8333/tcp, 8888/tcp, 9333/tcp, 18080/tcp, 18888/tcp, 19333/tcp
|
||||||
|
mtusi2026lab5-s3-init-1 python:3.11.7-slim-bookworm "sh -c 'pip install тАж" s3-init 30 minutes ago Exited (0) Less than a second ago
|
||||||
|
mtusi2026lab5-worker-1 mtusi2026lab5-worker "python worker.py" worker 30 minutes ago Up Less than a second (health: starting) 8080/tcp
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"id": "9becbd32-e413-417a-8762-79bcc195e296",
|
||||||
|
"chat_id": "5db90ff5-9604-4853-b002-db4144868bcb",
|
||||||
|
"owner_id": "fdeb71d4-ee4f-47a6-8968-ffc7740e5242",
|
||||||
|
"filename": "sample.png",
|
||||||
|
"content_type": "image/png",
|
||||||
|
"size_bytes": 111817,
|
||||||
|
"sha256": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||||
|
"status": "ready",
|
||||||
|
"variants": [
|
||||||
|
{
|
||||||
|
"name": "thumbnail",
|
||||||
|
"width": 256,
|
||||||
|
"height": 160,
|
||||||
|
"sha256": "7bc40bf315c8e18e907889b23514d978be213a44223d64b737290188ee808594",
|
||||||
|
"size_bytes": 4679,
|
||||||
|
"content_type": "image/jpeg"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"error_code": null,
|
||||||
|
"created_at": "2026-11-16T13:43:41.734766Z",
|
||||||
|
"updated_at": "2026-11-16T13:43:55.375481Z"
|
||||||
|
}
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 4.6 KiB |
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"id": "9becbd32-e413-417a-8762-79bcc195e296",
|
||||||
|
"chat_id": "5db90ff5-9604-4853-b002-db4144868bcb",
|
||||||
|
"owner_id": "fdeb71d4-ee4f-47a6-8968-ffc7740e5242",
|
||||||
|
"filename": "sample.png",
|
||||||
|
"content_type": "image/png",
|
||||||
|
"size_bytes": 111817,
|
||||||
|
"sha256": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||||
|
"status": "ready",
|
||||||
|
"variants": [
|
||||||
|
{
|
||||||
|
"name": "thumbnail",
|
||||||
|
"width": 256,
|
||||||
|
"height": 160,
|
||||||
|
"sha256": "7bc40bf315c8e18e907889b23514d978be213a44223d64b737290188ee808594",
|
||||||
|
"size_bytes": 4679,
|
||||||
|
"content_type": "image/jpeg"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"error_code": null,
|
||||||
|
"created_at": "2026-11-16T13:43:41.734766Z",
|
||||||
|
"updated_at": "2026-11-16T13:43:55.375481Z"
|
||||||
|
}
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 1.8 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.6 KiB |
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,74 @@
|
|||||||
|
CREATE TABLE IF NOT EXISTS users (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
username VARCHAR(32) UNIQUE NOT NULL,
|
||||||
|
display_name VARCHAR(100) NOT NULL,
|
||||||
|
password_hash VARCHAR(255) NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS chats (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
title VARCHAR(100) NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS chat_members (
|
||||||
|
chat_id UUID REFERENCES chats(id) ON DELETE CASCADE,
|
||||||
|
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
PRIMARY KEY (chat_id, user_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS messages (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
chat_id UUID REFERENCES chats(id) ON DELETE CASCADE,
|
||||||
|
sender_id UUID REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
text VARCHAR(4000) NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL,
|
||||||
|
attachment_ids UUID[] DEFAULT '{}'
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_messages_chat_pagination ON messages(chat_id, created_at, id);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS idempotency_keys (
|
||||||
|
user_id UUID NOT NULL,
|
||||||
|
method VARCHAR(16) NOT NULL,
|
||||||
|
path TEXT NOT NULL,
|
||||||
|
key VARCHAR(128) NOT NULL,
|
||||||
|
request_hash VARCHAR(64) NOT NULL,
|
||||||
|
response_status INT NOT NULL,
|
||||||
|
response_body JSONB NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
PRIMARY KEY (user_id, method, path, key)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS attachments (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
chat_id UUID REFERENCES chats(id) ON DELETE CASCADE,
|
||||||
|
owner_id UUID REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
filename VARCHAR(255) NOT NULL,
|
||||||
|
content_type VARCHAR(50) NOT NULL,
|
||||||
|
size_bytes BIGINT NOT NULL,
|
||||||
|
sha256 VARCHAR(64) NOT NULL,
|
||||||
|
status VARCHAR(32) NOT NULL DEFAULT 'queued',
|
||||||
|
variants JSONB NOT NULL DEFAULT '[]',
|
||||||
|
error_code VARCHAR(64),
|
||||||
|
pipeline_version INT NOT NULL DEFAULT 1,
|
||||||
|
attempt INT NOT NULL DEFAULT 0,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL,
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS outbox (
|
||||||
|
id UUID PRIMARY KEY,
|
||||||
|
stream VARCHAR(64) NOT NULL,
|
||||||
|
payload JSONB NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
published_at TIMESTAMPTZ
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_attachments_chat ON attachments(chat_id);
|
||||||
|
|
||||||
|
ALTER TABLE messages ADD COLUMN IF NOT EXISTS attachment_ids UUID[] DEFAULT '{}';
|
||||||
|
ALTER TABLE attachments ADD COLUMN IF NOT EXISTS pipeline_version INT NOT NULL DEFAULT 1;
|
||||||
|
ALTER TABLE attachments ADD COLUMN IF NOT EXISTS attempt INT NOT NULL DEFAULT 0;
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_outbox_unpublished ON outbox(created_at) WHERE published_at IS NULL;
|
||||||
+28
@@ -0,0 +1,28 @@
|
|||||||
|
events { worker_connections 1024; }
|
||||||
|
|
||||||
|
http {
|
||||||
|
upstream messenger_api {
|
||||||
|
server api-1:8080 max_fails=1 fail_timeout=5s;
|
||||||
|
server api-2:8080 max_fails=1 fail_timeout=5s;
|
||||||
|
}
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 8080;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://messenger_api;
|
||||||
|
|
||||||
|
proxy_connect_timeout 2s;
|
||||||
|
proxy_send_timeout 5s;
|
||||||
|
proxy_read_timeout 5s;
|
||||||
|
|
||||||
|
proxy_next_upstream error timeout http_502 http_503 http_504;
|
||||||
|
proxy_next_upstream_tries 2;
|
||||||
|
proxy_next_upstream_timeout 5s;
|
||||||
|
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
fastapi==0.110.0
|
||||||
|
uvicorn==0.28.0
|
||||||
|
pydantic==2.6.2
|
||||||
|
starlette==0.36.3
|
||||||
|
idna==3.6
|
||||||
|
anyio==4.3.0
|
||||||
|
sniffio==1.3.1
|
||||||
|
click==8.1.7
|
||||||
|
h11==0.14.0
|
||||||
|
annotated-types==0.6.0
|
||||||
|
psycopg2-binary==2.9.9
|
||||||
|
bcrypt==4.1.2
|
||||||
|
redis==5.0.1
|
||||||
|
boto3==1.34.0
|
||||||
|
Pillow==10.2.0
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
import os
|
||||||
|
import boto3
|
||||||
|
from botocore.client import Config
|
||||||
|
|
||||||
|
S3_ENDPOINT = os.getenv("S3_ENDPOINT", "http://s3:8333")
|
||||||
|
S3_BUCKET = os.getenv("S3_BUCKET", "messenger-attachments")
|
||||||
|
S3_ROOT_ACCESS_KEY = os.getenv("S3_ROOT_ACCESS_KEY", "minioadmin")
|
||||||
|
S3_ROOT_SECRET_KEY = os.getenv("S3_ROOT_SECRET_KEY", "minioadminpassword")
|
||||||
|
|
||||||
|
s3_client = boto3.client(
|
||||||
|
"s3",
|
||||||
|
endpoint_url=S3_ENDPOINT,
|
||||||
|
aws_access_key_id=S3_ROOT_ACCESS_KEY,
|
||||||
|
aws_secret_access_key=S3_ROOT_SECRET_KEY,
|
||||||
|
config=Config(signature_version="s3v4"),
|
||||||
|
region_name="us-east-1"
|
||||||
|
)
|
||||||
|
|
||||||
|
def init_bucket():
|
||||||
|
print(f"Checking/Creating bucket: {S3_BUCKET}")
|
||||||
|
try:
|
||||||
|
s3_client.head_bucket(Bucket=S3_BUCKET)
|
||||||
|
print("Bucket already exists.")
|
||||||
|
except Exception:
|
||||||
|
s3_client.create_bucket(Bucket=S3_BUCKET)
|
||||||
|
print("Bucket successfully created.")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
init_bucket()
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
import os, sys, urllib.request, urllib.error
|
||||||
|
BUCKET = os.getenv("S3_BUCKET", "messenger-attachments")
|
||||||
|
KEY = os.getenv("S3_KEY")
|
||||||
|
URL = f"http://s3:8333/{BUCKET}/{KEY}"
|
||||||
|
try:
|
||||||
|
urllib.request.urlopen(URL)
|
||||||
|
print("FAIL: object accessible without auth"); sys.exit(1)
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
if e.code in (401, 403):
|
||||||
|
print(f"PASS: anonymous GET -> {e.code}"); sys.exit(0)
|
||||||
|
print(f"FAIL: unexpected {e.code}"); sys.exit(1)
|
||||||
@@ -8,8 +8,8 @@ import re
|
|||||||
ROOT = Path(__file__).resolve().parents[1]
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
lab = json.loads((ROOT/'lab.json').read_text())['number']
|
lab = json.loads((ROOT/'lab.json').read_text(encoding='utf-8'))['number']
|
||||||
spec = json.loads((ROOT/'contracts/openapi.json').read_text())
|
spec = json.loads((ROOT/'contracts/openapi.json').read_text(encoding='utf-8'))
|
||||||
assert spec['openapi']=='3.1.0'
|
assert spec['openapi']=='3.1.0'
|
||||||
assert spec['info']['version']=='1.%d.0'%lab
|
assert spec['info']['version']=='1.%d.0'%lab
|
||||||
operation_ids = set()
|
operation_ids = set()
|
||||||
@@ -41,11 +41,11 @@ def main():
|
|||||||
assert ('/api/v1/auth/sessions' in spec['paths']) == (lab>=2)
|
assert ('/api/v1/auth/sessions' in spec['paths']) == (lab>=2)
|
||||||
assert ('/api/v1/attachments/{attachment_id}' in spec['paths']) == (lab>=4)
|
assert ('/api/v1/attachments/{attachment_id}' in spec['paths']) == (lab>=4)
|
||||||
for file in list((ROOT/'tests').glob('*.py'))+list((ROOT/'scripts').glob('*.py')):
|
for file in list((ROOT/'tests').glob('*.py'))+list((ROOT/'scripts').glob('*.py')):
|
||||||
ast.parse(file.read_text(),filename=str(file))
|
ast.parse(file.read_text(encoding='utf-8'),filename=str(file))
|
||||||
for file in ROOT.rglob('*.json'):
|
for file in ROOT.rglob('*.json'):
|
||||||
if any(part in {'.git','.venv','node_modules','vendor','output'} for part in file.parts):
|
if any(part in {'.git','.venv','node_modules','vendor','output'} for part in file.parts):
|
||||||
continue
|
continue
|
||||||
json.loads(file.read_text())
|
json.loads(file.read_text(encoding='utf-8-sig'))
|
||||||
for required in ('README.md','COURSE.md','CONTRIBUTING.md','REPORT.md','contracts/README.md','tests/smoke.py','tests/README.md'):
|
for required in ('README.md','COURSE.md','CONTRIBUTING.md','REPORT.md','contracts/README.md','tests/smoke.py','tests/README.md'):
|
||||||
assert (ROOT/required).is_file(), 'missing '+required
|
assert (ROOT/required).is_file(), 'missing '+required
|
||||||
print('PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.')
|
print('PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.')
|
||||||
|
|||||||
@@ -0,0 +1,81 @@
|
|||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import boto3
|
||||||
|
from botocore.client import Config
|
||||||
|
import psycopg2
|
||||||
|
import psycopg2.extras
|
||||||
|
from datetime import datetime, timezone, timedelta
|
||||||
|
|
||||||
|
DATABASE_URL = os.getenv("DATABASE_URL", "postgresql://postgres:postgres@postgres:5432/messenger")
|
||||||
|
S3_ENDPOINT = os.getenv("S3_ENDPOINT", "http://s3:8333")
|
||||||
|
S3_BUCKET = os.getenv("S3_BUCKET", "messenger-attachments")
|
||||||
|
S3_ACCESS_KEY = os.getenv("S3_ACCESS_KEY_ID", "appuser")
|
||||||
|
S3_SECRET_KEY = os.getenv("S3_SECRET_ACCESS_KEY", "appsecretpassword")
|
||||||
|
|
||||||
|
s3_client = boto3.client(
|
||||||
|
"s3",
|
||||||
|
endpoint_url=S3_ENDPOINT,
|
||||||
|
aws_access_key_id=S3_ACCESS_KEY,
|
||||||
|
aws_secret_access_key=S3_SECRET_KEY,
|
||||||
|
config=Config(signature_version="s3v4"),
|
||||||
|
region_name="us-east-1"
|
||||||
|
)
|
||||||
|
|
||||||
|
def run_reconciliation(dry_run: bool = False):
|
||||||
|
print(f"Starting reconciliation (dry_run={dry_run})...")
|
||||||
|
conn = psycopg2.connect(DATABASE_URL, cursor_factory=psycopg2.extras.RealDictCursor)
|
||||||
|
|
||||||
|
threshold = datetime.now(timezone.utc) - timedelta(hours=1)
|
||||||
|
|
||||||
|
with conn.cursor() as cur:
|
||||||
|
cur.execute("""
|
||||||
|
SELECT id, chat_id, filename, created_at FROM attachments
|
||||||
|
WHERE status = 'pending' AND created_at < %s
|
||||||
|
""", (threshold,))
|
||||||
|
stale_pending = cur.fetchall()
|
||||||
|
|
||||||
|
for row in stale_pending:
|
||||||
|
att_id = str(row['id'])
|
||||||
|
chat_id = str(row['chat_id'])
|
||||||
|
filename = row['filename']
|
||||||
|
s3_key = f"attachments/{chat_id}/{att_id}/{filename}"
|
||||||
|
print(f"[DB Stale Pending] Found stale attachment {att_id}, marking as failed and cleaning S3.")
|
||||||
|
|
||||||
|
if not dry_run:
|
||||||
|
try:
|
||||||
|
s3_client.delete_object(Bucket=S3_BUCKET, Key=s3_key)
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Failed to delete S3 key {s3_key}: {e}")
|
||||||
|
with conn.cursor() as cur:
|
||||||
|
cur.execute("UPDATE attachments SET status = 'failed', error_code = 'timeout_stale' WHERE id = %s", (att_id,))
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
with conn.cursor() as cur:
|
||||||
|
cur.execute("SELECT id, chat_id, filename FROM attachments")
|
||||||
|
all_db_attachments = cur.fetchall()
|
||||||
|
|
||||||
|
valid_s3_keys = set()
|
||||||
|
for row in all_db_attachments:
|
||||||
|
valid_s3_keys.add(f"attachments/{str(row['chat_id'])}/{str(row['id'])}/{row['filename']}")
|
||||||
|
|
||||||
|
paginator = s3_client.get_paginator('list_objects_v2')
|
||||||
|
for page in paginator.paginate(Bucket=S3_BUCKET, Prefix="attachments/"):
|
||||||
|
for obj in page.get('Contents', []):
|
||||||
|
key = obj['Key']
|
||||||
|
last_modified = obj['LastModified']
|
||||||
|
|
||||||
|
if key not in valid_s3_keys and last_modified < threshold:
|
||||||
|
print(f"[S3 Orphan] Found orphan S3 object {key} (Modified: {last_modified})")
|
||||||
|
if not dry_run:
|
||||||
|
try:
|
||||||
|
s3_client.delete_object(Bucket=S3_BUCKET, Key=key)
|
||||||
|
print(f"Deleted orphan object {key} from S3.")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Failed to delete orphan {key}: {e}")
|
||||||
|
|
||||||
|
conn.close()
|
||||||
|
print("Reconciliation completed successfully.")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
dry_run_mode = "--dry-run" in sys.argv
|
||||||
|
run_reconciliation(dry_run=dry_run_mode)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
@echo off
|
||||||
|
docker compose restart
|
||||||
|
exit /b %ERRORLEVEL%
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
docker compose restart
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
wait_ready() {
|
||||||
|
svc=$1
|
||||||
|
timeout=60
|
||||||
|
echo "Waiting for $svc readiness (up to ${timeout}s)..."
|
||||||
|
while [ "$timeout" -gt 0 ]; do
|
||||||
|
if docker compose exec -T "$svc" python -c \
|
||||||
|
"import urllib.request; urllib.request.urlopen('http://localhost:8080/health/ready')" 2>/dev/null; then
|
||||||
|
echo "$svc is ready!"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
timeout=$((timeout - 1))
|
||||||
|
done
|
||||||
|
echo "$svc failed to become ready" >&2
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "Starting Rolling Update"
|
||||||
|
|
||||||
|
echo "1. Rebuilding and replacing api-1..."
|
||||||
|
docker compose up -d --no-deps --build --force-recreate api-1
|
||||||
|
wait_ready api-1
|
||||||
|
echo "Reloading nginx to pick up new api-1 IP..."
|
||||||
|
docker compose exec -T nginx nginx -s reload
|
||||||
|
|
||||||
|
echo "2. Rebuilding and replacing api-2..."
|
||||||
|
docker compose up -d --no-deps --build --force-recreate api-2
|
||||||
|
wait_ready api-2
|
||||||
|
echo "Reloading nginx to pick up new api-2 IP..."
|
||||||
|
docker compose exec -T nginx nginx -s reload
|
||||||
|
|
||||||
|
echo "Rolling Update Completed Successfully"
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
|
||||||
|
|
||||||
|
if [ "$TARGET_INSTANCE_ID" = "api-replica-1" ]; then
|
||||||
|
docker compose stop -t 3 api-1
|
||||||
|
elif [ "$TARGET_INSTANCE_ID" = "api-replica-2" ]; then
|
||||||
|
docker compose stop -t 3 api-2
|
||||||
|
else
|
||||||
|
echo "Unknown target instance: $TARGET_INSTANCE_ID" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
import base64, concurrent.futures, json, pathlib, time, urllib.request, uuid
|
||||||
|
|
||||||
|
BASE = "http://localhost:8080"
|
||||||
|
|
||||||
|
def _req(method, path, body=None, headers=None, token=None):
|
||||||
|
h = {"Accept": "application/json"}
|
||||||
|
if token: h["Authorization"] = f"Bearer {token}"
|
||||||
|
h.update(headers or {})
|
||||||
|
req = urllib.request.Request(BASE + path, data=body, headers=h, method=method)
|
||||||
|
with urllib.request.urlopen(req) as r:
|
||||||
|
return r.status, r.read()
|
||||||
|
|
||||||
|
def register_and_login():
|
||||||
|
u = f"u_{uuid.uuid4().hex[:10]}"
|
||||||
|
p = "T3st-" + uuid.uuid4().hex
|
||||||
|
_req("POST", "/api/v1/users",
|
||||||
|
json.dumps({"username": u, "display_name": "Load", "password": p}).encode(),
|
||||||
|
{"Content-Type": "application/json"})
|
||||||
|
basic = base64.b64encode(f"{u}:{p}".encode()).decode()
|
||||||
|
_, body = _req("POST", "/api/v1/auth/sessions",
|
||||||
|
headers={"Authorization": f"Basic {basic}"})
|
||||||
|
return json.loads(body)["token"]
|
||||||
|
|
||||||
|
def make_chat(token):
|
||||||
|
_, me = _req("GET", "/api/v1/users/me", token=token)
|
||||||
|
uid = json.loads(me)["id"]
|
||||||
|
_, chat = _req("POST", "/api/v1/chats",
|
||||||
|
json.dumps({"title": "load", "member_ids": [uid]}).encode(),
|
||||||
|
{"Content-Type": "application/json", "Idempotency-Key": uuid.uuid4().hex},
|
||||||
|
token=token)
|
||||||
|
return json.loads(chat)["id"]
|
||||||
|
|
||||||
|
def main():
|
||||||
|
token = register_and_login()
|
||||||
|
chat = make_chat(token)
|
||||||
|
png = pathlib.Path("tests/fixtures/sample.png").read_bytes()
|
||||||
|
|
||||||
|
def one(_):
|
||||||
|
try:
|
||||||
|
status, _ = _req("POST",
|
||||||
|
f"/api/v1/chats/{chat}/attachments?filename=f.png",
|
||||||
|
png, {"Content-Type": "image/png"}, token)
|
||||||
|
return status
|
||||||
|
except Exception as e:
|
||||||
|
return repr(e)
|
||||||
|
|
||||||
|
t0 = time.monotonic()
|
||||||
|
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as ex:
|
||||||
|
results = list(ex.map(one, range(10)))
|
||||||
|
print(json.dumps({"results": results,
|
||||||
|
"elapsed_s": round(time.monotonic() - t0, 3)}))
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
+1
-1
@@ -14,7 +14,7 @@ import uuid
|
|||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[1]
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
LAB = json.loads((ROOT / 'lab.json').read_text())['number']
|
LAB = json.loads((ROOT / 'lab.json').read_text())['number']
|
||||||
SPEC = json.loads((ROOT / 'contracts/openapi.json').read_text())
|
SPEC = json.loads((ROOT / 'contracts/openapi.json').read_text(encoding='utf-8'))
|
||||||
|
|
||||||
class Failure(AssertionError):
|
class Failure(AssertionError):
|
||||||
pass
|
pass
|
||||||
|
|||||||
@@ -0,0 +1,109 @@
|
|||||||
|
import concurrent.futures
|
||||||
|
import urllib.request
|
||||||
|
import urllib.error
|
||||||
|
import json
|
||||||
|
import unittest
|
||||||
|
from client import Client
|
||||||
|
|
||||||
|
class TestMessengerLab4(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.client = Client()
|
||||||
|
self.alice = self.client.user("alice")
|
||||||
|
self.bob = self.client.user("bob")
|
||||||
|
self.eve = self.client.user("eve")
|
||||||
|
|
||||||
|
res = self.client.call(
|
||||||
|
'POST', '/api/v1/chats', 201,
|
||||||
|
actor=self.alice,
|
||||||
|
body={"title": "Test Chat", "member_ids": [self.alice["id"], self.bob["id"]]},
|
||||||
|
schema='Chat'
|
||||||
|
)
|
||||||
|
self.chat = res['json']
|
||||||
|
|
||||||
|
def test_unauthorized_access_to_chat(self):
|
||||||
|
self.client.call('GET', f"/api/v1/chats/{self.chat['id']}", 404, actor=self.eve)
|
||||||
|
|
||||||
|
def test_empty_message(self):
|
||||||
|
self.client.call('POST', f"/api/v1/chats/{self.chat['id']}/messages", 400, actor=self.alice, body={"text": ""})
|
||||||
|
|
||||||
|
def test_malformed_json(self):
|
||||||
|
url = self.client.base + f"/api/v1/chats/{self.chat['id']}/messages"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url,
|
||||||
|
data=b"{broken json content without quotes",
|
||||||
|
headers={
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"X-User-Id": self.alice["id"]
|
||||||
|
},
|
||||||
|
method="POST"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req) as resp:
|
||||||
|
self.fail("Expected HTTP 400 for malformed JSON")
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
self.assertEqual(e.code, 400)
|
||||||
|
body = json.loads(e.read().decode())
|
||||||
|
self.assertEqual(body["error"]["code"], "invalid_request")
|
||||||
|
|
||||||
|
def test_duplicate_username(self):
|
||||||
|
self.client.call('POST', '/api/v1/users', 409, body=self.alice["registration"])
|
||||||
|
|
||||||
|
def test_pagination_multiple_pages(self):
|
||||||
|
import time
|
||||||
|
msg_texts = ["msg-page-1", "msg-page-2", "msg-page-3"]
|
||||||
|
for txt in msg_texts:
|
||||||
|
self.client.message(self.alice, self.chat, txt)
|
||||||
|
time.sleep(0.01)
|
||||||
|
|
||||||
|
path = f"/api/v1/chats/{self.chat['id']}/messages"
|
||||||
|
collected_items = []
|
||||||
|
cursor = None
|
||||||
|
|
||||||
|
for _ in range(5):
|
||||||
|
url_suffix = "?limit=1"
|
||||||
|
if cursor:
|
||||||
|
url_suffix += f"&cursor={cursor}"
|
||||||
|
|
||||||
|
page_res = self.client.call('GET', path + url_suffix, actor=self.alice, schema='MessagePage')
|
||||||
|
page = page_res['json']
|
||||||
|
|
||||||
|
for item in page['items']:
|
||||||
|
self.assertNotIn(item['id'], [x['id'] for x in collected_items])
|
||||||
|
collected_items.append(item)
|
||||||
|
|
||||||
|
cursor = page.get('next_keyword') or page.get('next_cursor')
|
||||||
|
if not cursor:
|
||||||
|
break
|
||||||
|
|
||||||
|
self.assertEqual(len(collected_items), 3)
|
||||||
|
self.assertEqual(sorted([x['text'] for x in collected_items]), sorted(msg_texts))
|
||||||
|
|
||||||
|
def test_concurrent_messages_strict(self):
|
||||||
|
def worker_send(worker_idx):
|
||||||
|
local_client = Client()
|
||||||
|
ids = []
|
||||||
|
for i in range(10):
|
||||||
|
res = local_client.message(self.alice, self.chat, f"w-{worker_idx}-msg-{i}")
|
||||||
|
ids.append(res['json']['id'])
|
||||||
|
return ids
|
||||||
|
|
||||||
|
all_ids = []
|
||||||
|
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
|
||||||
|
futures = [executor.submit(worker_send, w) for w in range(10)]
|
||||||
|
for f in concurrent.futures.as_completed(futures):
|
||||||
|
all_ids.extend(f.result())
|
||||||
|
|
||||||
|
self.assertEqual(len(all_ids), 100)
|
||||||
|
self.assertEqual(len(set(all_ids)), 100)
|
||||||
|
|
||||||
|
page = self.client.call(
|
||||||
|
'GET', f"/api/v1/chats/{self.chat['id']}/messages?limit=100",
|
||||||
|
actor=self.alice,
|
||||||
|
schema='MessagePage'
|
||||||
|
)
|
||||||
|
server_ids = [m['id'] for m in page['json']['items']]
|
||||||
|
self.assertEqual(len(server_ids), 100)
|
||||||
|
self.assertEqual(len(set(server_ids)), 100)
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,139 @@
|
|||||||
|
import urllib.request
|
||||||
|
import urllib.error
|
||||||
|
import json
|
||||||
|
import unittest
|
||||||
|
from client import Client
|
||||||
|
|
||||||
|
class TestMessengerLevel4Complete(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.client = Client()
|
||||||
|
self.alice = self.client.user("alice")
|
||||||
|
self.bob = self.client.user("bob")
|
||||||
|
self.eve = self.client.user("eve")
|
||||||
|
|
||||||
|
res_chat1 = self.client.call(
|
||||||
|
'POST', '/api/v1/chats', 201,
|
||||||
|
actor=self.alice,
|
||||||
|
body={"title": "Team Chat", "member_ids": [self.alice["id"], self.bob["id"]]},
|
||||||
|
schema='Chat'
|
||||||
|
)
|
||||||
|
self.chat1 = res_chat1['json']
|
||||||
|
|
||||||
|
res_chat2 = self.client.call(
|
||||||
|
'POST', '/api/v1/chats', 201,
|
||||||
|
actor=self.eve,
|
||||||
|
body={"title": "Private Chat", "member_ids": [self.eve["id"], self.bob["id"]]},
|
||||||
|
schema='Chat'
|
||||||
|
)
|
||||||
|
self.chat2 = res_chat2['json']
|
||||||
|
|
||||||
|
def test_unsupported_media_type(self):
|
||||||
|
"""Проверка 415: передача текстового файла под видом картинки."""
|
||||||
|
fake_bytes = b"Just a simple plain text file content."
|
||||||
|
url = self.client.base + f"/api/v1/chats/{self.chat1['id']}/attachments?filename=notes.txt"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url, data=fake_bytes,
|
||||||
|
headers={"Authorization": f"Bearer {self.alice['token']}", "Content-Type": "image/png"},
|
||||||
|
method="POST"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req) as resp:
|
||||||
|
self.fail("Expected HTTP 415")
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
self.assertEqual(e.code, 415)
|
||||||
|
|
||||||
|
def test_payload_too_large(self):
|
||||||
|
"""Проверка 413: файл размером больше 10 MiB."""
|
||||||
|
large_data = b'\x89PNG\r\n\x1a\n' + b'\x00' * (10 * 1024 * 1024 + 100)
|
||||||
|
url = self.client.base + f"/api/v1/chats/{self.chat1['id']}/attachments?filename=huge.png"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url, data=large_data,
|
||||||
|
headers={"Authorization": f"Bearer {self.alice['token']}", "Content-Type": "image/png"},
|
||||||
|
method="POST"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req) as resp:
|
||||||
|
self.fail("Expected HTTP 413")
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
self.assertEqual(e.code, 413)
|
||||||
|
|
||||||
|
def test_corrupted_image_signature(self):
|
||||||
|
"""Проверка 415: битая структура / неверные размеры PNG."""
|
||||||
|
corrupted_png = b'\x89PNG\r\n\x1a\n' + b'\xFF' * 50
|
||||||
|
url = self.client.base + f"/api/v1/chats/{self.chat1['id']}/attachments?filename=corrupt.png"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url, data=corrupted_png,
|
||||||
|
headers={"Authorization": f"Bearer {self.alice['token']}", "Content-Type": "image/png"},
|
||||||
|
method="POST"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req) as resp:
|
||||||
|
self.fail("Expected HTTP 415")
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
self.assertEqual(e.code, 415)
|
||||||
|
|
||||||
|
def test_invalid_filename_characters(self):
|
||||||
|
"""Проверка 400: недопустимые символы (path traversal) в имени файла."""
|
||||||
|
valid_png = (
|
||||||
|
b'\x89PNG\r\n\x1a\n'
|
||||||
|
b'\x00\x00\x00\rIHDR'
|
||||||
|
b'\x00\x00\x00\n\x00\x00\x00\n'
|
||||||
|
b'\x08\x02\x00\x00\x00'
|
||||||
|
b'\x02p\xdd\xb5'
|
||||||
|
b'\x00\x00\x00\x0fcIDATx\x9cc`\x00\x00\x00\x02\x00\x01H\xaf\xa4q'
|
||||||
|
b'\x00\x00\x00\x00IEND\xaeB`\x82'
|
||||||
|
)
|
||||||
|
url = self.client.base + f"/api/v1/chats/{self.chat1['id']}/attachments?filename=../../etc/passwd"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url, data=valid_png,
|
||||||
|
headers={"Authorization": f"Bearer {self.alice['token']}", "Content-Type": "image/png"},
|
||||||
|
method="POST"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req) as resp:
|
||||||
|
self.fail("Expected HTTP 400")
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
self.assertEqual(e.code, 400)
|
||||||
|
|
||||||
|
def test_nonexistent_attachment_id(self):
|
||||||
|
"""Проверка 404: передача несуществующего ID вложения при отправке сообщения."""
|
||||||
|
fake_uuid = "12345678-1234-4123-8123-123456789abc"
|
||||||
|
res = self.client.call(
|
||||||
|
'POST', f"/api/v1/chats/{self.chat1['id']}/messages", 404,
|
||||||
|
actor=self.alice,
|
||||||
|
body={"text": "Check this out", "attachment_ids": [fake_uuid]},
|
||||||
|
headers={"Idempotency-Key": "test-key-lvl4-1"}
|
||||||
|
)
|
||||||
|
self.assertEqual(res['status'], 404)
|
||||||
|
|
||||||
|
def test_cross_chat_attachment_isolation(self):
|
||||||
|
"""Проверка 404: попытка прикрепить вложение из чужого чата."""
|
||||||
|
valid_png = (
|
||||||
|
b'\x89PNG\r\n\x1a\n'
|
||||||
|
b'\x00\x00\x00\rIHDR'
|
||||||
|
b'\x00\x00\x00\n\x00\x00\x00\n'
|
||||||
|
b'\x08\x02\x00\x00\x00'
|
||||||
|
b'\x02p\xdd\xb5'
|
||||||
|
b'\x00\x00\x00\x0fcID1Tx\x9cc`\x00\x00\x00\x02\x00\x01H\xaf\xa4q'
|
||||||
|
b'\x00\x00\x00\x00IEND\xaeB`\x82'
|
||||||
|
)
|
||||||
|
url = self.client.base + f"/api/v1/chats/{self.chat2['id']}/attachments?filename=secret.png"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url, data=valid_png,
|
||||||
|
headers={"Authorization": f"Bearer {self.eve['token']}", "Content-Type": "image/png"},
|
||||||
|
method="POST"
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req) as resp:
|
||||||
|
data = json.loads(resp.read().decode())
|
||||||
|
eve_att_id = data["id"]
|
||||||
|
|
||||||
|
res = self.client.call(
|
||||||
|
'POST', f"/api/v1/chats/{self.chat1['id']}/messages", 404,
|
||||||
|
actor=self.alice,
|
||||||
|
body={"text": "Stealing file", "attachment_ids": [eve_att_id]},
|
||||||
|
headers={"Idempotency-Key": "test-key-lvl4-2"}
|
||||||
|
)
|
||||||
|
self.assertEqual(res['status'], 404)
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
+1
-1
@@ -55,7 +55,7 @@ def main():
|
|||||||
check(len(observed)>=2,'less than two API instances observed before failure; verify LB/session affinity')
|
check(len(observed)>=2,'less than two API instances observed before failure; verify LB/session affinity')
|
||||||
print('Executing explicit '+args.mode+' hook; hook output suppressed to avoid leaking secrets.')
|
print('Executing explicit '+args.mode+' hook; hook output suppressed to avoid leaking secrets.')
|
||||||
# No shell interpolation and no Docker commands in this test.
|
# No shell interpolation and no Docker commands in this test.
|
||||||
proc = subprocess.Popen([str(script)],env=env,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL)
|
proc = subprocess.Popen(['sh', str(script)], env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||||
started = time.monotonic()
|
started = time.monotonic()
|
||||||
successes,failures,after_instances = [],0,[]
|
successes,failures,after_instances = [],0,[]
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -0,0 +1,405 @@
|
|||||||
|
import os
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import time
|
||||||
|
import signal
|
||||||
|
import random
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from concurrent.futures import ThreadPoolExecutor
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
import psycopg2.extras
|
||||||
|
import redis
|
||||||
|
import boto3
|
||||||
|
import botocore.exceptions
|
||||||
|
from botocore.client import Config
|
||||||
|
from PIL import Image, ImageOps, UnidentifiedImageError
|
||||||
|
|
||||||
|
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
|
||||||
|
logger = logging.getLogger("attachment-worker")
|
||||||
|
|
||||||
|
DATABASE_URL = os.getenv("DATABASE_URL", "postgresql://postgres:postgres@postgres:5432/messenger")
|
||||||
|
REDIS_URL = os.getenv("REDIS_URL", "redis://redis:6379/0")
|
||||||
|
S3_ENDPOINT = os.getenv("S3_ENDPOINT", "http://s3:8333")
|
||||||
|
S3_BUCKET = os.getenv("S3_BUCKET", "messenger-attachments")
|
||||||
|
S3_ACCESS_KEY = os.getenv("S3_ACCESS_KEY_ID", "appuser")
|
||||||
|
S3_SECRET_KEY = os.getenv("S3_SECRET_ACCESS_KEY", "appsecretpassword")
|
||||||
|
|
||||||
|
STREAM_NAME = os.getenv("BROKER_STREAM", "attachment_jobs")
|
||||||
|
DLQ_STREAM_NAME = f"{STREAM_NAME}_dlq"
|
||||||
|
GROUP_NAME = os.getenv("BROKER_GROUP", "attachment_workers")
|
||||||
|
CONSUMER_NAME = os.getenv("HOSTNAME", "worker-1")
|
||||||
|
JOB_MAX_ATTEMPTS = int(os.getenv("JOB_MAX_ATTEMPTS", "5"))
|
||||||
|
MAX_IMAGE_PIXELS = int(os.getenv("MAX_IMAGE_PIXELS", "40000000"))
|
||||||
|
JOB_TIMEOUT_SECONDS = int(os.getenv("JOB_TIMEOUT_SECONDS", "30"))
|
||||||
|
WORKER_CONCURRENCY = int(os.getenv("WORKER_CONCURRENCY", "4"))
|
||||||
|
|
||||||
|
CLAIM_MIN_IDLE_MS = (JOB_TIMEOUT_SECONDS + 20) * 1000
|
||||||
|
|
||||||
|
Image.MAX_IMAGE_PIXELS = MAX_IMAGE_PIXELS
|
||||||
|
|
||||||
|
redis_client = redis.Redis.from_url(REDIS_URL, decode_responses=True)
|
||||||
|
s3_client = boto3.client(
|
||||||
|
"s3", endpoint_url=S3_ENDPOINT,
|
||||||
|
aws_access_key_id=S3_ACCESS_KEY, aws_secret_access_key=S3_SECRET_KEY,
|
||||||
|
config=Config(signature_version="s3v4"), region_name="us-east-1"
|
||||||
|
)
|
||||||
|
|
||||||
|
running = True
|
||||||
|
|
||||||
|
def handle_sigterm(signum, frame):
|
||||||
|
global running
|
||||||
|
logger.info("Received shutdown signal, finishing in-flight jobs...")
|
||||||
|
running = False
|
||||||
|
|
||||||
|
signal.signal(signal.SIGTERM, handle_sigterm)
|
||||||
|
signal.signal(signal.SIGINT, handle_sigterm)
|
||||||
|
|
||||||
|
|
||||||
|
def get_db_connection():
|
||||||
|
return psycopg2.connect(DATABASE_URL, cursor_factory=psycopg2.extras.RealDictCursor)
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_consumer_groups():
|
||||||
|
while running:
|
||||||
|
try:
|
||||||
|
redis_client.xgroup_create(STREAM_NAME, GROUP_NAME, id="0", mkstream=True)
|
||||||
|
redis_client.xgroup_create(DLQ_STREAM_NAME, GROUP_NAME, id="0", mkstream=True)
|
||||||
|
logger.info("Consumer groups ensured on '%s' and '%s'", STREAM_NAME, DLQ_STREAM_NAME)
|
||||||
|
return
|
||||||
|
except redis.exceptions.ResponseError as e:
|
||||||
|
if "BUSYGROUP" in str(e):
|
||||||
|
return
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Waiting for Redis: %s", type(e).__name__)
|
||||||
|
time.sleep(1)
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_base_image(raw_bytes):
|
||||||
|
img = Image.open(io.BytesIO(raw_bytes))
|
||||||
|
img = ImageOps.exif_transpose(img)
|
||||||
|
if img.mode in ("RGBA", "LA") or (img.mode == "P" and "transparency" in img.info):
|
||||||
|
rgba = img.convert("RGBA")
|
||||||
|
bg = Image.new("RGB", rgba.size, (255, 255, 255))
|
||||||
|
bg.paste(rgba, mask=rgba.split()[3])
|
||||||
|
img = bg
|
||||||
|
elif img.mode != "RGB":
|
||||||
|
img = img.convert("RGB")
|
||||||
|
return img
|
||||||
|
|
||||||
|
|
||||||
|
def generate_grade4_variants(raw_bytes):
|
||||||
|
base_img = prepare_base_image(raw_bytes)
|
||||||
|
orig_w, orig_h = base_img.size
|
||||||
|
meta = []
|
||||||
|
blobs = {}
|
||||||
|
|
||||||
|
thumb = base_img.copy()
|
||||||
|
if orig_w <= 256 and orig_h <= 256:
|
||||||
|
t_w, t_h = orig_w, orig_h
|
||||||
|
else:
|
||||||
|
s = min(256.0 / orig_w, 256.0 / orig_h)
|
||||||
|
t_w, t_h = max(1, int(orig_w * s)), max(1, int(orig_h * s))
|
||||||
|
if (t_w, t_h) != (orig_w, orig_h):
|
||||||
|
thumb = thumb.resize((t_w, t_h), Image.Resampling.LANCZOS)
|
||||||
|
buf = io.BytesIO(); thumb.save(buf, format="JPEG", quality=85, optimize=True)
|
||||||
|
tb = buf.getvalue(); blobs["thumbnail"] = tb
|
||||||
|
meta.append({"name": "thumbnail", "content_type": "image/jpeg",
|
||||||
|
"size_bytes": len(tb), "width": t_w, "height": t_h,
|
||||||
|
"sha256": hashlib.sha256(tb).hexdigest()})
|
||||||
|
|
||||||
|
opt = base_img.copy()
|
||||||
|
if orig_w <= 1280 and orig_h <= 1280:
|
||||||
|
o_w, o_h = orig_w, orig_h
|
||||||
|
else:
|
||||||
|
s = min(1280.0 / orig_w, 1280.0 / orig_h)
|
||||||
|
o_w, o_h = max(1, int(orig_w * s)), max(1, int(orig_h * s))
|
||||||
|
if (o_w, o_h) != (orig_w, orig_h):
|
||||||
|
opt = opt.resize((o_w, o_h), Image.Resampling.LANCZOS)
|
||||||
|
buf = io.BytesIO(); opt.save(buf, format="WEBP", quality=85)
|
||||||
|
ob = buf.getvalue(); blobs["optimized"] = ob
|
||||||
|
meta.append({"name": "optimized", "content_type": "image/webp",
|
||||||
|
"size_bytes": len(ob), "width": o_w, "height": o_h,
|
||||||
|
"sha256": hashlib.sha256(ob).hexdigest()})
|
||||||
|
|
||||||
|
c = min(orig_w, orig_h)
|
||||||
|
left, top = (orig_w - c) // 2, (orig_h - c) // 2
|
||||||
|
crop = base_img.crop((left, top, left + c, top + c))
|
||||||
|
if c != 128:
|
||||||
|
crop = crop.resize((128, 128), Image.Resampling.LANCZOS)
|
||||||
|
buf = io.BytesIO(); crop.save(buf, format="JPEG", quality=85, optimize=True)
|
||||||
|
cb = buf.getvalue(); blobs["crop"] = cb
|
||||||
|
meta.append({"name": "crop", "content_type": "image/jpeg",
|
||||||
|
"size_bytes": len(cb), "width": 128, "height": 128,
|
||||||
|
"sha256": hashlib.sha256(cb).hexdigest()})
|
||||||
|
|
||||||
|
return meta, blobs
|
||||||
|
|
||||||
|
|
||||||
|
def mark_failed(attachment_id, error_code):
|
||||||
|
now = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
conn = get_db_connection()
|
||||||
|
try:
|
||||||
|
with conn.cursor() as cur:
|
||||||
|
cur.execute("UPDATE attachments SET status='failed', error_code=%s, updated_at=%s "
|
||||||
|
"WHERE id=%s AND status != 'ready'",
|
||||||
|
(error_code, now, attachment_id))
|
||||||
|
conn.commit()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
PERMANENT_ERRORS = (
|
||||||
|
UnidentifiedImageError, Image.DecompressionBombError, ValueError,
|
||||||
|
psycopg2.errors.DataError,
|
||||||
|
)
|
||||||
|
TRANSIENT_ERRORS = (
|
||||||
|
TimeoutError,
|
||||||
|
botocore.exceptions.EndpointConnectionError,
|
||||||
|
botocore.exceptions.ReadTimeoutError,
|
||||||
|
psycopg2.OperationalError,
|
||||||
|
redis.exceptions.ConnectionError,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def process_job(message_id, fields):
|
||||||
|
try:
|
||||||
|
job = json.loads(fields.get("payload", "{}"))
|
||||||
|
except Exception:
|
||||||
|
logger.error("Malformed job %s, ack to drop", message_id)
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
attachment_id = job.get("attachment_id")
|
||||||
|
job_id = job.get("job_id")
|
||||||
|
pipeline_version = int(job.get("pipeline_version", 1))
|
||||||
|
attempt = int(job.get("attempt", 1))
|
||||||
|
|
||||||
|
if not attachment_id:
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
conn = get_db_connection()
|
||||||
|
try:
|
||||||
|
with conn.cursor() as cur:
|
||||||
|
cur.execute("SELECT * FROM attachments WHERE id=%s FOR UPDATE", (attachment_id,))
|
||||||
|
att = cur.fetchone()
|
||||||
|
if not att:
|
||||||
|
conn.commit()
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
return
|
||||||
|
if att["status"] == "ready" or att["pipeline_version"] > pipeline_version:
|
||||||
|
conn.commit()
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
return
|
||||||
|
if att["status"] == "failed":
|
||||||
|
conn.commit()
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
if att["status"] == "processing":
|
||||||
|
updated = att["updated_at"]
|
||||||
|
if isinstance(updated, str):
|
||||||
|
updated = datetime.fromisoformat(updated.replace("Z", "+00:00"))
|
||||||
|
age_seconds = (datetime.now(timezone.utc) - updated).total_seconds()
|
||||||
|
if age_seconds < (JOB_TIMEOUT_SECONDS + 10):
|
||||||
|
conn.commit()
|
||||||
|
logger.info("Attachment %s already processing (age=%.1fs), ack duplicate",
|
||||||
|
attachment_id, age_seconds)
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
now = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
cur.execute(
|
||||||
|
"UPDATE attachments SET status='processing', attempt=%s, updated_at=%s WHERE id=%s",
|
||||||
|
(attempt, now, attachment_id)
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
chat_id = str(att["chat_id"])
|
||||||
|
staging_key = f"staging/{chat_id}/{attachment_id}/original"
|
||||||
|
final_original_key = f"attachments/{chat_id}/{attachment_id}/original"
|
||||||
|
|
||||||
|
start_time = time.monotonic()
|
||||||
|
|
||||||
|
def check_timeout():
|
||||||
|
if time.monotonic() - start_time > JOB_TIMEOUT_SECONDS:
|
||||||
|
raise TimeoutError("Job exceeded time limit")
|
||||||
|
|
||||||
|
try:
|
||||||
|
try:
|
||||||
|
obj = s3_client.get_object(Bucket=S3_BUCKET, Key=staging_key)
|
||||||
|
except Exception:
|
||||||
|
obj = s3_client.get_object(Bucket=S3_BUCKET, Key=final_original_key)
|
||||||
|
raw_bytes = obj["Body"].read()
|
||||||
|
check_timeout()
|
||||||
|
|
||||||
|
if hashlib.sha256(raw_bytes).hexdigest() != att["sha256"] or len(raw_bytes) != att["size_bytes"]:
|
||||||
|
raise ValueError("Staging checksum mismatch")
|
||||||
|
|
||||||
|
variants_meta, variants_bytes = generate_grade4_variants(raw_bytes)
|
||||||
|
check_timeout()
|
||||||
|
|
||||||
|
s3_client.put_object(
|
||||||
|
Bucket=S3_BUCKET, Key=final_original_key,
|
||||||
|
Body=raw_bytes, ContentType=att["content_type"]
|
||||||
|
)
|
||||||
|
for var_name, var_bytes in variants_bytes.items():
|
||||||
|
ct = "image/webp" if var_name == "optimized" else "image/jpeg"
|
||||||
|
s3_client.put_object(
|
||||||
|
Bucket=S3_BUCKET,
|
||||||
|
Key=f"attachments/{chat_id}/{attachment_id}/{var_name}",
|
||||||
|
Body=var_bytes, ContentType=ct
|
||||||
|
)
|
||||||
|
check_timeout()
|
||||||
|
|
||||||
|
ready_at = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||||
|
conn = get_db_connection()
|
||||||
|
try:
|
||||||
|
with conn.cursor() as cur:
|
||||||
|
cur.execute("""
|
||||||
|
UPDATE attachments SET status='ready', variants=%s::jsonb,
|
||||||
|
error_code=NULL, updated_at=%s
|
||||||
|
WHERE id=%s AND pipeline_version <= %s
|
||||||
|
""", (json.dumps(variants_meta), ready_at, attachment_id, pipeline_version))
|
||||||
|
conn.commit()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
logger.info("Job %s completed: attachment %s ready", job_id, attachment_id)
|
||||||
|
|
||||||
|
try:
|
||||||
|
s3_client.delete_object(Bucket=S3_BUCKET, Key=staging_key)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
except (UnidentifiedImageError, Image.DecompressionBombError) as e:
|
||||||
|
logger.error("Permanent image error for %s: %s", attachment_id, type(e).__name__)
|
||||||
|
mark_failed(attachment_id, "invalid_image")
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
|
||||||
|
except ValueError as e:
|
||||||
|
logger.error("Checksum error for %s: %s", attachment_id, e)
|
||||||
|
mark_failed(attachment_id, "invalid_checksum")
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
|
||||||
|
except psycopg2.errors.DataError as e:
|
||||||
|
logger.error("DB data error for %s: %s", attachment_id, type(e).__name__)
|
||||||
|
mark_failed(attachment_id, "data_error")
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
|
||||||
|
except TRANSIENT_ERRORS as e:
|
||||||
|
logger.warning(
|
||||||
|
"Transient error for %s (attempt %d/%d): %s",
|
||||||
|
attachment_id, attempt, JOB_MAX_ATTEMPTS, type(e).__name__
|
||||||
|
)
|
||||||
|
if attempt >= JOB_MAX_ATTEMPTS:
|
||||||
|
mark_failed(attachment_id, "retry_exhausted")
|
||||||
|
try:
|
||||||
|
dlq_job = dict(job)
|
||||||
|
dlq_job["attempt"] = 1
|
||||||
|
redis_client.xadd(
|
||||||
|
DLQ_STREAM_NAME,
|
||||||
|
{"payload": json.dumps(dlq_job), "error": str(e)}
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
else:
|
||||||
|
delay = min(2 ** attempt, 30) + random.uniform(0, 1)
|
||||||
|
time.sleep(delay)
|
||||||
|
next_job = dict(job)
|
||||||
|
next_job["attempt"] = attempt + 1
|
||||||
|
redis_client.xadd(STREAM_NAME, {"payload": json.dumps(next_job)})
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("Unknown error for %s: %s, marking failed",
|
||||||
|
attachment_id, type(e).__name__)
|
||||||
|
mark_failed(attachment_id, "processing_failed")
|
||||||
|
redis_client.xack(STREAM_NAME, GROUP_NAME, message_id)
|
||||||
|
|
||||||
|
def run_staging_cleanup():
|
||||||
|
try:
|
||||||
|
cutoff = datetime.now(timezone.utc).timestamp() - 3600
|
||||||
|
paginator = s3_client.get_paginator("list_objects_v2")
|
||||||
|
for page in paginator.paginate(Bucket=S3_BUCKET, Prefix="staging/"):
|
||||||
|
for obj in page.get("Contents", []):
|
||||||
|
if obj["LastModified"].timestamp() < cutoff:
|
||||||
|
try:
|
||||||
|
s3_client.delete_object(Bucket=S3_BUCKET, Key=obj["Key"])
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug("Staging cleanup: %s", e)
|
||||||
|
|
||||||
|
|
||||||
|
def safe_process(msg_id, fields):
|
||||||
|
"""Wrapper - exceptions в threads иначе исчезают молча."""
|
||||||
|
try:
|
||||||
|
process_job(msg_id, fields)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("Unhandled in process_job %s: %s", msg_id, type(e).__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ensure_consumer_groups()
|
||||||
|
logger.info("Worker '%s' started, concurrency=%d", CONSUMER_NAME, WORKER_CONCURRENCY)
|
||||||
|
|
||||||
|
executor = ThreadPoolExecutor(max_workers=WORKER_CONCURRENCY)
|
||||||
|
last_claim = 0
|
||||||
|
last_cleanup = 0
|
||||||
|
|
||||||
|
try:
|
||||||
|
while running:
|
||||||
|
try:
|
||||||
|
now = time.time()
|
||||||
|
|
||||||
|
if now - last_claim > 15:
|
||||||
|
last_claim = now
|
||||||
|
try:
|
||||||
|
claimed = redis_client.xautoclaim(
|
||||||
|
STREAM_NAME, GROUP_NAME, CONSUMER_NAME,
|
||||||
|
min_idle_time=CLAIM_MIN_IDLE_MS,
|
||||||
|
start_id="0-0", count=WORKER_CONCURRENCY
|
||||||
|
)
|
||||||
|
if claimed and len(claimed) > 1 and claimed[1]:
|
||||||
|
for msg_id, fields in claimed[1]:
|
||||||
|
if not running:
|
||||||
|
break
|
||||||
|
executor.submit(safe_process, msg_id, fields)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
if now - last_cleanup > 600:
|
||||||
|
last_cleanup = now
|
||||||
|
executor.submit(run_staging_cleanup)
|
||||||
|
|
||||||
|
resp = redis_client.xreadgroup(
|
||||||
|
GROUP_NAME, CONSUMER_NAME, {STREAM_NAME: ">"},
|
||||||
|
count=WORKER_CONCURRENCY, block=2000
|
||||||
|
)
|
||||||
|
if not resp:
|
||||||
|
continue
|
||||||
|
for _, messages in resp:
|
||||||
|
for msg_id, fields in messages:
|
||||||
|
if not running:
|
||||||
|
break
|
||||||
|
executor.submit(safe_process, msg_id, fields)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Worker loop: %s", type(e).__name__)
|
||||||
|
time.sleep(1)
|
||||||
|
finally:
|
||||||
|
logger.info("Shutting down, waiting for in-flight jobs...")
|
||||||
|
executor.shutdown(wait=True)
|
||||||
|
logger.info("Worker stopped cleanly")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in New Issue
Block a user