Author SHA1 Message Date
ivanstorozhevandCursor 8057852823 Оригинал принимается в S3 и в одну транзакцию с outbox, поэтому ответ 202 уже не зависит от брокера. Worker коммитит смещение Kafka только после записи вариантов в фиксированные ключи, так что повторная доставка не создаёт второй набор.
Автор / группа: ivanstorozhev, БВТ2304
Целевая оценка (3/4/5): 4
Трек 5, если заявлен: не заявлен

Что реализовано и что перенесено: из lab4 перенесены Spring Boot API, PostgreSQL, Redis-кэш сессий, Argon2id, Liquibase через одноразовый сервис migrate, HAProxy без sticky session, две реплики, X-Instance-Id, обязательный Idempotency-Key, scripts/stop-one.sh и scripts/restart.sh, приватный SeaweedFS, таблица attachments и скачивание оригинала через API. Ключ объекта назначает сервер. У API отдельный ключ S3 с правами Read/Write/List; Admin используется только чтобы создать bucket. Объект без строки в базе старше двух минут удаляет фоновая очистка. Добавлены ответ 202, staging, transactional outbox, отдельный worker, Kafka 3.9.1 (KRaft, volume kafka-data), состояния queued/processing/ready/failed, JPEG thumbnail и crop, lossy WebP, retry с backoff и jitter, топик attachments.failed, явный replay и lease. Сообщение брокера содержит только идентификаторы. Подтверждение записи брокером (acks=all) и commit смещения потребителя разделены.

Команды запуска из чистого clone:
cp .env.example .env
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build
curl -fsS http://127.0.0.1:8080/health/ready
make test
make test-persistence ACTION_SCRIPT=scripts/restart.sh
make test-failover ACTION_SCRIPT=scripts/stop-one.sh
docker compose --env-file .env start api
python3 scripts/grade4-evidence.py

Результаты проверок и ссылка на REPORT.md: подробности и числа в REPORT.md
make check — PASS: offline skeleton structure, local refs and Python syntax.
make test — PASS: lab5 public API scenario.
make test-persistence ACTION_SCRIPT=scripts/restart.sh — PASS: persistence preserved confirmed data and existing session. Хук перезапускает и kafka.
make test-failover ACTION_SCRIPT=scripts/stop-one.sh — PASS: две реплики до отказа, восстановление записи за 1.795 с, 117 успешных записей и 1 ошибка перехода.
python3 scripts/grade4-evidence.py — PASS. Worker остановлен: после рестарта API вложение осталось queued, затем стало ready; thumbnail JPEG 256×160, crop JPEG 128×128, optimized WebP 320×200, оригинал 111817 байт совпал. Конец топика attachments.process вырос 2→3 и после restart kafka остался 3. Пауза outbox пережила kill API, unpublished=1, затем тот же id стал ready. System.exit после записи объектов: RestartCount 0→1, один набор вариантов. Постоянный сбой processing_failed, топик attachments.failed, replay доводит то же вложение до ready, второй replay остаётся ready.
mvn -q test в backend/ — код выхода 0, 22 теста: 202 и варианты, слишком большой файл 413, порча и подмена MIME 415, чужой id и другой чат 404, сирота в S3 удаляется.
Анонимный GET объекта в сети стенда — HTTP 403.

Ограничения / известные ошибки: трек 5 не делался. Один Docker-хост, единственные PostgreSQL, Redis, SeaweedFS, Kafka и HAProxy остаются точками отказа. Replication factor топиков равен 1. Доставка at least once: повтор перезаписывает те же ключи, но это не exactly-once на стороне S3. Если процесс умер после PUT staging и до COMMIT, объект без строки живёт не дольше grace period, затем удаляется. Staging failed-задания хранится сутки; позже replay уже не находит байты. Пока HAProxy не снял backend, часть запросов на останавливаемую реплику завершается ошибкой соединения; повтор с тем же Idempotency-Key безопасен для отправки сообщения.

Источники и использование ИИ: контракт contracts/openapi.json, contracts/README.md, contracts/attachment-states.md, contracts/jobs.schema.json; код lab4; Spring Boot 4.1.1, Spring Data JPA, spring-security-crypto, Spring Data Redis, Spring Kafka, Liquibase, Actuator, MapStruct 1.6.3, OpenAPI Generator 7.17.0, AWS SDK for Java 2.31.50, Thumbnailator 0.4.20, webp-imageio 0.11.0, Bouncy Castle, PostgreSQL 16.10, Redis 7.4.6, HAProxy 3.2.6, SeaweedFS 4.52, Apache Kafka 3.9.1, Testcontainers. Реализация собрана в Cursor.

Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами.
Контракт и публичные тесты не ослаблены.
make check и make test выполнены; приложен реальный вывод.
Сценарии отказов своего уровня воспроизведены.
Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens.
README задания сохранён, REPORT.md содержит инструкцию именно моего решения.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-12 19:33:05 +03:00
114 changed files with 7390 additions and 23 deletions
+27 -9
View File
@@ -1,25 +1,43 @@
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
# These are logical interface suggestions; map actual framework names in REPORT.md.
APP_HOST=0.0.0.0
APP_PORT=8080
APP_ENV=lab
SESSION_TTL_SECONDS=3600
# Choose credentials locally. File paths contain no secret values.
DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
API_REPLICAS=2
IDEMPOTENCY_TTL_SECONDS=86400
S3_ENDPOINT=http://s3:9000
POSTGRES_USER=messenger
POSTGRES_PASSWORD=messenger
POSTGRES_DB=messenger
SPRING_DATASOURCE_URL=jdbc:postgresql://postgres:5432/messenger
SPRING_DATASOURCE_USERNAME=messenger
SPRING_DATASOURCE_PASSWORD=messenger
SPRING_DATA_REDIS_HOST=redis
SPRING_DATA_REDIS_PORT=6379
S3_ENDPOINT=http://s3:8333
S3_REGION=us-east-1
S3_BUCKET=messenger-attachments
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
S3_ACCESS_KEY=messenger
S3_SECRET_KEY=messenger-lab-secret
S3_BOOTSTRAP_ACCESS_KEY=messenger-bootstrap
S3_BOOTSTRAP_SECRET_KEY=messenger-bootstrap-secret
S3_ORPHAN_GRACE=2m
MAX_UPLOAD_BYTES=10485760
MAX_IMAGE_PIXELS=40000000
BROKER_URL_FILE=/run/secrets/broker_url
# Kafka слушает только сеть стенда, SASL нет: порт наружу не опубликован.
SPRING_KAFKA_BOOTSTRAP_SERVERS=kafka:9092
WORKER_CONCURRENCY=2
JOB_MAX_ATTEMPTS=5
JOB_TIMEOUT_SECONDS=60
JOB_BACKOFF_BASE_MS=1000
JOB_BACKOFF_CAP_MS=30000
STAGING_RETENTION_SECONDS=86400
DEMO_TOKEN=lab-demo
JPEG_QUALITY=0.85
WEBP_QUALITY=0.8
# Course names kept for the report mapping. This stand passes credentials as env vars, not files.
DATABASE_URL_FILE=/run/secrets/database_url
CACHE_URL_FILE=/run/secrets/cache_url
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
BROKER_URL_FILE=/run/secrets/broker_url
+22
View File
@@ -0,0 +1,22 @@
FROM maven:3.9.11-eclipse-temurin-21 AS build
WORKDIR /src
COPY backend/pom.xml backend/pom.xml
COPY contracts contracts
COPY backend/src backend/src
RUN --mount=type=cache,target=/root/.m2 \
mvn -f backend/pom.xml -DskipTests -Dmaven.wagon.http.retryHandler.count=5 package \
|| mvn -f backend/pom.xml -DskipTests -Dmaven.wagon.http.retryHandler.count=5 package
FROM eclipse-temurin:21.0.8_9-jre-jammy
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl \
&& rm -rf /var/lib/apt/lists/* \
&& useradd --uid 10001 --create-home --shell /usr/sbin/nologin app
WORKDIR /app
COPY --from=build /src/backend/target/messenger.jar /app/messenger.jar
USER app
EXPOSE 8080
ENV LIQUIBASE_ANALYTICS_ENABLED=false
HEALTHCHECK --interval=5s --timeout=3s --start-period=60s --retries=3 \
CMD curl -fsS http://127.0.0.1:8080/health/ready || exit 1
ENTRYPOINT ["java", "-jar", "/app/messenger.jar"]
+103 -14
View File
@@ -1,37 +1,126 @@
# Отчёт — лабораторная 5
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: ivanstorozhev, БВТ2304
- Целевая оценка: 4
- Трек 5: не выбран
- Commit/PR: ветка `solution/bvt2304-storozhev`
- Использованные источники, библиотеки и помощь ИИ: контракт `contracts/openapi.json`, `contracts/README.md`, `contracts/attachment-states.md`, `contracts/jobs.schema.json`. Код lab4 перенесён как основа (Spring Boot 4.1.1, Spring Data JPA, Liquibase, Redis-кэш сессий, HAProxy, Argon2id, OpenAPI Generator 7.17.0, AWS SDK for Java 2.31.50, SeaweedFS 4.52). Новое: Spring Kafka, Apache Kafka 3.9.1 (KRaft), Thumbnailator 0.4.20, webp-imageio 0.11.0. Помощь ИИ: перенос lab4, обработка вложений и отчёт сделаны в Cursor.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Нужны Docker Desktop (проверено 28.5.1), Docker Compose и Python 3.9+. JDK 21 и Maven 3.9 нужны только для `mvn test` вне образа. На этой машине JDK — `/opt/homebrew/opt/openjdk@21`.
```sh
cp .env.example .env
# Перед общим стендом замените учебные пароли PostgreSQL и ключи S3 в .env.
docker compose --env-file .env config --quiet
docker compose --env-file .env up -d --build
curl -fsS http://127.0.0.1:8080/health/ready
make test
make test-persistence ACTION_SCRIPT=scripts/restart.sh
make test-failover ACTION_SCRIPT=scripts/stop-one.sh
# Поднять остановленную реплику:
docker compose --env-file .env start api
python3 scripts/grade4-evidence.py
```
Образ приложения собирает сервис `migrate` (`messenger-lab5:1.0.0`). У `api` и `worker` тот же образ и `pull_policy: never`. Миграции — одноразовый `migrate`, профиль `migrate`, Liquibase до changelog `007-jobs`. Реплики `api` и `worker` стартуют после него с `SPRING_LIQUIBASE_ENABLED=false`. Bucket `messenger-attachments` создаёт API при readiness.
Остановка без удаления данных: `docker compose --env-file .env stop`. Снятие контейнеров с сохранением volumes: `docker compose --env-file .env down`. Намеренное удаление учебных данных: `docker compose --env-file .env down -v`. Файл `.env` в Git не входит.
Свои проверки: `JAVA_HOME=/opt/homebrew/opt/openjdk@21 mvn -f backend/pom.xml test`. Testcontainers поднимает PostgreSQL, Redis, Kafka и SeaweedFS.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab4, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
Клиент ходит только в HAProxy на `127.0.0.1:8080`. Две реплики `api` портов наружу не публикуют. Пользователи, чаты, сообщения, сессии и идемпотентность перенесены из lab4: PostgreSQL — источник истины, Redis — только кэш разбора Bearer.
Загрузка больше не делает производные в запросе. `POST /api/v1/chats/{chat_id}/attachments` проверяет тип, размер и заголовки картинки, пишет байты в ключ `staging/{id}` и в одной транзакции сохраняет строку `attachments` (`queued`) и строку `job_outbox`. После COMMIT API отвечает 202. В брокер на этом шаге ещё ничего не уходит. Если транзакция не прошла, тот же запрос удаляет staging-объект.
Публикацию делает отдельный цикл outbox и в API, и в worker. Строка берётся `FOR UPDATE SKIP LOCKED`, запись уходит в топик `attachments.process` с `acks=all`, и только потом ставится `published_at`. Пока брокер не подтвердил запись, строка остаётся и будет отправлена снова. Пауза публикатора для демонстрации лежит в таблице `demo_flags`, поэтому переживает `kill` API. Сообщение — JSON `attachment.process.v1`: идентификаторы, номер попытки и версия конвейера. Байтов и ключей S3 в нём нет. Ключ Kafka-записи — `attachment_id`. Топик лежит на volume `kafka-data`.
Worker — отдельный процесс (`--spring.profiles.active=worker`, без HTTP). Он коммитит смещение Kafka только после записи исхода в PostgreSQL. Оригинал копируется в `originals/{id}` без изменения байтов. Производные пишутся в фиксированные ключи `variants/{id}/thumbnail`, `crop`, `optimized`. Повторная доставка перезаписывает те же ключи и заново вставляет тот же набор строк `attachment_variants`. Ключ эффекта — `(attachment_id, pipeline_version, variant)`, сейчас `pipeline_version=1`.
Состояния в API: `queued`, `processing`, `ready`, `failed`. Пока статус не `ready` и не успешный `original` после ready, `GET .../content` отвечает 409. Failed-вложение к сообщению не прикрепляется (409). Queued и processing прикрепить можно. `sha256` и `size_bytes` описывают оригинал.
Производные, качество записано в `ImageVariants` и в `.env.example` (`JPEG_QUALITY=0.85`, `WEBP_QUALITY=0.8`):
- thumbnail — JPEG, вписать в 256×256 без увеличения, размеры вниз, минимум 1 px. Фикстура 320×200 даёт 256×160;
- crop — JPEG 128×128 из центра, увеличение разрешено;
- optimized — lossy WebP, вписать в 1280×1280 без увеличения. Та же фикстура остаётся 320×200.
Перед изменением размера применяется EXIF orientation. Альфа кладётся на белый. В JPEG и WebP метаданные исходника не копируются. WebP quality 0.8 — это качество libwebp 80.
Лимиты: тело не больше 10 MiB (`MAX_UPLOAD_BYTES`), пиксели не больше 40 млн (`MAX_IMAGE_PIXELS`, проверка по заголовку до полного decode). Воркер обрывает обработку по `JOB_TIMEOUT_SECONDS` (60 с). Контейнер worker — 1 CPU и 1024 MiB, JVM `-XX:MaxRAMPercentage=60 -XX:+ExitOnOutOfMemoryError`. Одновременно воркер держит `WORKER_CONCURRENCY` потребителей (2) и такой же prefetch.
Повтор: не больше `JOB_MAX_ATTEMPTS` (5). Постоянные ошибки `invalid_image` и `processing_failed` сразу дают `failed`. Временные (S3, обрыв, неожиданный runtime) возвращают задание в `queued` с задержкой. База задержки — 1 с, на каждую следующую попытку она удваивается и упирается в 30 с; фактическая пауза выбирается случайно от половины этой базы до всей базы. После исчерпания попыток код `retry_exhausted`. Исход `failed` пишется в `failed_jobs` и best-effort публикуется в топик `attachments.failed`. Битый JSON пишется туда же, и только потом коммитится смещение, иначе запись навсегда заняла бы партицию.
Явный replay: `POST /internal/demo/attachments/{id}/replay` с заголовком `X-Demo-Token` (`DEMO_TOKEN`). Эти маршруты вне публичного контракта. Для `ready` вызов ничего не ставит в очередь. Для `failed` тот же `attachment_id` снова становится `queued`, счётчик попыток обнуляется, в outbox появляется одна новая строка. Staging failed-задания хранится до `STAGING_RETENTION_SECONDS` (86400), поэтому replay ещё может прочитать байты. После ready staging удаляется только если транзакция исхода закоммитилась.
На время `processing` ставится lease на `JOB_TIMEOUT_SECONDS`. Пока этот процесс сам держит `lease_token`, вторая запись того же задания пропускается. Если процесс умер до `acknowledge()`, новый процесс токена не помнит и забирает `processing` себе. Если запись брокера так и не вернулась, reaper раз в 5 с возвращает просроченный lease в outbox либо закрывает его как `retry_exhausted`.
`/health/ready` смотрит PostgreSQL и S3. Kafka в readiness не входит: брокер может лежать, а upload всё равно отвечает 202 за счёт outbox. Таймауты HAProxy подняты (клиент и разбор запроса 60 с, сервер 30 с), чтобы загрузка не обрывалась на старых 5 с.
| Роль | Сервис | Образ | Готовность | Volume | Сеть | Credentials |
| --- | --- | --- | --- | --- | --- | --- |
| API | `api` × `API_REPLICAS` | `messenger-lab5:1.0.0` | `GET /health/ready` | нет файла оригинала | порт наружу не опубликован | env Spring, S3, Kafka |
| Worker | `worker` | тот же образ, профиль `worker` | `kill -0 1`, restart unless-stopped | нет | порт наружу не опубликован | те же env, `JAVA_TOOL_OPTIONS` |
| Балансировщик | `lb` | `haproxy:3.2.6` | процесс слушает 8080 | нет | только `127.0.0.1:8080` | нет |
| Метаданные | `postgres` | `postgres:16.10` | `pg_isready` | `postgres-data` | 5432 только внутри сети | `POSTGRES_*` из `.env` |
| Кэш сессии | `redis` | `redis:7.4.6` | `redis-cli ping` | `redis-data` | 6379 только внутри сети | без пароля, сеть стенда |
| Брокер | `kafka` | `apache/kafka:3.9.1` | `kafka-broker-api-versions` | `kafka-data` | 9092 только внутри сети | без SASL, порт наружу не опубликован |
| Объекты | `s3` | `chrislusf/seaweedfs:4.52` | `GET :9333/cluster/status` | `s3-data` | 8333/9333 только внутри сети | API: Read/Write/List. Bootstrap: Admin, только создание bucket |
| Миграции | `migrate` | тот же образ | код 0 | нет | нет | тот же datasource |
Лимиты: API и worker по 1 CPU и 1024 MiB, PostgreSQL 1 CPU и 512 MiB, S3 0.5 CPU и 512 MiB, Kafka 0.5 CPU и 768 MiB (heap 256 MiB), Redis и HAProxy по 0.25 CPU и 128 MiB.
Соответствие `.env.example`: переменные lab4 те же. Добавлены `SPRING_KAFKA_BOOTSTRAP_SERVERS`, `WORKER_CONCURRENCY`, `JOB_MAX_ATTEMPTS`, `JOB_TIMEOUT_SECONDS`, `JOB_BACKOFF_BASE_MS`, `JOB_BACKOFF_CAP_MS`, `STAGING_RETENTION_SECONDS`, `DEMO_TOKEN`, `JPEG_QUALITY`, `WEBP_QUALITY`. Имена `*_FILE` из курса оставлены комментарием: стенд передаёт доступ переменными окружения, не файлами секретов. Kafka без пароля, потому что слушатель не опубликован наружу.
## Доказательства критериев
Стенд: Mac, arm64, 12 CPU, 24 GiB RAM, Docker 28.5.1. Прогон 12 ноября 2026. Команды и цифры — `evidence/results.txt` и `evidence/grade4.txt`. Токены туда не записывались.
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Публичный smoke: 202, ready, оригинал и JPEG thumbnail | `make test` | `PASS: lab5 public API scenario` | `evidence/results.txt` |
| Оригинал и метаданные переживают рестарт всего стенда, включая worker и Kafka | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | После смены брокера: `PASS`. Хук перезапускает и `kafka`, volumes не удаляет | `evidence/results.txt`, `scripts/restart.sh` |
| Другая реплика API отдаёт тот же оригинал | `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` на прогретом стенде | 2 instance id до отказа; 117 успешных записей, 1 ошибка на переходе, первая успешная запись через 1.795 с. `PASS`. Реплика поднята `start api` | `evidence/results.txt` |
| Worker остановлен, upload 202, сообщение с вложением, рестарт API, затем worker | `python3 scripts/grade4-evidence.py` | `9c4ab448-af4d-4acf-883d-4a8929a8b5e1`: после рестарта API всё ещё `queued`, content 409 `conflict`. После старта worker — `ready`. Оригинал 111817 байт совпал. Thumbnail JPEG 256×160, crop JPEG 128×128, optimized WebP 320×200 | `evidence/grade4.txt` |
| Задание в брокере переживает рестарт Kafka с volume | worker остановлен, upload, `compose restart kafka` | Конец топика `attachments.process` вырос 2→3 и после рестарта остался 3. Затем то же вложение `45a5da06-a3c3-4d12-ad25-3d5a9154d531` стало `ready` | `evidence/grade4.txt` |
| Crash между COMMIT и publish | outbox на паузе, upload, `compose kill api`, `start api`, снять паузу | До и после kill: `paused=true`, `unpublished=1`, статус `queued`, id `26bd80e5-7074-44b8-b2d5-1a28b14343fe`. После resume тот же id `ready` с тремя вариантами | `evidence/grade4.txt` |
| Падение worker после записи объектов и до commit смещения | файл `/tmp/messenger-crash-after-write` в контейнере worker, один upload | `RestartCount` 0→1. Id `9e1ed645-d5b7-4006-a84c-75ac547eb165` стал `ready`. В БД по одной строке crop, optimized и thumbnail. Replay ready вернул `ready` и не добавил вариант | `evidence/grade4.txt` |
| Постоянная ошибка, DLQ/реестр, явный replay | файл `/tmp/messenger-fail-permanent`, затем replay дважды | Id `05e76bf9-a6d0-4dc9-9ea0-ee1950a1218b`: `failed`, `error_code=processing_failed`. Сообщение с ним — 409 `conflict`. Реестр failed 1→2. Топик `attachments.failed` дошёл до смещения 1. Replay вернул `queued`, затем тот же id `ready`, в БД снова по одной строке варианта. Второй replay остался `ready` | `evidence/grade4.txt` |
| Мусор вместо картинки | POST 415 в том же сценарии | `unsupported_media_type`. Лимиты 10 MiB и 40 млн пикселей закрывает `AttachmentIntegrationTest` | `evidence/grade4.txt`, `mvn test` |
| Приватный bucket и порты | `docker ps` / `inspect` в том же сценарии, анонимный GET из сети стенда | Наружу только `lb` на `127.0.0.1:8080`. У api, worker, postgres, redis, kafka и s3 host-порт пустой. Kafka 768 MiB. Анонимный GET объекта — 403 | `evidence/grade4.txt` |
| Конкуренция worker | лог старта worker | Два потребителя группы `attachment-worker`, партиции `attachments.process-0` и `attachments.process-1` | лог `messenger-lab5-worker-1` |
| Свои тесты | `mvn -f backend/pom.xml -q test` | код выхода 0, 22 теста | `evidence/results.txt` |
| Трек 5 | не заявлен | — | — |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
Живой прогон ждал 60 с истёкшего lease не делал: фикстура обрабатывается быстрее таймаута. Проверен соседний путь — повторная доставка при ещё живом lease после `System.exit` между PUT и COMMIT. Reaper для просроченного lease в коде worker есть (`JOB_TIMEOUT_SECONDS`, период 5 с).
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Отдельного замера RPS нет: оценка 4 этого не требует. Наблюдения ниже сняты тем же стендом.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
| `scripts/restart.sh` сразу после серии kill/crash, пауза хука 5 с | тот же Bearer читает оригинал | `TimeoutError` на GET content: клиент теста ждёт 2 с | подтверждённые данные на месте; хук увеличен до 12 с |
| Тот же хук с паузой 12 с | `/health/ready`, затем чтение оригинала | `PASS` | метаданные, сессия и байты на месте |
| `stop-one.sh` сразу после холодного рестарта | загрузка и остановка одной реплики | `TimeoutError` на POST вложения ещё до hook | повтор на прогретом стенде ниже |
| `stop-one.sh` на прогретом стенде | другая реплика принимает запись, остановленная сама не поднимается | 2 instance id, восстановление 1.795 с, 117 записей и 1 ошибка, `PASS` | подтверждённые сообщения и оригинал на месте; реплика поднята `start api` |
| Worker остановлен, затем убит API | 202 и строка задания не пропадают | статус остался `queued`, после старта worker — `ready` | работа не потеряна |
| Рестарт Kafka при записи в топике | запись остаётся на volume | конец топика 3 до и после рестарта, затем `ready` | задание не потеряно |
| Пауза outbox и `kill` обеих реплик API | строка outbox переживает процесс | `unpublished=1` до и после kill, затем тот же id `ready` | новое задание вручную не создавалось |
| `System.exit` после PUT и до ack | повторная доставка заканчивает тот же результат | контейнер сам поднялся, один набор вариантов | дубля вариантов нет |
| Постоянный сбой до записи вариантов | `failed`, replay, повторный replay ready | `processing_failed`, потом `ready`, второй replay остался `ready` | второй набор вариантов не создан |
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
Один Docker-host не переживает потерю этой машины. PostgreSQL, Redis, SeaweedFS, Kafka и HAProxy в одном экземпляре. Replication factor топиков равен 1.
Доставка at least once. Подтверждение брокера (`acks=all`) и commit смещения потребителя — разные границы. Повтор перезаписывает те же ключи и те же строки вариантов, но это не exactly-once на стороне S3: объект может быть записан дважды одинаковыми байтами.
Если процесс умирает после PUT staging и до COMMIT строки, объект без строки забирает sweeper после grace. Если строка `failed` живёт дольше `STAGING_RETENTION_SECONDS`, replay уже не находит байты и завершается `processing_failed`.
Пока HAProxy не снял backend, часть запросов попадает в останавливаемую реплику. Балансировщик POST не повторяет. Клиент может повторить тот же `Idempotency-Key`. Клиент публичных тестов отказов обрывает запрос через 2 с, поэтому хук рестарта ждёт не только `/health/ready`, но и 12 с, пока SeaweedFS начинает отдавать объекты быстрее этого таймаута.
Внутренние demo-маршруты закрыты общим токеном из окружения и не входят в OpenAPI. Для учебной демонстрации этого достаточно; отдельной роли оператора нет.
+218
View File
@@ -0,0 +1,218 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>ru.mtuci</groupId>
<artifactId>messenger</artifactId>
<version>1.0.0-SNAPSHOT</version>
<name>messenger</name>
<description>MTUSI messenger lab 5</description>
<properties>
<java.version>21</java.version>
<mapstruct.version>1.6.3</mapstruct.version>
<openapi-generator.version>7.17.0</openapi-generator.version>
<swagger-annotations.version>2.2.36</swagger-annotations.version>
<aws.sdk.version>2.31.50</aws.sdk.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webmvc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-liquibase</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-crypto</artifactId>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk18on</artifactId>
<version>1.86</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.mapstruct</groupId>
<artifactId>mapstruct</artifactId>
<version>${mapstruct.version}</version>
</dependency>
<dependency>
<groupId>io.swagger.core.v3</groupId>
<artifactId>swagger-annotations-jakarta</artifactId>
<version>${swagger-annotations.version}</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>${aws.sdk.version}</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>apache-client</artifactId>
<version>${aws.sdk.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-kafka</artifactId>
</dependency>
<dependency>
<groupId>net.coobird</groupId>
<artifactId>thumbnailator</artifactId>
<version>0.4.20</version>
</dependency>
<dependency>
<groupId>com.github.usefulness</groupId>
<artifactId>webp-imageio</artifactId>
<version>0.11.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webmvc-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-testcontainers</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-postgresql</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-kafka</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<finalName>messenger</finalName>
<plugins>
<plugin>
<groupId>org.openapitools</groupId>
<artifactId>openapi-generator-maven-plugin</artifactId>
<version>${openapi-generator.version}</version>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<inputSpec>${project.basedir}/../contracts/openapi.json</inputSpec>
<generatorName>spring</generatorName>
<output>${project.build.directory}/generated-sources/openapi</output>
<modelPackage>ru.mtuci.messenger.api.model</modelPackage>
<apiPackage>ru.mtuci.messenger.api.generated</apiPackage>
<generateApis>false</generateApis>
<generateModels>true</generateModels>
<generateSupportingFiles>false</generateSupportingFiles>
<typeMappings>
<typeMapping>set=List</typeMapping>
</typeMappings>
<instantiationTypes>
<instantiationType>set=ArrayList</instantiationType>
</instantiationTypes>
<configOptions>
<useSpringBoot3>true</useSpringBoot3>
<useJakartaEe>true</useJakartaEe>
<interfaceOnly>true</interfaceOnly>
<dateLibrary>java8</dateLibrary>
<openApiNullable>false</openApiNullable>
<serializationLibrary>jackson</serializationLibrary>
<hideGenerationTimestamp>true</hideGenerationTimestamp>
<additionalModelTypeAnnotations>@com.fasterxml.jackson.annotation.JsonInclude(com.fasterxml.jackson.annotation.JsonInclude.Include.ALWAYS)</additionalModelTypeAnnotations>
</configOptions>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>jackson3-model-annotations</id>
<phase>generate-sources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<replace dir="${project.build.directory}/generated-sources/openapi" token="import com.fasterxml.jackson.databind.annotation.JsonDeserialize;" value="">
<include name="**/*.java"/>
</replace>
<replace dir="${project.build.directory}/generated-sources/openapi" token="@JsonDeserialize(as = LinkedHashSet.class)" value="">
<include name="**/*.java"/>
</replace>
<!-- Generator builds a required-args constructor that drops attachment_ids. Jackson uses it, so copy the list back. -->
<replace dir="${project.build.directory}/generated-sources/openapi" token="public CreateMessage(String text, Object attachmentIds) {${line.separator} this.text = text;${line.separator} }" value="public CreateMessage(String text, Object attachmentIds) {${line.separator} this.text = text;${line.separator} if (attachmentIds instanceof java.util.List&lt;?&gt; ids) {${line.separator} this.attachmentIds = new java.util.ArrayList&lt;&gt;();${line.separator} for (Object id : ids) {${line.separator} this.attachmentIds.add(java.util.UUID.fromString(String.valueOf(id)));${line.separator} }${line.separator} }${line.separator} }">
<include name="**/CreateMessage.java"/>
</replace>
</target>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<annotationProcessorPaths>
<path>
<groupId>org.mapstruct</groupId>
<artifactId>mapstruct-processor</artifactId>
<version>${mapstruct.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
</plugins>
</build>
</project>
@@ -0,0 +1,20 @@
package ru.mtuci.messenger;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.security.crypto.argon2.Argon2PasswordEncoder;
import org.springframework.security.crypto.password.PasswordEncoder;
@SpringBootApplication
public class MessengerApplication {
public static void main(String[] args) {
SpringApplication.run(MessengerApplication.class, args);
}
@Bean
PasswordEncoder passwordEncoder() {
return Argon2PasswordEncoder.defaultsForSpringSecurity_v5_8();
}
}
@@ -0,0 +1,52 @@
package ru.mtuci.messenger.api.attachment;
import jakarta.servlet.http.HttpServletRequest;
import org.springframework.core.io.InputStreamResource;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import ru.mtuci.messenger.api.model.Attachment;
import ru.mtuci.messenger.service.AttachmentService;
import ru.mtuci.messenger.web.BearerAuthFilter;
import java.io.IOException;
import java.util.UUID;
@RestController
public class AttachmentController {
private final AttachmentService attachments;
public AttachmentController(AttachmentService attachments) {
this.attachments = attachments;
}
@PostMapping(value = "/api/v1/chats/{chatId}/attachments", consumes = {"image/png", "image/jpeg"})
public ResponseEntity<Attachment> upload(HttpServletRequest request, @PathVariable UUID chatId,
@RequestParam("filename") String filename) throws IOException {
long length = request.getContentLengthLong();
Attachment created = attachments.upload(
BearerAuthFilter.userId(request), chatId, filename, request.getContentType(), length, request.getInputStream());
return ResponseEntity.status(HttpStatus.ACCEPTED).body(created);
}
@GetMapping("/api/v1/attachments/{attachmentId}")
public Attachment get(HttpServletRequest request, @PathVariable UUID attachmentId) {
return attachments.get(BearerAuthFilter.userId(request), attachmentId);
}
@GetMapping("/api/v1/attachments/{attachmentId}/content")
public ResponseEntity<InputStreamResource> content(HttpServletRequest request, @PathVariable UUID attachmentId,
@RequestParam(required = false) String variant) {
var download = attachments.content(BearerAuthFilter.userId(request), attachmentId, variant);
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType(download.contentType()))
.contentLength(download.size())
.body(new InputStreamResource(download.body()));
}
}
@@ -0,0 +1,51 @@
package ru.mtuci.messenger.api.chat;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.validation.Valid;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.Size;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import ru.mtuci.messenger.api.model.Chat;
import ru.mtuci.messenger.api.model.ChatPage;
import ru.mtuci.messenger.api.model.CreateChat;
import ru.mtuci.messenger.service.ChatService;
import ru.mtuci.messenger.web.BearerAuthFilter;
import java.util.UUID;
@Validated
@RestController
public class ChatController {
private final ChatService chats;
public ChatController(ChatService chats) {
this.chats = chats;
}
@PostMapping("/api/v1/chats")
public ResponseEntity<Chat> create(HttpServletRequest request, @Valid @RequestBody CreateChat body) {
return ResponseEntity.status(HttpStatus.CREATED).body(chats.create(BearerAuthFilter.userId(request), body));
}
@GetMapping("/api/v1/chats")
public ChatPage list(HttpServletRequest request,
@RequestParam(required = false) @Min(1) @Max(100) Integer limit,
@RequestParam(required = false) @Size(min = 1) String cursor) {
return chats.list(BearerAuthFilter.userId(request), limit, cursor);
}
@GetMapping("/api/v1/chats/{chatId}")
public Chat get(HttpServletRequest request, @PathVariable UUID chatId) {
return chats.get(BearerAuthFilter.userId(request), chatId);
}
}
@@ -0,0 +1,77 @@
package ru.mtuci.messenger.api.demo;
import jakarta.servlet.http.HttpServletRequest;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Profile;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RestController;
import ru.mtuci.messenger.api.model.Attachment;
import ru.mtuci.messenger.exception.UnauthorizedException;
import ru.mtuci.messenger.repository.FailedJobRepository;
import ru.mtuci.messenger.repository.OutboxRepository;
import ru.mtuci.messenger.service.AttachmentService;
import ru.mtuci.messenger.service.DemoFlags;
import java.util.Map;
import java.util.UUID;
@RestController
@Profile("!worker")
public class DemoController {
private final DemoFlags flags;
private final OutboxRepository outbox;
private final FailedJobRepository failedJobs;
private final AttachmentService attachments;
private final String token;
public DemoController(DemoFlags flags, OutboxRepository outbox, FailedJobRepository failedJobs,
AttachmentService attachments, @Value("${app.demo.token:lab-demo}") String token) {
this.flags = flags;
this.outbox = outbox;
this.failedJobs = failedJobs;
this.attachments = attachments;
this.token = token;
}
@GetMapping("/internal/demo/outbox")
public Map<String, Object> outbox(HttpServletRequest request) {
authorize(request);
return Map.of("paused", flags.outboxPaused(), "unpublished", outbox.countByPublishedAtIsNull());
}
@PostMapping("/internal/demo/outbox-pause")
public Map<String, Object> pause(HttpServletRequest request) {
authorize(request);
flags.setOutboxPaused(true);
return Map.of("paused", true);
}
@PostMapping("/internal/demo/outbox-resume")
public Map<String, Object> resume(HttpServletRequest request) {
authorize(request);
flags.setOutboxPaused(false);
return Map.of("paused", false);
}
@GetMapping("/internal/demo/failed")
public Map<String, Object> failed(HttpServletRequest request) {
authorize(request);
return Map.of("count", failedJobs.countByReplayedAtIsNull());
}
@PostMapping("/internal/demo/attachments/{attachmentId}/replay")
public Attachment replay(HttpServletRequest request, @PathVariable UUID attachmentId) {
authorize(request);
return attachments.replay(attachmentId);
}
private void authorize(HttpServletRequest request) {
String header = request.getHeader("X-Demo-Token");
if (header == null || !header.equals(token)) {
throw UnauthorizedException.bearer();
}
}
}
@@ -0,0 +1,46 @@
package ru.mtuci.messenger.api.health;
import org.springframework.boot.health.actuate.endpoint.HealthDescriptor;
import org.springframework.context.annotation.Profile;
import org.springframework.boot.health.actuate.endpoint.HealthEndpoint;
import org.springframework.boot.health.contributor.Status;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import ru.mtuci.messenger.api.model.Health;
import ru.mtuci.messenger.exception.UnavailableException;
@RestController
@Profile("!worker")
public class HealthController {
private final HealthEndpoint healthEndpoint;
public HealthController(HealthEndpoint healthEndpoint) {
this.healthEndpoint = healthEndpoint;
}
@GetMapping("/health/live")
public Health live() {
if (!up("liveness")) {
throw new IllegalStateException("liveness");
}
return ok();
}
@GetMapping("/health/ready")
public Health ready() {
if (!up("readiness")) {
throw new UnavailableException("Сервис не готов");
}
return ok();
}
private boolean up(String path) {
HealthDescriptor descriptor = healthEndpoint.healthForPath(path);
return descriptor != null && Status.UP.equals(descriptor.getStatus());
}
private Health ok() {
return new Health(Health.StatusEnum.OK);
}
}
@@ -0,0 +1,50 @@
package ru.mtuci.messenger.api.message;
import jakarta.validation.Valid;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.Size;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import ru.mtuci.messenger.api.model.CreateMessage;
import ru.mtuci.messenger.api.model.Message;
import ru.mtuci.messenger.api.model.MessagePage;
import jakarta.servlet.http.HttpServletRequest;
import ru.mtuci.messenger.service.MessageService;
import ru.mtuci.messenger.web.BearerAuthFilter;
import java.util.UUID;
@Validated
@RestController
public class MessageController {
private final MessageService messageService;
public MessageController(MessageService messageService) {
this.messageService = messageService;
}
@PostMapping("/api/v1/chats/{chatId}/messages")
public ResponseEntity<Message> create(HttpServletRequest request, @PathVariable UUID chatId,
@RequestHeader(value = "Idempotency-Key", required = false) String idempotencyKey,
@Valid @RequestBody CreateMessage body) {
return ResponseEntity.status(HttpStatus.CREATED)
.body(messageService.create(BearerAuthFilter.userId(request), chatId, idempotencyKey, body));
}
@GetMapping("/api/v1/chats/{chatId}/messages")
public MessagePage list(HttpServletRequest request, @PathVariable UUID chatId,
@RequestParam(required = false) @Min(1) @Max(100) Integer limit,
@RequestParam(required = false) @Size(min = 1) String cursor) {
return messageService.list(BearerAuthFilter.userId(request), chatId, limit, cursor);
}
}
@@ -0,0 +1,47 @@
package ru.mtuci.messenger.api.user;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.validation.Valid;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
import ru.mtuci.messenger.api.model.CreateUser;
import ru.mtuci.messenger.api.model.Session;
import ru.mtuci.messenger.api.model.User;
import ru.mtuci.messenger.service.UserService;
import ru.mtuci.messenger.web.BearerAuthFilter;
@RestController
public class UserController {
private final UserService users;
public UserController(UserService users) {
this.users = users;
}
@PostMapping("/api/v1/users")
public ResponseEntity<User> register(@Valid @RequestBody CreateUser request) {
return ResponseEntity.status(HttpStatus.CREATED).body(users.register(request));
}
@GetMapping("/api/v1/users/me")
public User me(HttpServletRequest request) {
return users.current(BearerAuthFilter.userId(request));
}
@PostMapping("/api/v1/auth/sessions")
public ResponseEntity<Session> login(HttpServletRequest request) {
return ResponseEntity.status(HttpStatus.CREATED).body(users.login(request.getHeader("Authorization")));
}
@DeleteMapping("/api/v1/auth/sessions/current")
public ResponseEntity<Void> logout(HttpServletRequest request) {
users.revoke((String) request.getAttribute(BearerAuthFilter.TOKEN));
return ResponseEntity.noContent().build();
}
}
@@ -0,0 +1,61 @@
package ru.mtuci.messenger.cache;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.dao.DataAccessException;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;
import java.time.Duration;
import java.time.Instant;
import java.util.Optional;
import java.util.UUID;
/**
* Кэш Bearer-сессий: session:{sha256(token)} -> userId, TTL = остаток жизни сессии.
* Источник истины — таблица sessions; при ошибке Redis вызывающий идёт в PostgreSQL.
*/
@Component
public class SessionCache {
private static final Logger log = LoggerFactory.getLogger(SessionCache.class);
private final StringRedisTemplate redis;
public SessionCache(StringRedisTemplate redis) {
this.redis = redis;
}
public Optional<UUID> find(String tokenHash) {
try {
return Optional.ofNullable(redis.opsForValue().get(key(tokenHash))).map(UUID::fromString);
} catch (DataAccessException exception) {
log.warn("session cache bypass: {}", exception.getClass().getSimpleName());
return Optional.empty();
}
}
public void put(String tokenHash, UUID userId, Instant expiresAt) {
Duration ttl = Duration.between(Instant.now(), expiresAt);
if (ttl.isNegative() || ttl.isZero()) {
return;
}
try {
redis.opsForValue().set(key(tokenHash), userId.toString(), ttl);
} catch (DataAccessException exception) {
log.warn("session cache write skipped: {}", exception.getClass().getSimpleName());
}
}
public void evict(String tokenHash) {
try {
redis.delete(key(tokenHash));
} catch (DataAccessException exception) {
log.warn("session cache evict skipped: {}", exception.getClass().getSimpleName());
}
}
private static String key(String tokenHash) {
return "session:" + tokenHash;
}
}
@@ -0,0 +1,210 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.UUID;
@Entity
@Table(name = "attachments")
public class AttachmentEntity {
@Id
private UUID id;
@Column(name = "chat_id", nullable = false)
private UUID chatId;
@Column(name = "owner_id", nullable = false)
private UUID ownerId;
@Column(nullable = false, length = 255)
private String filename;
@Column(name = "content_type", nullable = false, length = 32)
private String contentType;
@Column(name = "size_bytes", nullable = false)
private long sizeBytes;
@Column(nullable = false, length = 64)
private String sha256;
@Column(name = "object_key", nullable = false, length = 120)
private String objectKey;
@Column(nullable = false, length = 16)
private String status;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
@Column(name = "updated_at", nullable = false)
private Instant updatedAt;
@Column(name = "error_code", length = 64)
private String errorCode;
@Column(name = "pipeline_version", nullable = false)
private int pipelineVersion;
@Column(nullable = false)
private int attempt;
@Column(name = "job_id")
private UUID jobId;
@Column(name = "staging_key", length = 120)
private String stagingKey;
@Column(name = "lease_until")
private Instant leaseUntil;
@Column(name = "lease_token")
private UUID leaseToken;
protected AttachmentEntity() {
}
public AttachmentEntity(UUID id, UUID chatId, UUID ownerId, String filename, String contentType, long sizeBytes,
String sha256, String objectKey, String stagingKey, UUID jobId) {
Instant now = Instant.now().truncatedTo(ChronoUnit.MICROS);
this.id = id;
this.chatId = chatId;
this.ownerId = ownerId;
this.filename = filename;
this.contentType = contentType;
this.sizeBytes = sizeBytes;
this.sha256 = sha256;
this.objectKey = objectKey;
this.stagingKey = stagingKey;
this.jobId = jobId;
this.status = "queued";
this.pipelineVersion = 1;
this.attempt = 0;
this.createdAt = now;
this.updatedAt = now;
}
public boolean leaseHeld(Instant now) {
return leaseUntil != null && leaseUntil.isAfter(now);
}
public void markQueued(Instant now) {
this.status = "queued";
this.errorCode = null;
this.leaseUntil = null;
this.leaseToken = null;
this.updatedAt = now;
}
public void markProcessing(int attempt, UUID leaseToken, Instant leaseUntil, Instant now) {
this.status = "processing";
this.attempt = attempt;
this.leaseToken = leaseToken;
this.leaseUntil = leaseUntil;
this.errorCode = null;
this.updatedAt = now;
}
public void markReady(Instant now) {
this.status = "ready";
this.errorCode = null;
this.leaseUntil = null;
this.leaseToken = null;
this.updatedAt = now;
}
public void markFailed(String errorCode, Instant now) {
this.status = "failed";
this.errorCode = errorCode;
this.leaseUntil = null;
this.leaseToken = null;
this.updatedAt = now;
}
public void prepareReplay(Instant now) {
this.status = "queued";
this.errorCode = null;
this.attempt = 0;
this.leaseUntil = null;
this.leaseToken = null;
this.updatedAt = now;
}
public UUID getId() {
return id;
}
public UUID getChatId() {
return chatId;
}
public UUID getOwnerId() {
return ownerId;
}
public String getFilename() {
return filename;
}
public String getContentType() {
return contentType;
}
public long getSizeBytes() {
return sizeBytes;
}
public String getSha256() {
return sha256;
}
public String getObjectKey() {
return objectKey;
}
public String getStatus() {
return status;
}
public Instant getCreatedAt() {
return createdAt;
}
public Instant getUpdatedAt() {
return updatedAt;
}
public String getErrorCode() {
return errorCode;
}
public int getPipelineVersion() {
return pipelineVersion;
}
public int getAttempt() {
return attempt;
}
public UUID getJobId() {
return jobId;
}
public String getStagingKey() {
return stagingKey;
}
public Instant getLeaseUntil() {
return leaseUntil;
}
public UUID getLeaseToken() {
return leaseToken;
}
}
@@ -0,0 +1,100 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.util.UUID;
@Entity
@Table(name = "attachment_variants")
public class AttachmentVariantEntity {
@Id
private UUID id;
@Column(name = "attachment_id", nullable = false)
private UUID attachmentId;
@Column(name = "pipeline_version", nullable = false)
private int pipelineVersion;
@Column(nullable = false, length = 16)
private String name;
@Column(name = "content_type", nullable = false, length = 32)
private String contentType;
@Column(name = "size_bytes", nullable = false)
private long sizeBytes;
@Column(nullable = false)
private int width;
@Column(nullable = false)
private int height;
@Column(nullable = false, length = 64)
private String sha256;
@Column(name = "object_key", nullable = false, length = 120)
private String objectKey;
protected AttachmentVariantEntity() {
}
public AttachmentVariantEntity(UUID id, UUID attachmentId, int pipelineVersion, String name, String contentType,
long sizeBytes, int width, int height, String sha256, String objectKey) {
this.id = id;
this.attachmentId = attachmentId;
this.pipelineVersion = pipelineVersion;
this.name = name;
this.contentType = contentType;
this.sizeBytes = sizeBytes;
this.width = width;
this.height = height;
this.sha256 = sha256;
this.objectKey = objectKey;
}
public UUID getId() {
return id;
}
public UUID getAttachmentId() {
return attachmentId;
}
public int getPipelineVersion() {
return pipelineVersion;
}
public String getName() {
return name;
}
public String getContentType() {
return contentType;
}
public long getSizeBytes() {
return sizeBytes;
}
public int getWidth() {
return width;
}
public int getHeight() {
return height;
}
public String getSha256() {
return sha256;
}
public String getObjectKey() {
return objectKey;
}
}
@@ -0,0 +1,45 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.UUID;
@Entity
@Table(name = "chats")
public class ChatEntity {
@Id
private UUID id;
@Column(nullable = false, length = 100)
private String title;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
protected ChatEntity() {
}
public ChatEntity(String title) {
this.id = UUID.randomUUID();
this.title = title;
this.createdAt = Instant.now().truncatedTo(ChronoUnit.MICROS);
}
public UUID getId() {
return id;
}
public String getTitle() {
return title;
}
public Instant getCreatedAt() {
return createdAt;
}
}
@@ -0,0 +1,47 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.IdClass;
import jakarta.persistence.Table;
import java.util.UUID;
@Entity
@Table(name = "chat_members")
@IdClass(ChatMemberId.class)
public class ChatMemberEntity {
@Id
@Column(name = "chat_id", nullable = false)
private UUID chatId;
@Id
@Column(name = "user_id", nullable = false)
private UUID userId;
@Column(name = "ordinal", nullable = false)
private int ordinal;
protected ChatMemberEntity() {
}
public ChatMemberEntity(UUID chatId, UUID userId, int ordinal) {
this.chatId = chatId;
this.userId = userId;
this.ordinal = ordinal;
}
public UUID getChatId() {
return chatId;
}
public UUID getUserId() {
return userId;
}
public int getOrdinal() {
return ordinal;
}
}
@@ -0,0 +1,34 @@
package ru.mtuci.messenger.domain;
import java.io.Serializable;
import java.util.UUID;
public class ChatMemberId implements Serializable {
private UUID chatId;
private UUID userId;
public ChatMemberId() {
}
public ChatMemberId(UUID chatId, UUID userId) {
this.chatId = chatId;
this.userId = userId;
}
@Override
public boolean equals(Object other) {
if (this == other) {
return true;
}
if (!(other instanceof ChatMemberId that)) {
return false;
}
return chatId.equals(that.chatId) && userId.equals(that.userId);
}
@Override
public int hashCode() {
return chatId.hashCode() * 31 + userId.hashCode();
}
}
@@ -0,0 +1,34 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
@Entity
@Table(name = "demo_flags")
public class DemoFlag {
@Id
@Column(nullable = false, length = 64)
private String name;
@Column(nullable = false)
private boolean enabled;
protected DemoFlag() {
}
public DemoFlag(String name, boolean enabled) {
this.name = name;
this.enabled = enabled;
}
public boolean isEnabled() {
return enabled;
}
public void setEnabled(boolean enabled) {
this.enabled = enabled;
}
}
@@ -0,0 +1,51 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.Instant;
import java.util.UUID;
@Entity
@Table(name = "failed_jobs")
public class FailedJobEntity {
@Id
private UUID id;
@Column(name = "job_id", nullable = false)
private UUID jobId;
@Column(name = "attachment_id", nullable = false)
private UUID attachmentId;
@Column(name = "error_code", nullable = false, length = 64)
private String errorCode;
@Column(nullable = false)
private int attempt;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
@Column(name = "replayed_at")
private Instant replayedAt;
protected FailedJobEntity() {
}
public FailedJobEntity(UUID id, UUID jobId, UUID attachmentId, String errorCode, int attempt, Instant createdAt) {
this.id = id;
this.jobId = jobId;
this.attachmentId = attachmentId;
this.errorCode = errorCode;
this.attempt = attempt;
this.createdAt = createdAt;
}
public void markReplayed(Instant replayedAt) {
this.replayedAt = replayedAt;
}
}
@@ -0,0 +1,69 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.Instant;
import java.util.UUID;
@Entity
@Table(name = "idempotency_records")
public class IdempotencyRecord {
@Id
private UUID id;
@Column(name = "user_id", nullable = false)
private UUID userId;
@Column(name = "http_method", nullable = false, length = 8)
private String httpMethod;
@Column(nullable = false, length = 256)
private String path;
@Column(name = "idempotency_key", nullable = false, length = 128)
private String idempotencyKey;
@Column(name = "request_hash", nullable = false, length = 64)
private String requestHash;
@Column(name = "message_id", nullable = false)
private UUID messageId;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
@Column(name = "expires_at", nullable = false)
private Instant expiresAt;
protected IdempotencyRecord() {
}
public IdempotencyRecord(UUID userId, String httpMethod, String path, String idempotencyKey, String requestHash,
UUID messageId, Instant createdAt, Instant expiresAt) {
this.id = UUID.randomUUID();
this.userId = userId;
this.httpMethod = httpMethod;
this.path = path;
this.idempotencyKey = idempotencyKey;
this.requestHash = requestHash;
this.messageId = messageId;
this.createdAt = createdAt;
this.expiresAt = expiresAt;
}
public UUID getMessageId() {
return messageId;
}
public String getRequestHash() {
return requestHash;
}
public Instant getExpiresAt() {
return expiresAt;
}
}
@@ -0,0 +1,79 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.CollectionTable;
import jakarta.persistence.Column;
import jakarta.persistence.ElementCollection;
import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.Id;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.OrderColumn;
import jakarta.persistence.Table;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
@Entity
@Table(name = "messages")
public class MessageEntity {
@Id
private UUID id;
@Column(name = "chat_id", nullable = false)
private UUID chatId;
@Column(name = "sender_id", nullable = false)
private UUID senderId;
@Column(nullable = false, length = 4000)
private String text;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
@ElementCollection(fetch = FetchType.EAGER)
@CollectionTable(name = "message_attachments", joinColumns = @JoinColumn(name = "message_id"))
@OrderColumn(name = "sort_order")
@Column(name = "attachment_id", nullable = false)
private List<UUID> attachmentIds = new ArrayList<>();
protected MessageEntity() {
}
public MessageEntity(UUID chatId, UUID senderId, String text, List<UUID> attachmentIds) {
this.id = UUID.randomUUID();
this.chatId = chatId;
this.senderId = senderId;
this.text = text;
this.createdAt = Instant.now().truncatedTo(ChronoUnit.MICROS);
this.attachmentIds = new ArrayList<>(attachmentIds);
}
public UUID getId() {
return id;
}
public UUID getChatId() {
return chatId;
}
public UUID getSenderId() {
return senderId;
}
public String getText() {
return text;
}
public Instant getCreatedAt() {
return createdAt;
}
public List<UUID> getAttachmentIds() {
return attachmentIds;
}
}
@@ -0,0 +1,72 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.Instant;
import java.util.UUID;
@Entity
@Table(name = "job_outbox")
public class OutboxMessage {
@Id
private UUID id;
@Column(name = "job_id", nullable = false)
private UUID jobId;
@Column(name = "attachment_id", nullable = false)
private UUID attachmentId;
@Column(nullable = false)
private int attempt;
@Column(nullable = false, length = 4000)
private String payload;
@Column(name = "available_at", nullable = false)
private Instant availableAt;
@Column(name = "published_at")
private Instant publishedAt;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
protected OutboxMessage() {
}
public OutboxMessage(UUID id, UUID jobId, UUID attachmentId, int attempt, String payload, Instant availableAt,
Instant createdAt) {
this.id = id;
this.jobId = jobId;
this.attachmentId = attachmentId;
this.attempt = attempt;
this.payload = payload;
this.availableAt = availableAt;
this.createdAt = createdAt;
}
public UUID getId() {
return id;
}
public UUID getAttachmentId() {
return attachmentId;
}
public String getPayload() {
return payload;
}
public Instant getPublishedAt() {
return publishedAt;
}
public void setPublishedAt(Instant publishedAt) {
this.publishedAt = publishedAt;
}
}
@@ -0,0 +1,53 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.Duration;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.UUID;
@Entity
@Table(name = "sessions")
public class SessionEntity {
@Id
private UUID id;
@Column(name = "token_hash", nullable = false, unique = true, length = 64)
private String tokenHash;
@Column(name = "user_id", nullable = false)
private UUID userId;
@Column(name = "expires_at", nullable = false)
private Instant expiresAt;
@Column(name = "revoked_at")
private Instant revokedAt;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
protected SessionEntity() {
}
public SessionEntity(String tokenHash, UUID userId, Duration ttl) {
this.id = UUID.randomUUID();
this.tokenHash = tokenHash;
this.userId = userId;
this.createdAt = Instant.now().truncatedTo(ChronoUnit.MICROS);
this.expiresAt = createdAt.plus(ttl);
}
public UUID getUserId() {
return userId;
}
public Instant getExpiresAt() {
return expiresAt;
}
}
@@ -0,0 +1,61 @@
package ru.mtuci.messenger.domain;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.UUID;
@Entity
@Table(name = "users")
public class UserAccount {
@Id
private UUID id;
@Column(nullable = false, unique = true, length = 32)
private String username;
@Column(name = "display_name", nullable = false, length = 100)
private String displayName;
@Column(name = "password_hash", nullable = false, length = 255)
private String passwordHash;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
protected UserAccount() {
}
public UserAccount(String username, String displayName, String passwordHash) {
this.id = UUID.randomUUID();
this.username = username;
this.displayName = displayName;
this.passwordHash = passwordHash;
this.createdAt = Instant.now().truncatedTo(ChronoUnit.MICROS);
}
public UUID getId() {
return id;
}
public String getUsername() {
return username;
}
public String getDisplayName() {
return displayName;
}
public String getPasswordHash() {
return passwordHash;
}
public Instant getCreatedAt() {
return createdAt;
}
}
@@ -0,0 +1,24 @@
package ru.mtuci.messenger.exception;
import org.springframework.http.HttpStatus;
import ru.mtuci.messenger.api.model.ErrorError;
public class ApiException extends RuntimeException {
private final HttpStatus status;
private final ErrorError.CodeEnum code;
public ApiException(HttpStatus status, ErrorError.CodeEnum code, String message) {
super(message);
this.status = status;
this.code = code;
}
public HttpStatus status() {
return status;
}
public ErrorError.CodeEnum code() {
return code;
}
}
@@ -0,0 +1,11 @@
package ru.mtuci.messenger.exception;
import org.springframework.http.HttpStatus;
import ru.mtuci.messenger.api.model.ErrorError;
public class ConflictException extends ApiException {
public ConflictException(String message) {
super(HttpStatus.CONFLICT, ErrorError.CodeEnum.CONFLICT, message);
}
}
@@ -0,0 +1,11 @@
package ru.mtuci.messenger.exception;
import org.springframework.http.HttpStatus;
import ru.mtuci.messenger.api.model.ErrorError;
public class InvalidRequestException extends ApiException {
public InvalidRequestException(String message) {
super(HttpStatus.BAD_REQUEST, ErrorError.CodeEnum.INVALID_REQUEST, message);
}
}
@@ -0,0 +1,11 @@
package ru.mtuci.messenger.exception;
import org.springframework.http.HttpStatus;
import ru.mtuci.messenger.api.model.ErrorError;
public class NotFoundException extends ApiException {
public NotFoundException() {
super(HttpStatus.NOT_FOUND, ErrorError.CodeEnum.NOT_FOUND, "Объект не найден");
}
}
@@ -0,0 +1,11 @@
package ru.mtuci.messenger.exception;
import org.springframework.http.HttpStatus;
import ru.mtuci.messenger.api.model.ErrorError;
public class PayloadTooLargeException extends ApiException {
public PayloadTooLargeException() {
super(HttpStatus.PAYLOAD_TOO_LARGE, ErrorError.CodeEnum.PAYLOAD_TOO_LARGE, "Файл слишком большой");
}
}
@@ -0,0 +1,26 @@
package ru.mtuci.messenger.exception;
import org.springframework.http.HttpStatus;
import ru.mtuci.messenger.api.model.ErrorError;
public class UnauthorizedException extends ApiException {
private final String challenge;
private UnauthorizedException(String challenge) {
super(HttpStatus.UNAUTHORIZED, ErrorError.CodeEnum.UNAUTHORIZED, "Требуется аутентификация");
this.challenge = challenge;
}
public static UnauthorizedException basic() {
return new UnauthorizedException("Basic realm=\"messenger\", charset=\"UTF-8\"");
}
public static UnauthorizedException bearer() {
return new UnauthorizedException("Bearer realm=\"messenger\"");
}
public String challenge() {
return challenge;
}
}
@@ -0,0 +1,11 @@
package ru.mtuci.messenger.exception;
import org.springframework.http.HttpStatus;
import ru.mtuci.messenger.api.model.ErrorError;
public class UnavailableException extends ApiException {
public UnavailableException(String message) {
super(HttpStatus.SERVICE_UNAVAILABLE, ErrorError.CodeEnum.UNAVAILABLE, message);
}
}
@@ -0,0 +1,11 @@
package ru.mtuci.messenger.exception;
import org.springframework.http.HttpStatus;
import ru.mtuci.messenger.api.model.ErrorError;
public class UnsupportedMediaException extends ApiException {
public UnsupportedMediaException() {
super(HttpStatus.UNSUPPORTED_MEDIA_TYPE, ErrorError.CodeEnum.UNSUPPORTED_MEDIA_TYPE, "Неподдерживаемый тип содержимого");
}
}
@@ -0,0 +1,393 @@
package ru.mtuci.messenger.job;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
import ru.mtuci.messenger.domain.AttachmentEntity;
import ru.mtuci.messenger.domain.AttachmentVariantEntity;
import ru.mtuci.messenger.domain.FailedJobEntity;
import ru.mtuci.messenger.domain.OutboxMessage;
import ru.mtuci.messenger.exception.ConflictException;
import ru.mtuci.messenger.exception.NotFoundException;
import ru.mtuci.messenger.exception.PayloadTooLargeException;
import ru.mtuci.messenger.exception.UnavailableException;
import ru.mtuci.messenger.exception.UnsupportedMediaException;
import ru.mtuci.messenger.repository.AttachmentRepository;
import ru.mtuci.messenger.repository.AttachmentVariantRepository;
import ru.mtuci.messenger.repository.FailedJobRepository;
import ru.mtuci.messenger.repository.OutboxRepository;
import ru.mtuci.messenger.service.ImageFiles;
import ru.mtuci.messenger.service.ImageVariants;
import ru.mtuci.messenger.service.RetryDelay;
import ru.mtuci.messenger.storage.ObjectStorage;
import tools.jackson.databind.json.JsonMapper;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.time.Duration;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.HexFormat;
import java.util.Set;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
@Service
public class AttachmentJobs {
private static final Logger log = LoggerFactory.getLogger(AttachmentJobs.class);
private static final Path CRASH_AFTER_WRITE = Path.of("/tmp/messenger-crash-after-write");
private static final Path FAIL_PERMANENT = Path.of("/tmp/messenger-fail-permanent");
private final AttachmentRepository attachments;
private final AttachmentVariantRepository variants;
private final OutboxRepository outbox;
private final FailedJobRepository failedJobs;
private final ObjectStorage storage;
private final JsonMapper json;
private final TransactionTemplate transactions;
private final ObjectProvider<BrokerPublisher> brokers;
private final int maxAttempts;
private final Duration timeout;
private final long backoffBaseMs;
private final long backoffCapMs;
private final int maxBytes;
private final long maxPixels;
private final float jpegQuality;
private final float webpQuality;
private final Set<UUID> localLeases = ConcurrentHashMap.newKeySet();
public AttachmentJobs(AttachmentRepository attachments, AttachmentVariantRepository variants, OutboxRepository outbox,
FailedJobRepository failedJobs, ObjectStorage storage, JsonMapper json,
PlatformTransactionManager transactionManager, ObjectProvider<BrokerPublisher> brokers,
@Value("${app.job.max-attempts:5}") int maxAttempts,
@Value("${app.job.timeout:60s}") Duration timeout,
@Value("${app.job.backoff-base-ms:1000}") long backoffBaseMs,
@Value("${app.job.backoff-cap-ms:30000}") long backoffCapMs,
@Value("${app.upload.max-bytes:10485760}") int maxBytes,
@Value("${app.upload.max-pixels:40000000}") long maxPixels,
@Value("${app.image.jpeg-quality:0.85}") float jpegQuality,
@Value("${app.image.webp-quality:0.8}") float webpQuality) {
this.attachments = attachments;
this.variants = variants;
this.outbox = outbox;
this.failedJobs = failedJobs;
this.storage = storage;
this.json = json;
this.transactions = new TransactionTemplate(transactionManager);
this.brokers = brokers;
this.maxAttempts = maxAttempts;
this.timeout = timeout;
this.backoffBaseMs = backoffBaseMs;
this.backoffCapMs = backoffCapMs;
this.maxBytes = maxBytes;
this.maxPixels = maxPixels;
this.jpegQuality = jpegQuality;
this.webpQuality = webpQuality;
}
public void enqueue(AttachmentEntity attachment, int attempt, Instant availableAt) {
JobPayload payload = new JobPayload(1, attachment.getJobId(), "attachment.process.v1", attachment.getId(),
attachment.getPipelineVersion(), attempt, attachment.getCreatedAt());
outbox.save(new OutboxMessage(UUID.randomUUID(), attachment.getJobId(), attachment.getId(), attempt,
json.writeValueAsString(payload), availableAt, now()));
}
public void handle(String body, boolean redelivered) {
JobPayload job = parse(body);
Claim claim = transactions.execute(status -> claim(job, redelivered));
if (claim == null || !claim.process()) {
if (claim != null && claim.dlq()) {
publishFailed(body);
}
return;
}
try {
try {
Processed processed = process(claim);
Boolean saved = transactions.execute(status -> finishReady(claim, processed));
if (Boolean.TRUE.equals(saved)) {
storage.deleteQuietly(claim.stagingKey());
}
} catch (PermanentJobException exception) {
Boolean saved = transactions.execute(status -> finishFailed(claim, exception.errorCode()));
if (Boolean.TRUE.equals(saved)) {
publishFailed(body);
}
} catch (TransientJobException exception) {
log.info("transient job failure attachment={} type={}", claim.attachmentId(), exception.getClass().getSimpleName());
transactions.executeWithoutResult(status -> finishRetry(claim));
}
} finally {
localLeases.remove(claim.leaseToken());
}
}
public void rejectPoison(String body) {
log.warn("poison job");
publishFailed(body);
}
public AttachmentEntity replay(UUID attachmentId) {
return transactions.execute(status -> {
AttachmentEntity attachment = attachments.lockById(attachmentId).orElseThrow(NotFoundException::new);
if ("ready".equals(attachment.getStatus())) {
return attachment;
}
if (!"failed".equals(attachment.getStatus())) {
throw new ConflictException("Задание ещё выполняется");
}
Instant now = now();
attachment.prepareReplay(now);
for (FailedJobEntity failed : failedJobs.findByAttachmentIdAndReplayedAtIsNull(attachmentId)) {
failed.markReplayed(now);
}
enqueue(attachment, 1, now);
log.info("replay attachment={}", attachmentId);
return attachment;
});
}
public int reapExpired() {
int recovered = 0;
for (UUID id : attachments.findExpiredLeaseIds(Instant.now())) {
Boolean changed = transactions.execute(status -> recoverOne(id));
if (Boolean.TRUE.equals(changed)) {
recovered++;
}
}
return recovered;
}
private boolean recoverOne(UUID id) {
AttachmentEntity attachment = attachments.lockById(id).orElse(null);
Instant now = now();
if (attachment == null || !"processing".equals(attachment.getStatus()) || attachment.leaseHeld(now)) {
return false;
}
if (attachment.getAttempt() >= maxAttempts) {
markFailed(attachment, "retry_exhausted", now);
return true;
}
attachment.markQueued(now);
enqueue(attachment, attachment.getAttempt() + 1, now);
log.info("lease expired attachment={} attempt={}", id, attachment.getAttempt());
return true;
}
private Claim claim(JobPayload job, boolean redelivered) {
AttachmentEntity attachment = attachments.lockById(job.attachmentId()).orElse(null);
if (attachment == null || attachment.getPipelineVersion() > job.pipelineVersion()) {
return Claim.done(false);
}
if ("ready".equals(attachment.getStatus()) || "failed".equals(attachment.getStatus())) {
return Claim.done(false);
}
Instant now = now();
boolean ownedHere = attachment.getLeaseToken() != null && localLeases.contains(attachment.getLeaseToken());
if ("processing".equals(attachment.getStatus()) && attachment.leaseHeld(now) && ownedHere && !redelivered) {
return Claim.done(false);
}
int next = attachment.getAttempt() + 1;
if (next > maxAttempts) {
markFailed(attachment, "retry_exhausted", now);
return Claim.done(true);
}
if (attachment.getStagingKey() == null || attachment.getJobId() == null) {
markFailed(attachment, "processing_failed", now);
return Claim.done(true);
}
UUID token = UUID.randomUUID();
attachment.markProcessing(next, token, now.plus(timeout), now);
localLeases.add(token);
return new Claim(true, false, attachment.getId(), attachment.getJobId(), token, next, attachment.getStagingKey(),
attachment.getObjectKey(), attachment.getContentType(), attachment.getSizeBytes(), attachment.getSha256(),
attachment.getPipelineVersion());
}
private Processed process(Claim claim) {
long started = System.nanoTime();
byte[] original = readStaging(claim);
checkTime(started);
if (original.length != claim.sizeBytes() || !sha256(original).equals(claim.sha256())) {
throw new PermanentJobException("invalid_image");
}
try {
ImageFiles.inspect(original, claim.contentType(), maxPixels);
} catch (UnsupportedMediaException | PayloadTooLargeException exception) {
throw new PermanentJobException("invalid_image");
}
if (failPermanentRequested()) {
throw new PermanentJobException("processing_failed");
}
try {
checkTime(started);
storage.put(claim.objectKey(), original, claim.contentType());
var rendered = ImageVariants.render(original, jpegQuality, webpQuality);
checkTime(started);
for (ImageVariants.VariantBytes variant : rendered) {
storage.put(variantKey(claim.attachmentId(), variant.name()), variant.bytes(), variant.contentType());
}
if (crashAfterWriteRequested()) {
log.warn("fault injection: exit after object write before result commit");
System.exit(1);
}
return new Processed(rendered);
} catch (PermanentJobException | TransientJobException exception) {
throw exception;
} catch (UnavailableException exception) {
throw new TransientJobException(exception);
} catch (IOException exception) {
throw new PermanentJobException("invalid_image");
} catch (RuntimeException exception) {
log.warn("transient processing failure type={}", exception.getClass().getSimpleName());
throw new TransientJobException(exception);
}
}
private byte[] readStaging(Claim claim) {
try (InputStream input = storage.get(claim.stagingKey())) {
return ImageFiles.readLimited(input, maxBytes);
} catch (PayloadTooLargeException exception) {
throw new PermanentJobException("invalid_image");
} catch (NotFoundException exception) {
throw new PermanentJobException("processing_failed");
} catch (UnavailableException exception) {
throw new TransientJobException(exception);
} catch (IOException exception) {
throw new TransientJobException(exception);
}
}
private boolean finishReady(Claim claim, Processed processed) {
AttachmentEntity attachment = attachments.lockById(claim.attachmentId()).orElse(null);
if (attachment == null || lost(claim, attachment) || "ready".equals(attachment.getStatus())) {
return false;
}
variants.deleteForPipeline(attachment.getId(), attachment.getPipelineVersion());
for (ImageVariants.VariantBytes variant : processed.variants()) {
variants.save(new AttachmentVariantEntity(UUID.randomUUID(), attachment.getId(), attachment.getPipelineVersion(),
variant.name(), variant.contentType(), variant.bytes().length, variant.width(), variant.height(),
sha256(variant.bytes()), variantKey(attachment.getId(), variant.name())));
}
attachment.markReady(now());
log.info("attachment ready id={}", attachment.getId());
return true;
}
private boolean finishFailed(Claim claim, String code) {
AttachmentEntity attachment = attachments.lockById(claim.attachmentId()).orElse(null);
if (attachment == null || lost(claim, attachment) || "ready".equals(attachment.getStatus())
|| "failed".equals(attachment.getStatus())) {
return false;
}
markFailed(attachment, code, now());
return true;
}
private void publishFailed(String body) {
BrokerPublisher publisher = brokers.getIfAvailable();
if (publisher != null) {
publisher.failed(body);
}
}
private void finishRetry(Claim claim) {
AttachmentEntity attachment = attachments.lockById(claim.attachmentId()).orElse(null);
if (attachment == null || lost(claim, attachment) || "ready".equals(attachment.getStatus())
|| "failed".equals(attachment.getStatus())) {
return;
}
Instant now = now();
if (attachment.getAttempt() >= maxAttempts) {
markFailed(attachment, "retry_exhausted", now);
return;
}
attachment.markQueued(now);
long delay = RetryDelay.millis(attachment.getAttempt(), backoffBaseMs, backoffCapMs);
enqueue(attachment, attachment.getAttempt() + 1, now.plusMillis(delay));
}
private void markFailed(AttachmentEntity attachment, String code, Instant now) {
attachment.markFailed(code, now);
failedJobs.save(new FailedJobEntity(UUID.randomUUID(), attachment.getJobId(), attachment.getId(), code,
attachment.getAttempt(), now));
log.info("attachment failed id={} code={}", attachment.getId(), code);
}
private static boolean lost(Claim claim, AttachmentEntity attachment) {
return attachment.getLeaseToken() == null || !claim.leaseToken().equals(attachment.getLeaseToken());
}
private void checkTime(long started) {
if (System.nanoTime() - started > timeout.toNanos()) {
throw new TransientJobException("timeout");
}
}
private JobPayload parse(String body) {
try {
JobPayload job = json.readValue(body, JobPayload.class);
if (job == null || job.schemaVersion() != 1 || job.attachmentId() == null || job.jobId() == null
|| !"attachment.process.v1".equals(job.type()) || job.pipelineVersion() < 1 || job.attempt() < 1) {
throw new PoisonJobException();
}
return job;
} catch (PoisonJobException exception) {
throw exception;
} catch (RuntimeException exception) {
throw new PoisonJobException();
}
}
private static boolean crashAfterWriteRequested() {
try {
return Files.deleteIfExists(CRASH_AFTER_WRITE);
} catch (IOException exception) {
return false;
}
}
private static boolean failPermanentRequested() {
try {
return Files.deleteIfExists(FAIL_PERMANENT);
} catch (IOException exception) {
return false;
}
}
static String variantKey(UUID attachmentId, String name) {
return "variants/" + attachmentId + "/" + name;
}
private static String sha256(byte[] bytes) {
try {
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes));
} catch (NoSuchAlgorithmException exception) {
throw new IllegalStateException(exception);
}
}
private static Instant now() {
return Instant.now().truncatedTo(ChronoUnit.MICROS);
}
private record Claim(boolean process, boolean dlq, UUID attachmentId, UUID jobId, UUID leaseToken, int attempt,
String stagingKey, String objectKey, String contentType, long sizeBytes, String sha256,
int pipelineVersion) {
static Claim done(boolean dlq) {
return new Claim(false, dlq, null, null, null, 0, null, null, null, 0, null, 0);
}
}
private record Processed(java.util.List<ImageVariants.VariantBytes> variants) {
}
}
@@ -0,0 +1,35 @@
package ru.mtuci.messenger.job;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.support.Acknowledgment;
import org.springframework.stereotype.Component;
@Component
@ConditionalOnProperty(name = "app.worker.consume", havingValue = "true")
public class AttachmentWorker {
private static final Logger log = LoggerFactory.getLogger(AttachmentWorker.class);
private final AttachmentJobs jobs;
public AttachmentWorker(AttachmentJobs jobs) {
this.jobs = jobs;
}
@KafkaListener(topics = BrokerConfig.PROCESS, groupId = "attachment-worker")
public void onMessage(String body, Acknowledgment acknowledgment) {
try {
jobs.handle(body, false);
acknowledgment.acknowledge();
} catch (PoisonJobException exception) {
jobs.rejectPoison(body);
acknowledgment.acknowledge();
} catch (RuntimeException exception) {
log.warn("job not acked type={}", exception.getClass().getSimpleName());
throw exception;
}
}
}
@@ -0,0 +1,45 @@
package ru.mtuci.messenger.job;
import org.apache.kafka.clients.admin.NewTopic;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.EnableKafka;
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
import org.springframework.kafka.config.TopicBuilder;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.listener.ContainerProperties;
import org.springframework.kafka.listener.DefaultErrorHandler;
import org.springframework.util.backoff.FixedBackOff;
@Configuration
@EnableKafka
@ConditionalOnProperty(name = "app.broker.enabled", havingValue = "true")
public class BrokerConfig {
public static final String PROCESS = "attachments.process";
public static final String FAILED = "attachments.failed";
@Bean
NewTopic processTopic(@Value("${app.worker.concurrency:2}") int concurrency) {
return TopicBuilder.name(PROCESS).partitions(concurrency).replicas(1).build();
}
@Bean
NewTopic failedTopic() {
return TopicBuilder.name(FAILED).partitions(1).replicas(1).build();
}
@Bean
ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory(
ConsumerFactory<String, String> consumerFactory,
@Value("${app.worker.concurrency:2}") int concurrency) {
ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory);
factory.setConcurrency(concurrency);
factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL);
factory.setCommonErrorHandler(new DefaultErrorHandler(new FixedBackOff(1000L, FixedBackOff.UNLIMITED_ATTEMPTS)));
return factory;
}
}
@@ -0,0 +1,51 @@
package ru.mtuci.messenger.job;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.SendResult;
import org.springframework.stereotype.Component;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
@Component
@ConditionalOnProperty(name = "app.broker.enabled", havingValue = "true")
public class BrokerPublisher {
private static final Logger log = LoggerFactory.getLogger(BrokerPublisher.class);
private final KafkaTemplate<String, String> kafka;
public BrokerPublisher(KafkaTemplate<String, String> kafka) {
this.kafka = kafka;
}
public void confirm(String key, String payload) {
send(BrokerConfig.PROCESS, key, payload);
}
public void failed(String payload) {
try {
send(BrokerConfig.FAILED, null, payload);
} catch (RuntimeException exception) {
log.warn("dlq publish failed type={}", exception.getClass().getSimpleName());
}
}
private void send(String topic, String key, String payload) {
try {
SendResult<String, String> result = kafka.send(topic, key, payload).get(5, TimeUnit.SECONDS);
if (result == null || result.getRecordMetadata() == null) {
throw new IllegalStateException("broker nack");
}
} catch (InterruptedException exception) {
Thread.currentThread().interrupt();
throw new IllegalStateException("broker confirm interrupted");
} catch (TimeoutException | ExecutionException exception) {
throw new IllegalStateException("broker confirm failed");
}
}
}
@@ -0,0 +1,19 @@
package ru.mtuci.messenger.job;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;
import java.time.Instant;
import java.util.UUID;
@JsonIgnoreProperties(ignoreUnknown = true)
public record JobPayload(
@JsonProperty("schema_version") int schemaVersion,
@JsonProperty("job_id") UUID jobId,
@JsonProperty("type") String type,
@JsonProperty("attachment_id") UUID attachmentId,
@JsonProperty("pipeline_version") int pipelineVersion,
@JsonProperty("attempt") int attempt,
@JsonProperty("created_at") Instant createdAt
) {
}
@@ -0,0 +1,21 @@
package ru.mtuci.messenger.job;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
@Component
@ConditionalOnProperty(name = "app.worker.consume", havingValue = "true")
public class LeaseReaper {
private final AttachmentJobs jobs;
public LeaseReaper(AttachmentJobs jobs) {
this.jobs = jobs;
}
@Scheduled(fixedDelayString = "${app.job.reaper-ms:5000}")
public void reap() {
jobs.reapExpired();
}
}
@@ -0,0 +1,70 @@
package ru.mtuci.messenger.job;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
import ru.mtuci.messenger.domain.OutboxMessage;
import ru.mtuci.messenger.repository.OutboxRepository;
import ru.mtuci.messenger.service.DemoFlags;
import java.time.Instant;
import java.util.List;
import java.util.UUID;
@Component
@ConditionalOnProperty(name = "app.outbox.enabled", havingValue = "true", matchIfMissing = true)
public class OutboxPublisher {
private static final Logger log = LoggerFactory.getLogger(OutboxPublisher.class);
private final OutboxRepository outbox;
private final DemoFlags flags;
private final BrokerPublisher broker;
private final TransactionTemplate transactions;
public OutboxPublisher(OutboxRepository outbox, DemoFlags flags, BrokerPublisher broker,
PlatformTransactionManager transactionManager) {
this.outbox = outbox;
this.flags = flags;
this.broker = broker;
this.transactions = new TransactionTemplate(transactionManager);
}
@Scheduled(fixedDelayString = "${app.outbox.poll-ms:500}")
public void publishDue() {
if (flags.outboxPaused()) {
return;
}
for (int i = 0; i < 10; i++) {
if (flags.outboxPaused()) {
return;
}
Boolean sent = transactions.execute(status -> publishOne());
if (!Boolean.TRUE.equals(sent)) {
return;
}
}
}
private boolean publishOne() {
if (flags.outboxPaused()) {
return false;
}
List<UUID> ids = outbox.lockDueIds(Instant.now(), 1);
if (ids.isEmpty()) {
return false;
}
OutboxMessage row = outbox.findById(ids.getFirst()).orElseThrow();
if (row.getPublishedAt() != null) {
return false;
}
broker.confirm(row.getAttachmentId().toString(), row.getPayload());
row.setPublishedAt(Instant.now());
log.info("outbox published id={}", row.getId());
return true;
}
}
@@ -0,0 +1,15 @@
package ru.mtuci.messenger.job;
public class PermanentJobException extends RuntimeException {
private final String errorCode;
public PermanentJobException(String errorCode) {
super(errorCode);
this.errorCode = errorCode;
}
public String errorCode() {
return errorCode;
}
}
@@ -0,0 +1,8 @@
package ru.mtuci.messenger.job;
public class PoisonJobException extends RuntimeException {
public PoisonJobException() {
super("bad job");
}
}
@@ -0,0 +1,12 @@
package ru.mtuci.messenger.job;
public class TransientJobException extends RuntimeException {
public TransientJobException(String message) {
super(message);
}
public TransientJobException(Throwable cause) {
super(cause);
}
}
@@ -0,0 +1,20 @@
package ru.mtuci.messenger.mapper;
import org.mapstruct.Mapper;
import org.mapstruct.Mapping;
import org.mapstruct.ReportingPolicy;
import ru.mtuci.messenger.api.model.Chat;
import ru.mtuci.messenger.domain.ChatEntity;
import java.util.List;
import java.util.UUID;
@Mapper(componentModel = "spring", uses = TimeMapper.class, unmappedTargetPolicy = ReportingPolicy.ERROR)
public interface ChatMapper {
@Mapping(target = "id", source = "chat.id")
@Mapping(target = "title", source = "chat.title")
@Mapping(target = "createdAt", source = "chat.createdAt")
@Mapping(target = "memberIds", source = "memberIds")
Chat toModel(ChatEntity chat, List<UUID> memberIds);
}
@@ -0,0 +1,12 @@
package ru.mtuci.messenger.mapper;
import org.mapstruct.Mapper;
import org.mapstruct.ReportingPolicy;
import ru.mtuci.messenger.api.model.Message;
import ru.mtuci.messenger.domain.MessageEntity;
@Mapper(componentModel = "spring", uses = TimeMapper.class, unmappedTargetPolicy = ReportingPolicy.ERROR)
public interface MessageMapper {
Message toModel(MessageEntity message);
}
@@ -0,0 +1,15 @@
package ru.mtuci.messenger.mapper;
import org.mapstruct.Mapper;
import java.time.Instant;
import java.time.OffsetDateTime;
import java.time.ZoneOffset;
@Mapper(componentModel = "spring")
public interface TimeMapper {
default OffsetDateTime map(Instant instant) {
return instant == null ? null : instant.atOffset(ZoneOffset.UTC);
}
}
@@ -0,0 +1,19 @@
package ru.mtuci.messenger.mapper;
import org.mapstruct.Mapper;
import org.mapstruct.Mapping;
import org.mapstruct.ReportingPolicy;
import ru.mtuci.messenger.api.model.Session;
import ru.mtuci.messenger.api.model.User;
import ru.mtuci.messenger.domain.UserAccount;
import java.time.Instant;
@Mapper(componentModel = "spring", uses = TimeMapper.class, unmappedTargetPolicy = ReportingPolicy.ERROR)
public interface UserMapper {
User toModel(UserAccount user);
@Mapping(target = "tokenType", expression = "java(ru.mtuci.messenger.api.model.Session.TokenTypeEnum.BEARER)")
Session toSession(String token, Instant expiresAt, UserAccount user);
}
@@ -0,0 +1,34 @@
package ru.mtuci.messenger.migrate;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Component;
@Component
@Profile("migrate")
public class MigrateExit implements ApplicationRunner {
private final ConfigurableApplicationContext context;
public MigrateExit(ConfigurableApplicationContext context) {
this.context = context;
}
@Override
public void run(ApplicationArguments args) {
Thread exit = new Thread(() -> {
try {
Thread.sleep(500);
} catch (InterruptedException interrupted) {
Thread.currentThread().interrupt();
}
int code = SpringApplication.exit(context, () -> 0);
System.exit(code);
}, "migrate-exit");
exit.setDaemon(false);
exit.start();
}
}
@@ -0,0 +1,31 @@
package ru.mtuci.messenger.repository;
import jakarta.persistence.LockModeType;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Lock;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import ru.mtuci.messenger.domain.AttachmentEntity;
import java.time.Instant;
import java.util.Collection;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
public interface AttachmentRepository extends JpaRepository<AttachmentEntity, UUID> {
boolean existsByObjectKey(String objectKey);
boolean existsByStagingKeyAndStatusIn(String stagingKey, Collection<String> statuses);
@Lock(LockModeType.PESSIMISTIC_WRITE)
@Query("select a from AttachmentEntity a where a.id = :id")
Optional<AttachmentEntity> lockById(@Param("id") UUID id);
@Query("""
select a.id from AttachmentEntity a
where a.status = 'processing' and a.leaseUntil is not null and a.leaseUntil < :now
""")
List<UUID> findExpiredLeaseIds(@Param("now") Instant now);
}
@@ -0,0 +1,28 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import ru.mtuci.messenger.domain.AttachmentVariantEntity;
import java.util.List;
import java.util.Optional;
import java.util.UUID;
public interface AttachmentVariantRepository extends JpaRepository<AttachmentVariantEntity, UUID> {
List<AttachmentVariantEntity> findByAttachmentIdAndPipelineVersion(UUID attachmentId, int pipelineVersion);
Optional<AttachmentVariantEntity> findByAttachmentIdAndPipelineVersionAndName(UUID attachmentId, int pipelineVersion,
String name);
boolean existsByObjectKey(String objectKey);
@Modifying(flushAutomatically = true)
@Query("""
delete from AttachmentVariantEntity v
where v.attachmentId = :attachmentId and v.pipelineVersion = :pipelineVersion
""")
void deleteForPipeline(@Param("attachmentId") UUID attachmentId, @Param("pipelineVersion") int pipelineVersion);
}
@@ -0,0 +1,18 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import ru.mtuci.messenger.domain.ChatMemberEntity;
import ru.mtuci.messenger.domain.ChatMemberId;
import java.util.Collection;
import java.util.List;
import java.util.UUID;
public interface ChatMemberRepository extends JpaRepository<ChatMemberEntity, ChatMemberId> {
boolean existsByChatIdAndUserId(UUID chatId, UUID userId);
List<ChatMemberEntity> findByChatIdOrderByOrdinalAsc(UUID chatId);
List<ChatMemberEntity> findByChatIdInOrderByOrdinalAsc(Collection<UUID> chatIds);
}
@@ -0,0 +1,30 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import ru.mtuci.messenger.domain.ChatEntity;
import java.time.Instant;
import java.util.List;
import java.util.UUID;
public interface ChatRepository extends JpaRepository<ChatEntity, UUID> {
@Query("""
select c from ChatEntity c
where c.id in (select m.chatId from ChatMemberEntity m where m.userId = :userId)
order by c.createdAt asc, c.id asc
""")
List<ChatEntity> findFirst(UUID userId, Pageable pageable);
@Query(value = """
select c.* from chats c
where c.id in (select m.chat_id from chat_members m where m.user_id = :userId)
and (c.created_at, c.id) > (:createdAt, :id)
order by c.created_at asc, c.id asc
""", nativeQuery = true)
List<ChatEntity> findAfter(UUID userId, Instant createdAt, UUID id, Pageable pageable);
long countByTitle(String title);
}
@@ -0,0 +1,7 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import ru.mtuci.messenger.domain.DemoFlag;
public interface DemoFlagRepository extends JpaRepository<DemoFlag, String> {
}
@@ -0,0 +1,14 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import ru.mtuci.messenger.domain.FailedJobEntity;
import java.util.List;
import java.util.UUID;
public interface FailedJobRepository extends JpaRepository<FailedJobEntity, UUID> {
long countByReplayedAtIsNull();
List<FailedJobEntity> findByAttachmentIdAndReplayedAtIsNull(UUID attachmentId);
}
@@ -0,0 +1,13 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import ru.mtuci.messenger.domain.IdempotencyRecord;
import java.util.Optional;
import java.util.UUID;
public interface IdempotencyRepository extends JpaRepository<IdempotencyRecord, UUID> {
Optional<IdempotencyRecord> findByUserIdAndHttpMethodAndPathAndIdempotencyKey(
UUID userId, String httpMethod, String path, String idempotencyKey);
}
@@ -0,0 +1,31 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import ru.mtuci.messenger.domain.MessageEntity;
import java.time.Instant;
import java.util.List;
import java.util.UUID;
public interface MessageRepository extends JpaRepository<MessageEntity, UUID> {
long countByChatId(UUID chatId);
@Query("""
select m from MessageEntity m
where m.chatId = :chatId
order by m.createdAt asc, m.id asc
""")
List<MessageEntity> findFirst(UUID chatId, Pageable pageable);
@Query(value = """
select * from messages
where chat_id = :chatId
and (created_at, id) > (:createdAt, :id)
order by created_at asc, id asc
""", nativeQuery = true)
List<MessageEntity> findAfter(UUID chatId, Instant createdAt, UUID id, Pageable pageable);
}
@@ -0,0 +1,24 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import ru.mtuci.messenger.domain.OutboxMessage;
import java.time.Instant;
import java.util.List;
import java.util.UUID;
public interface OutboxRepository extends JpaRepository<OutboxMessage, UUID> {
long countByPublishedAtIsNull();
@Query(value = """
select id from job_outbox
where published_at is null and available_at <= :now
order by created_at
limit :limit
for update skip locked
""", nativeQuery = true)
List<UUID> lockDueIds(@Param("now") Instant now, @Param("limit") int limit);
}
@@ -0,0 +1,28 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.transaction.annotation.Transactional;
import ru.mtuci.messenger.domain.SessionEntity;
import java.time.Instant;
import java.util.Optional;
import java.util.UUID;
public interface SessionRepository extends JpaRepository<SessionEntity, UUID> {
@Query("""
select s from SessionEntity s
where s.tokenHash = :tokenHash and s.revokedAt is null and s.expiresAt > :now
""")
Optional<SessionEntity> findActive(String tokenHash, Instant now);
@Transactional
@Modifying
@Query("""
update SessionEntity s set s.revokedAt = :now
where s.tokenHash = :tokenHash and s.revokedAt is null and s.expiresAt > :now
""")
int revoke(String tokenHash, Instant now);
}
@@ -0,0 +1,15 @@
package ru.mtuci.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import ru.mtuci.messenger.domain.UserAccount;
import java.util.Collection;
import java.util.Optional;
import java.util.UUID;
public interface UserRepository extends JpaRepository<UserAccount, UUID> {
Optional<UserAccount> findByUsername(String username);
long countByIdIn(Collection<UUID> ids);
}
@@ -0,0 +1,230 @@
package ru.mtuci.messenger.service;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.InvalidMediaTypeException;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Service;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.transaction.support.TransactionTemplate;
import ru.mtuci.messenger.api.model.Attachment;
import ru.mtuci.messenger.api.model.Variant;
import ru.mtuci.messenger.domain.AttachmentEntity;
import ru.mtuci.messenger.domain.AttachmentVariantEntity;
import ru.mtuci.messenger.exception.ConflictException;
import ru.mtuci.messenger.exception.InvalidRequestException;
import ru.mtuci.messenger.exception.NotFoundException;
import ru.mtuci.messenger.exception.PayloadTooLargeException;
import ru.mtuci.messenger.exception.UnsupportedMediaException;
import ru.mtuci.messenger.job.AttachmentJobs;
import ru.mtuci.messenger.repository.AttachmentRepository;
import ru.mtuci.messenger.repository.AttachmentVariantRepository;
import ru.mtuci.messenger.repository.ChatMemberRepository;
import ru.mtuci.messenger.storage.ObjectStorage;
import java.io.InputStream;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HexFormat;
import java.util.List;
import java.util.Set;
import java.util.UUID;
@Service
public class AttachmentService {
private static final List<String> VARIANT_ORDER = List.of("thumbnail", "crop", "optimized");
private static final Set<String> VARIANTS = Set.copyOf(VARIANT_ORDER);
private final AttachmentRepository attachments;
private final AttachmentVariantRepository variants;
private final ChatMemberRepository members;
private final ObjectStorage storage;
private final AttachmentJobs jobs;
private final TransactionTemplate transactions;
private final int maxBytes;
private final long maxPixels;
public AttachmentService(AttachmentRepository attachments, AttachmentVariantRepository variants,
ChatMemberRepository members, ObjectStorage storage, AttachmentJobs jobs,
PlatformTransactionManager transactionManager,
@Value("${app.upload.max-bytes:10485760}") int maxBytes,
@Value("${app.upload.max-pixels:40000000}") long maxPixels) {
this.attachments = attachments;
this.variants = variants;
this.members = members;
this.storage = storage;
this.jobs = jobs;
this.transactions = new TransactionTemplate(transactionManager);
this.maxBytes = maxBytes;
this.maxPixels = maxPixels;
}
public Attachment upload(UUID actorId, UUID chatId, String filename, String contentType, long contentLength,
InputStream body) {
requireMember(chatId, actorId);
String displayName = filename(filename);
String declared = declaredType(contentType);
if (contentLength > maxBytes) {
throw new PayloadTooLargeException();
}
byte[] bytes;
try {
bytes = ImageFiles.readLimited(body, maxBytes);
} catch (PayloadTooLargeException tooLarge) {
throw tooLarge;
} catch (java.io.IOException exception) {
throw new ru.mtuci.messenger.exception.UnavailableException("Хранилище временно недоступно");
}
ImageFiles.Image image = ImageFiles.inspect(bytes, declared, maxPixels);
UUID id = UUID.randomUUID();
String stagingKey = "staging/" + id;
String objectKey = "originals/" + id;
storage.ensureBucket();
storage.put(stagingKey, bytes, image.contentType());
try {
AttachmentEntity saved = transactions.execute(status -> {
AttachmentEntity entity = attachments.saveAndFlush(new AttachmentEntity(
id, chatId, actorId, displayName, image.contentType(), bytes.length, sha256(bytes),
objectKey, stagingKey, UUID.randomUUID()));
jobs.enqueue(entity, 1, Instant.now());
return entity;
});
return toModel(saved);
} catch (RuntimeException exception) {
storage.deleteQuietly(stagingKey);
throw exception;
}
}
@Transactional(readOnly = true)
public Attachment get(UUID actorId, UUID attachmentId) {
return toModel(visible(actorId, attachmentId));
}
public Attachment replay(UUID attachmentId) {
return toModel(jobs.replay(attachmentId));
}
@Transactional(readOnly = true)
public Download content(UUID actorId, UUID attachmentId, String variant) {
String name = variant == null || variant.isBlank() ? "original" : variant;
if (!"original".equals(name) && !VARIANTS.contains(name)) {
throw new InvalidRequestException("Некорректный запрос");
}
AttachmentEntity attachment = visible(actorId, attachmentId);
if (!"ready".equals(attachment.getStatus())) {
throw new ConflictException("Вложение ещё не готово");
}
if ("original".equals(name)) {
return new Download(storage.get(attachment.getObjectKey()), attachment.getContentType(), attachment.getSizeBytes());
}
AttachmentVariantEntity row = variants.findByAttachmentIdAndPipelineVersionAndName(
attachment.getId(), attachment.getPipelineVersion(), name).orElseThrow(NotFoundException::new);
return new Download(storage.get(row.getObjectKey()), row.getContentType(), row.getSizeBytes());
}
private AttachmentEntity visible(UUID actorId, UUID attachmentId) {
AttachmentEntity attachment = attachments.findById(attachmentId).orElseThrow(NotFoundException::new);
if (!members.existsByChatIdAndUserId(attachment.getChatId(), actorId)) {
throw new NotFoundException();
}
return attachment;
}
private void requireMember(UUID chatId, UUID actorId) {
if (!members.existsByChatIdAndUserId(chatId, actorId)) {
throw new NotFoundException();
}
}
private static String filename(String value) {
if (value == null) {
throw new InvalidRequestException("Некорректный запрос");
}
int codePoints = value.codePointCount(0, value.length());
if (codePoints < 1 || codePoints > 255) {
throw new InvalidRequestException("Некорректный запрос");
}
int offset = 0;
while (offset < value.length()) {
int codePoint = value.codePointAt(offset);
if (codePoint == '/' || codePoint == '\\' || Character.isISOControl(codePoint)) {
throw new InvalidRequestException("Некорректный запрос");
}
offset += Character.charCount(codePoint);
}
return value;
}
private static String declaredType(String header) {
if (header == null || header.isBlank()) {
throw new UnsupportedMediaException();
}
MediaType media;
try {
media = MediaType.parseMediaType(header);
} catch (InvalidMediaTypeException exception) {
throw new UnsupportedMediaException();
}
if (media.equalsTypeAndSubtype(MediaType.IMAGE_PNG)) {
return "image/png";
}
if (media.equalsTypeAndSubtype(MediaType.parseMediaType("image/jpeg"))) {
return "image/jpeg";
}
throw new UnsupportedMediaException();
}
private static String sha256(byte[] bytes) {
try {
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes));
} catch (NoSuchAlgorithmException exception) {
throw new IllegalStateException(exception);
}
}
private Attachment toModel(AttachmentEntity entity) {
Attachment model = new Attachment();
model.setId(entity.getId());
model.setChatId(entity.getChatId());
model.setOwnerId(entity.getOwnerId());
model.setFilename(entity.getFilename());
model.setContentType(Attachment.ContentTypeEnum.fromValue(entity.getContentType()));
model.setSizeBytes((int) entity.getSizeBytes());
model.setSha256(entity.getSha256());
model.setStatus(Attachment.StatusEnum.fromValue(entity.getStatus()));
model.setVariants("ready".equals(entity.getStatus()) ? variantModels(entity) : new ArrayList<>());
model.setErrorCode(entity.getErrorCode());
model.setCreatedAt(entity.getCreatedAt().atOffset(java.time.ZoneOffset.UTC));
model.setUpdatedAt(entity.getUpdatedAt().atOffset(java.time.ZoneOffset.UTC));
return model;
}
private List<Variant> variantModels(AttachmentEntity entity) {
List<AttachmentVariantEntity> rows = new ArrayList<>(variants.findByAttachmentIdAndPipelineVersion(
entity.getId(), entity.getPipelineVersion()));
rows.sort(Comparator.comparingInt(row -> {
int index = VARIANT_ORDER.indexOf(row.getName());
return index < 0 ? VARIANT_ORDER.size() : index;
}));
List<Variant> models = new ArrayList<>();
for (AttachmentVariantEntity row : rows) {
Variant variant = new Variant();
variant.setName(Variant.NameEnum.fromValue(row.getName()));
variant.setContentType(Variant.ContentTypeEnum.fromValue(row.getContentType()));
variant.setSizeBytes((int) row.getSizeBytes());
variant.setWidth(row.getWidth());
variant.setHeight(row.getHeight());
variant.setSha256(row.getSha256());
models.add(variant);
}
return models;
}
public record Download(InputStream body, String contentType, long size) {
}
}
@@ -0,0 +1,99 @@
package ru.mtuci.messenger.service;
import org.springframework.data.domain.PageRequest;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import ru.mtuci.messenger.api.model.Chat;
import ru.mtuci.messenger.api.model.ChatPage;
import ru.mtuci.messenger.api.model.CreateChat;
import ru.mtuci.messenger.domain.ChatEntity;
import ru.mtuci.messenger.domain.ChatMemberEntity;
import ru.mtuci.messenger.exception.InvalidRequestException;
import ru.mtuci.messenger.exception.NotFoundException;
import ru.mtuci.messenger.mapper.ChatMapper;
import ru.mtuci.messenger.repository.ChatMemberRepository;
import ru.mtuci.messenger.repository.ChatRepository;
import ru.mtuci.messenger.repository.UserRepository;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.UUID;
@Service
public class ChatService {
private final ChatRepository chats;
private final ChatMemberRepository members;
private final UserRepository users;
private final ChatMapper mapper;
public ChatService(ChatRepository chats, ChatMemberRepository members, UserRepository users, ChatMapper mapper) {
this.chats = chats;
this.members = members;
this.users = users;
this.mapper = mapper;
}
@Transactional
public Chat create(UUID actorId, CreateChat request) {
TextRules.title(request.getTitle());
List<UUID> memberIds = request.getMemberIds();
if (memberIds == null || new HashSet<>(memberIds).size() != memberIds.size() || !memberIds.contains(actorId)) {
throw new InvalidRequestException("Некорректный запрос");
}
ChatEntity chat = chats.save(new ChatEntity(request.getTitle()));
if (users.countByIdIn(memberIds) != memberIds.size()) {
throw new InvalidRequestException("Некорректный запрос");
}
for (int index = 0; index < memberIds.size(); index++) {
members.save(new ChatMemberEntity(chat.getId(), memberIds.get(index), index));
}
return mapper.toModel(chat, memberIds);
}
@Transactional(readOnly = true)
public ChatPage list(UUID actorId, Integer limit, String cursor) {
int pageSize = limit == null ? 50 : limit;
CursorCodec.Position position = cursor == null ? null : CursorCodec.decode(cursor, null);
List<ChatEntity> rows = position == null
? chats.findFirst(actorId, PageRequest.of(0, pageSize + 1))
: chats.findAfter(actorId, position.createdAt(), position.id(), PageRequest.of(0, pageSize + 1));
boolean more = rows.size() > pageSize;
List<ChatEntity> visible = more ? rows.subList(0, pageSize) : rows;
Map<UUID, List<UUID>> byChat = memberIds(visible);
List<Chat> items = visible.stream()
.map(chat -> mapper.toModel(chat, byChat.getOrDefault(chat.getId(), List.of())))
.toList();
String next = null;
if (more) {
ChatEntity last = visible.get(visible.size() - 1);
next = CursorCodec.encode(null, last.getCreatedAt(), last.getId());
}
return new ChatPage(items, next);
}
@Transactional(readOnly = true)
public Chat get(UUID actorId, UUID chatId) {
if (!members.existsByChatIdAndUserId(chatId, actorId)) {
throw new NotFoundException();
}
ChatEntity chat = chats.findById(chatId).orElseThrow(NotFoundException::new);
List<UUID> memberIds = members.findByChatIdOrderByOrdinalAsc(chatId).stream().map(ChatMemberEntity::getUserId).toList();
return mapper.toModel(chat, memberIds);
}
private Map<UUID, List<UUID>> memberIds(List<ChatEntity> chats) {
Map<UUID, List<UUID>> result = new LinkedHashMap<>();
if (chats.isEmpty()) {
return result;
}
List<UUID> ids = chats.stream().map(ChatEntity::getId).toList();
for (ChatMemberEntity member : members.findByChatIdInOrderByOrdinalAsc(ids)) {
result.computeIfAbsent(member.getChatId(), ignored -> new ArrayList<>()).add(member.getUserId());
}
return result;
}
}
@@ -0,0 +1,48 @@
package ru.mtuci.messenger.service;
import ru.mtuci.messenger.exception.InvalidRequestException;
import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.util.Base64;
import java.util.UUID;
/**
* Непрозрачная позиция keyset-страницы: клиент получает её как next_cursor
* и возвращает в ?cursor=, чтобы следующая страница началась после (created_at, id).
* Для истории в курсор входит chatId: курсор другого чата отклоняется.
*/
public final class CursorCodec {
private CursorCodec() {
}
public static String encode(UUID scope, Instant createdAt, UUID id) {
String raw = scope == null ? createdAt + "|" + id : scope + "|" + createdAt + "|" + id;
return Base64.getUrlEncoder().withoutPadding().encodeToString(raw.getBytes(StandardCharsets.UTF_8));
}
public static Position decode(String cursor, UUID scope) {
String raw;
try {
raw = new String(Base64.getUrlDecoder().decode(cursor), StandardCharsets.UTF_8);
} catch (IllegalArgumentException exception) {
throw new InvalidRequestException("Некорректный курсор");
}
String[] parts = raw.split("\\|", -1);
try {
if (scope == null && parts.length == 2) {
return new Position(Instant.parse(parts[0]), UUID.fromString(parts[1]));
}
if (scope != null && parts.length == 3 && scope.equals(UUID.fromString(parts[0]))) {
return new Position(Instant.parse(parts[1]), UUID.fromString(parts[2]));
}
} catch (RuntimeException exception) {
throw new InvalidRequestException("Некорректный курсор");
}
throw new InvalidRequestException("Некорректный курсор");
}
public record Position(Instant createdAt, UUID id) {
}
}
@@ -0,0 +1,31 @@
package ru.mtuci.messenger.service;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import ru.mtuci.messenger.domain.DemoFlag;
import ru.mtuci.messenger.repository.DemoFlagRepository;
@Service
public class DemoFlags {
public static final String OUTBOX_PAUSED = "outbox_paused";
private final DemoFlagRepository flags;
public DemoFlags(DemoFlagRepository flags) {
this.flags = flags;
}
@Transactional(readOnly = true)
public boolean outboxPaused() {
return flags.findById(OUTBOX_PAUSED).map(DemoFlag::isEnabled).orElse(false);
}
@Transactional
public boolean setOutboxPaused(boolean paused) {
DemoFlag flag = flags.findById(OUTBOX_PAUSED).orElseGet(() -> new DemoFlag(OUTBOX_PAUSED, paused));
flag.setEnabled(paused);
flags.save(flag);
return paused;
}
}
@@ -0,0 +1,171 @@
package ru.mtuci.messenger.service;
import ru.mtuci.messenger.exception.PayloadTooLargeException;
import ru.mtuci.messenger.exception.UnsupportedMediaException;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
public final class ImageFiles {
private static final byte[] PNG_SIGNATURE = new byte[] {(byte) 137, 80, 78, 71, 13, 10, 26, 10};
private ImageFiles() {
}
public record Image(String contentType, long width, long height) {
}
public static byte[] readLimited(InputStream input, int maxBytes) throws IOException {
byte[] buffer = new byte[8192];
ByteArrayOutputStream body = new ByteArrayOutputStream(Math.min(maxBytes, 64 * 1024));
int total = 0;
int read;
while ((read = input.read(buffer)) >= 0) {
if (read == 0) {
continue;
}
if (total > maxBytes - read) {
throw new PayloadTooLargeException();
}
body.write(buffer, 0, read);
total += read;
}
return body.toByteArray();
}
public static Image inspect(byte[] data, String declaredType, long maxPixels) {
if (data.length == 0) {
throw new UnsupportedMediaException();
}
Image image = "image/png".equals(declaredType) ? png(data) : jpeg(data);
if (image == null || !declaredType.equals(image.contentType()) || image.width() <= 0 || image.height() <= 0) {
throw new UnsupportedMediaException();
}
if (image.width() > maxPixels || image.height() > maxPixels || image.width() * image.height() > maxPixels) {
throw new PayloadTooLargeException();
}
return image;
}
private static Image png(byte[] data) {
if (data.length < 33 || !startsWith(data, PNG_SIGNATURE)) {
return null;
}
int offset = 8;
Long width = null;
Long height = null;
boolean end = false;
while (offset + 12 <= data.length) {
long length = u32(data, offset);
if (length > data.length || offset + 12L + length > data.length) {
return null;
}
String type = new String(data, offset + 4, 4, StandardCharsets.US_ASCII);
if (!type.chars().allMatch(Character::isLetter)) {
return null;
}
if ("IHDR".equals(type)) {
if (length != 13 || width != null || !validPngHeader(data, offset + 8)) {
return null;
}
width = u32(data, offset + 8);
height = u32(data, offset + 12);
}
offset += 12 + (int) length;
if ("IEND".equals(type)) {
end = length == 0;
break;
}
}
if (!end || width == null || offset != data.length) {
return null;
}
return new Image("image/png", width, height);
}
private static boolean validPngHeader(byte[] data, int offset) {
int bitDepth = data[offset + 8] & 0xff;
int color = data[offset + 9] & 0xff;
int compression = data[offset + 10] & 0xff;
int filter = data[offset + 11] & 0xff;
int interlace = data[offset + 12] & 0xff;
if (compression != 0 || filter != 0 || (interlace != 0 && interlace != 1)) {
return false;
}
return switch (color) {
case 0 -> bitDepth == 1 || bitDepth == 2 || bitDepth == 4 || bitDepth == 8 || bitDepth == 16;
case 3 -> bitDepth == 1 || bitDepth == 2 || bitDepth == 4 || bitDepth == 8;
case 2, 4, 6 -> bitDepth == 8 || bitDepth == 16;
default -> false;
};
}
private static Image jpeg(byte[] data) {
if (data.length < 4 || (data[0] & 0xff) != 0xff || (data[1] & 0xff) != 0xd8) {
return null;
}
int offset = 2;
Long width = null;
Long height = null;
while (offset + 1 < data.length) {
if ((data[offset] & 0xff) != 0xff) {
return null;
}
while (offset < data.length && (data[offset] & 0xff) == 0xff) {
offset++;
}
if (offset >= data.length) {
return null;
}
int marker = data[offset++] & 0xff;
if (marker == 0xd9) {
return width == null ? null : new Image("image/jpeg", width, height);
}
if (marker == 0xd8 || marker == 0x01 || (marker >= 0xd0 && marker <= 0xd7)) {
continue;
}
if (offset + 1 >= data.length) {
return null;
}
int length = ((data[offset] & 0xff) << 8) | (data[offset + 1] & 0xff);
if (length < 2 || offset + length > data.length) {
return null;
}
if (isStartOfFrame(marker)) {
if (length < 7) {
return null;
}
height = (long) ((data[offset + 3] & 0xff) << 8) | (data[offset + 4] & 0xff);
width = (long) ((data[offset + 5] & 0xff) << 8) | (data[offset + 6] & 0xff);
}
offset += length;
}
return null;
}
private static boolean isStartOfFrame(int marker) {
return marker >= 0xc0 && marker <= 0xcf && marker != 0xc4 && marker != 0xc8 && marker != 0xcc;
}
private static boolean startsWith(byte[] data, byte[] prefix) {
if (data.length < prefix.length) {
return false;
}
for (int i = 0; i < prefix.length; i++) {
if (data[i] != prefix[i]) {
return false;
}
}
return true;
}
private static long u32(byte[] data, int offset) {
return ((data[offset] & 0xffL) << 24)
| ((data[offset + 1] & 0xffL) << 16)
| ((data[offset + 2] & 0xffL) << 8)
| (data[offset + 3] & 0xffL);
}
}
@@ -0,0 +1,118 @@
package ru.mtuci.messenger.service;
import com.luciad.imageio.webp.CompressionType;
import com.luciad.imageio.webp.WebPWriteParam;
import net.coobird.thumbnailator.Thumbnails;
import javax.imageio.IIOImage;
import javax.imageio.ImageIO;
import javax.imageio.ImageWriteParam;
import javax.imageio.ImageWriter;
import javax.imageio.stream.ImageOutputStream;
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.RenderingHints;
import java.awt.image.BufferedImage;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.List;
import java.util.Locale;
/**
* Derivatives for pipeline version 1.
* Thumbnail and crop are baseline JPEG at the given quality, metadata is not copied.
* EXIF orientation is applied before resize. Alpha is composited onto white.
* Optimized is lossy WebP inside 1280×1280 without upscaling; quality is a 0..1 factor
* (0.8 means libwebp quality 80). Small inputs are not promised to shrink.
*/
public final class ImageVariants {
public record VariantBytes(String name, String contentType, int width, int height, byte[] bytes) {
}
static {
ImageIO.scanForPlugins();
}
private ImageVariants() {
}
public static int[] fitted(int width, int height, int box, boolean upscale) {
double scale = Math.min(box / (double) width, box / (double) height);
if (!upscale) {
scale = Math.min(1.0, scale);
}
return new int[] {
Math.max(1, (int) Math.floor(width * scale)),
Math.max(1, (int) Math.floor(height * scale))
};
}
public static List<VariantBytes> render(byte[] original, float jpegQuality, float webpQuality) throws IOException {
BufferedImage oriented = Thumbnails.of(new ByteArrayInputStream(original))
.scale(1.0)
.useExifOrientation(true)
.asBufferedImage();
int width = oriented.getWidth();
int height = oriented.getHeight();
int[] thumb = fitted(width, height, 256, false);
int[] optimized = fitted(width, height, 1280, false);
VariantBytes thumbnail = new VariantBytes("thumbnail", "image/jpeg", thumb[0], thumb[1],
jpeg(scale(oriented, thumb[0], thumb[1]), jpegQuality));
VariantBytes crop = new VariantBytes("crop", "image/jpeg", 128, 128,
jpeg(cropSquare(oriented, 128), jpegQuality));
VariantBytes webp = new VariantBytes("optimized", "image/webp", optimized[0], optimized[1],
webp(scale(oriented, optimized[0], optimized[1]), webpQuality));
return List.of(thumbnail, crop, webp);
}
private static BufferedImage cropSquare(BufferedImage source, int size) {
int side = Math.min(source.getWidth(), source.getHeight());
int x = (source.getWidth() - side) / 2;
int y = (source.getHeight() - side) / 2;
return scale(source.getSubimage(x, y, side, side), size, size);
}
private static BufferedImage scale(BufferedImage source, int width, int height) {
BufferedImage target = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
Graphics2D graphics = target.createGraphics();
try {
graphics.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
graphics.setColor(Color.WHITE);
graphics.fillRect(0, 0, width, height);
graphics.drawImage(source, 0, 0, width, height, null);
} finally {
graphics.dispose();
}
return target;
}
private static byte[] jpeg(BufferedImage image, float quality) throws IOException {
ImageWriter writer = ImageIO.getImageWritersByFormatName("jpeg").next();
ImageWriteParam param = writer.getDefaultWriteParam();
param.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
param.setCompressionQuality(quality);
return write(writer, param, image);
}
private static byte[] webp(BufferedImage image, float quality) throws IOException {
ImageWriter writer = ImageIO.getImageWritersByMIMEType("image/webp").next();
WebPWriteParam param = new WebPWriteParam(Locale.ROOT);
param.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
param.setCompressionType(CompressionType.Lossy);
param.setCompressionQuality(quality);
return write(writer, param, image);
}
private static byte[] write(ImageWriter writer, ImageWriteParam param, BufferedImage image) throws IOException {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try (ImageOutputStream out = ImageIO.createImageOutputStream(bytes)) {
writer.setOutput(out);
writer.write(null, new IIOImage(image, null, null), param);
} finally {
writer.dispose();
}
return bytes.toByteArray();
}
}
@@ -0,0 +1,202 @@
package ru.mtuci.messenger.service;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import ru.mtuci.messenger.api.model.CreateMessage;
import ru.mtuci.messenger.api.model.Message;
import ru.mtuci.messenger.api.model.MessagePage;
import ru.mtuci.messenger.domain.AttachmentEntity;
import ru.mtuci.messenger.domain.IdempotencyRecord;
import ru.mtuci.messenger.domain.MessageEntity;
import ru.mtuci.messenger.exception.ConflictException;
import ru.mtuci.messenger.exception.InvalidRequestException;
import ru.mtuci.messenger.exception.NotFoundException;
import ru.mtuci.messenger.mapper.MessageMapper;
import ru.mtuci.messenger.repository.AttachmentRepository;
import ru.mtuci.messenger.repository.ChatMemberRepository;
import ru.mtuci.messenger.repository.IdempotencyRepository;
import ru.mtuci.messenger.repository.MessageRepository;
import org.springframework.data.domain.PageRequest;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.sql.Connection;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.HexFormat;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import java.util.UUID;
import java.util.regex.Pattern;
@Service
public class MessageService {
private static final Pattern IDEMPOTENCY_KEY = Pattern.compile("^[A-Za-z0-9._:-]{1,128}$");
private static final String METHOD = "POST";
private final MessageRepository messages;
private final AttachmentRepository attachments;
private final IdempotencyRepository idempotency;
private final ChatMemberRepository members;
private final MessageMapper mapper;
private final JdbcTemplate jdbc;
private final Duration idempotencyTtl;
public MessageService(MessageRepository messages, AttachmentRepository attachments, IdempotencyRepository idempotency,
ChatMemberRepository members, MessageMapper mapper, JdbcTemplate jdbc,
@Value("${app.idempotency-ttl:PT24H}") Duration idempotencyTtl) {
this.messages = messages;
this.attachments = attachments;
this.idempotency = idempotency;
this.members = members;
this.mapper = mapper;
this.jdbc = jdbc;
this.idempotencyTtl = idempotencyTtl;
}
@Transactional
public Message create(UUID actorId, UUID chatId, String idempotencyKey, CreateMessage request) {
requireMember(chatId, actorId);
String key = requireKey(idempotencyKey);
List<UUID> attachmentIds = attachmentIds(request);
requireText(request.getText(), !attachmentIds.isEmpty());
requireAttachments(chatId, attachmentIds);
String path = "/api/v1/chats/" + chatId + "/messages";
lock(actorId + "\n" + METHOD + "\n" + path + "\n" + key);
String hash = requestHash(request.getText(), attachmentIds);
Optional<IdempotencyRecord> existing = idempotency.findByUserIdAndHttpMethodAndPathAndIdempotencyKey(
actorId, METHOD, path, key);
if (existing.isPresent()) {
IdempotencyRecord record = existing.get();
if (record.getExpiresAt().isAfter(Instant.now())) {
if (!record.getRequestHash().equals(hash)) {
throw new ConflictException("Ключ идемпотентности уже использован с другим телом");
}
return mapper.toModel(messages.findById(record.getMessageId()).orElseThrow(NotFoundException::new));
}
idempotency.delete(record);
idempotency.flush();
}
Instant now = Instant.now();
MessageEntity saved = messages.saveAndFlush(new MessageEntity(chatId, actorId, request.getText(), attachmentIds));
idempotency.saveAndFlush(new IdempotencyRecord(
actorId, METHOD, path, key, hash, saved.getId(), now, now.plus(idempotencyTtl)));
return mapper.toModel(messages.findById(saved.getId()).orElseThrow(NotFoundException::new));
}
@Transactional(readOnly = true)
public MessagePage list(UUID actorId, UUID chatId, Integer limit, String cursor) {
requireMember(chatId, actorId);
int pageSize = limit == null ? 50 : limit;
CursorCodec.Position position = cursor == null ? null : CursorCodec.decode(cursor, chatId);
List<MessageEntity> rows = position == null
? messages.findFirst(chatId, PageRequest.of(0, pageSize + 1))
: messages.findAfter(chatId, position.createdAt(), position.id(), PageRequest.of(0, pageSize + 1));
boolean more = rows.size() > pageSize;
List<MessageEntity> visible = more ? rows.subList(0, pageSize) : rows;
String next = null;
if (more) {
MessageEntity last = visible.get(visible.size() - 1);
next = CursorCodec.encode(chatId, last.getCreatedAt(), last.getId());
}
return new MessagePage(visible.stream().map(mapper::toModel).toList(), next);
}
private void lock(String scope) {
byte[] digest = digest(scope);
long key = 0;
for (int i = 0; i < 8; i++) {
key = (key << 8) | (digest[i] & 0xffL);
}
long lockKey = key;
jdbc.execute((Connection connection) -> {
try (var statement = connection.prepareStatement("select pg_advisory_xact_lock(?)")) {
statement.setLong(1, lockKey);
statement.execute();
}
return null;
});
}
private static String requireKey(String key) {
if (key == null || !IDEMPOTENCY_KEY.matcher(key).matches()) {
throw new InvalidRequestException("Некорректный запрос");
}
return key;
}
private void requireAttachments(UUID chatId, List<UUID> attachmentIds) {
if (attachmentIds.isEmpty()) {
return;
}
List<AttachmentEntity> found = attachments.findAllById(attachmentIds);
if (found.size() != attachmentIds.size()) {
throw new NotFoundException();
}
for (AttachmentEntity attachment : found) {
if (!chatId.equals(attachment.getChatId())) {
throw new NotFoundException();
}
if ("failed".equals(attachment.getStatus())) {
throw new ConflictException("Вложение не удалось обработать");
}
}
}
private static List<UUID> attachmentIds(CreateMessage request) {
List<UUID> ids = request.getAttachmentIds();
if (ids == null || ids.isEmpty()) {
return List.of();
}
if (ids.size() > 10 || ids.stream().anyMatch(Objects::isNull) || new LinkedHashSet<>(ids).size() != ids.size()) {
throw new InvalidRequestException("Некорректный запрос");
}
return new ArrayList<>(ids);
}
private static void requireText(String text, boolean hasAttachments) {
if (text == null) {
throw new InvalidRequestException("Некорректный запрос");
}
int codePoints = text.codePointCount(0, text.length());
if (codePoints > 4000 || (text.isBlank() && !hasAttachments)) {
throw new InvalidRequestException("Некорректный запрос");
}
}
private static String requestHash(String text, List<UUID> attachmentIds) {
StringBuilder canonical = new StringBuilder(text == null ? "" : text).append('\n');
for (int i = 0; i < attachmentIds.size(); i++) {
if (i > 0) {
canonical.append(',');
}
canonical.append(attachmentIds.get(i));
}
return sha256(canonical.toString());
}
private static String sha256(String value) {
return HexFormat.of().formatHex(digest(value));
}
private static byte[] digest(String value) {
try {
return MessageDigest.getInstance("SHA-256").digest(value.getBytes(StandardCharsets.UTF_8));
} catch (NoSuchAlgorithmException exception) {
throw new IllegalStateException(exception);
}
}
private void requireMember(UUID chatId, UUID actorId) {
if (!members.existsByChatIdAndUserId(chatId, actorId)) {
throw new NotFoundException();
}
}
}
@@ -0,0 +1,30 @@
package ru.mtuci.messenger.service;
import java.util.concurrent.ThreadLocalRandom;
public final class RetryDelay {
private RetryDelay() {
}
public static long exponentialMillis(int attempt, long baseMs, long capMs) {
int steps = Math.max(attempt, 1);
long delay = Math.max(baseMs, 1);
for (int i = 1; i < steps && delay < capMs; i++) {
if (delay > capMs / 2) {
return capMs;
}
delay *= 2;
}
return Math.min(delay, capMs);
}
public static long millis(int attempt, long baseMs, long capMs) {
long exp = exponentialMillis(attempt, baseMs, capMs);
long low = exp / 2;
if (exp <= low) {
return exp;
}
return low + ThreadLocalRandom.current().nextLong(exp - low + 1);
}
}
@@ -0,0 +1,35 @@
package ru.mtuci.messenger.service;
import ru.mtuci.messenger.exception.InvalidRequestException;
public final class TextRules {
private TextRules() {
}
public static void displayName(String value) {
text(value, 1, 100);
}
public static void title(String value) {
text(value, 1, 100);
}
public static void message(String value) {
text(value, 1, 4000);
}
public static void password(String value) {
text(value, 12, 128);
}
private static void text(String value, int minCodePoints, int maxCodePoints) {
if (value == null || value.isBlank()) {
throw new InvalidRequestException("Некорректный запрос");
}
int codePoints = value.codePointCount(0, value.length());
if (codePoints < minCodePoints || codePoints > maxCodePoints) {
throw new InvalidRequestException("Некорректный запрос");
}
}
}
@@ -0,0 +1,147 @@
package ru.mtuci.messenger.service;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.dao.DataIntegrityViolationException;
import org.springframework.security.crypto.password.PasswordEncoder;
import org.springframework.stereotype.Service;
import org.springframework.transaction.support.TransactionTemplate;
import ru.mtuci.messenger.api.model.CreateUser;
import ru.mtuci.messenger.api.model.Session;
import ru.mtuci.messenger.api.model.User;
import ru.mtuci.messenger.cache.SessionCache;
import ru.mtuci.messenger.domain.SessionEntity;
import ru.mtuci.messenger.domain.UserAccount;
import ru.mtuci.messenger.exception.ConflictException;
import ru.mtuci.messenger.exception.NotFoundException;
import ru.mtuci.messenger.exception.UnauthorizedException;
import ru.mtuci.messenger.mapper.UserMapper;
import ru.mtuci.messenger.repository.SessionRepository;
import ru.mtuci.messenger.repository.UserRepository;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.time.Duration;
import java.time.Instant;
import java.util.Base64;
import java.util.HexFormat;
import java.util.Optional;
import java.util.UUID;
@Service
public class UserService {
private final UserRepository users;
private final SessionRepository sessions;
private final SessionCache cache;
private final UserMapper mapper;
private final PasswordEncoder passwords;
private final TransactionTemplate transactions;
private final Duration sessionTtl;
private final SecureRandom random = new SecureRandom();
private volatile String dummyHash;
public UserService(UserRepository users, SessionRepository sessions, SessionCache cache, UserMapper mapper,
PasswordEncoder passwords, TransactionTemplate transactions,
@Value("${app.session-ttl}") Duration sessionTtl) {
this.users = users;
this.sessions = sessions;
this.cache = cache;
this.mapper = mapper;
this.passwords = passwords;
this.transactions = transactions;
this.sessionTtl = sessionTtl;
}
public User register(CreateUser request) {
TextRules.displayName(request.getDisplayName());
TextRules.password(request.getPassword());
try {
return transactions.execute(status -> mapper.toModel(users.saveAndFlush(new UserAccount(
request.getUsername(),
request.getDisplayName(),
passwords.encode(request.getPassword())
))));
} catch (DataIntegrityViolationException exception) {
throw new ConflictException("Имя пользователя занято");
}
}
public User current(UUID userId) {
return users.findById(userId).map(mapper::toModel).orElseThrow(NotFoundException::new);
}
public Session login(String authorization) {
String[] credentials = basic(authorization);
UserAccount user = users.findByUsername(credentials[0]).orElse(null);
String hash = user == null ? dummyHash() : user.getPasswordHash();
if (user == null || !passwords.matches(credentials[1], hash)) {
throw UnauthorizedException.basic();
}
String token = newToken();
SessionEntity session = sessions.saveAndFlush(new SessionEntity(tokenHash(token), user.getId(), sessionTtl));
cache.put(tokenHash(token), user.getId(), session.getExpiresAt());
return mapper.toSession(token, session.getExpiresAt(), user);
}
public Optional<UUID> authenticate(String token) {
String hash = tokenHash(token);
Optional<UUID> cached = cache.find(hash);
if (cached.isPresent()) {
return cached;
}
return sessions.findActive(hash, Instant.now()).map(session -> {
cache.put(hash, session.getUserId(), session.getExpiresAt());
return session.getUserId();
});
}
public void revoke(String token) {
String hash = tokenHash(token);
if (sessions.revoke(hash, Instant.now()) == 0) {
throw UnauthorizedException.bearer();
}
cache.evict(hash);
}
public static String tokenHash(String token) {
try {
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(token.getBytes(StandardCharsets.UTF_8)));
} catch (NoSuchAlgorithmException exception) {
throw new IllegalStateException(exception);
}
}
private String newToken() {
byte[] bytes = new byte[32];
random.nextBytes(bytes);
return Base64.getUrlEncoder().withoutPadding().encodeToString(bytes);
}
private String dummyHash() {
String current = dummyHash;
if (current == null) {
current = passwords.encode("not-a-real-password");
dummyHash = current;
}
return current;
}
private static String[] basic(String header) {
if (header == null || !header.regionMatches(true, 0, "Basic ", 0, 6)) {
throw UnauthorizedException.basic();
}
String decoded;
try {
decoded = new String(Base64.getDecoder().decode(header.substring(6).trim()), StandardCharsets.UTF_8);
} catch (IllegalArgumentException exception) {
throw UnauthorizedException.basic();
}
int colon = decoded.indexOf(':');
if (colon <= 0) {
throw UnauthorizedException.basic();
}
return new String[]{decoded.substring(0, colon), decoded.substring(colon + 1)};
}
}
@@ -0,0 +1,149 @@
package ru.mtuci.messenger.storage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import ru.mtuci.messenger.exception.NotFoundException;
import ru.mtuci.messenger.exception.UnavailableException;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.BucketAlreadyExistsException;
import software.amazon.awssdk.services.s3.model.BucketAlreadyOwnedByYouException;
import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
import software.amazon.awssdk.services.s3.model.DeleteObjectRequest;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadBucketRequest;
import software.amazon.awssdk.services.s3.model.ListObjectsV2Request;
import software.amazon.awssdk.services.s3.model.ListObjectsV2Response;
import software.amazon.awssdk.services.s3.model.NoSuchBucketException;
import software.amazon.awssdk.services.s3.model.NoSuchKeyException;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.model.S3Object;
import java.io.InputStream;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
@Component
public class ObjectStorage {
private static final Logger log = LoggerFactory.getLogger(ObjectStorage.class);
private final S3Client data;
private final S3Client bootstrap;
private final String bucket;
public ObjectStorage(@Qualifier("s3Data") S3Client data,
@Qualifier("s3Bootstrap") S3Client bootstrap,
@Value("${app.s3.bucket}") String bucket) {
this.data = data;
this.bootstrap = bootstrap;
this.bucket = bucket;
}
public record StoredObject(String key, Instant modified) {
}
public void ensureBucket() {
try {
bootstrap.headBucket(HeadBucketRequest.builder().bucket(bucket).build());
} catch (NoSuchBucketException missing) {
createBucket();
} catch (S3Exception exception) {
if (exception.statusCode() == 404) {
createBucket();
return;
}
throw storageFailure(exception);
} catch (RuntimeException exception) {
throw storageFailure(exception);
}
}
public void put(String key, byte[] body, String contentType) {
try {
data.putObject(PutObjectRequest.builder()
.bucket(bucket)
.key(key)
.contentType(contentType)
.contentLength((long) body.length)
.build(), RequestBody.fromBytes(body));
} catch (RuntimeException exception) {
throw storageFailure(exception);
}
}
public InputStream get(String key) {
try {
return data.getObject(GetObjectRequest.builder().bucket(bucket).key(key).build());
} catch (NoSuchKeyException missing) {
throw new NotFoundException();
} catch (S3Exception exception) {
if (exception.statusCode() == 404) {
throw new NotFoundException();
}
throw storageFailure(exception);
} catch (RuntimeException exception) {
throw storageFailure(exception);
}
}
public List<StoredObject> list(String prefix) {
List<StoredObject> objects = new ArrayList<>();
String token = null;
do {
ListObjectsV2Request.Builder request = ListObjectsV2Request.builder()
.bucket(bucket)
.prefix(prefix);
if (token != null) {
request.continuationToken(token);
}
ListObjectsV2Response page;
try {
page = data.listObjectsV2(request.build());
} catch (RuntimeException exception) {
throw storageFailure(exception);
}
if (page.contents() != null) {
for (S3Object object : page.contents()) {
Instant modified = object.lastModified() == null ? Instant.EPOCH : object.lastModified();
objects.add(new StoredObject(object.key(), modified));
}
}
token = page.nextContinuationToken();
} while (token != null);
return objects;
}
public void deleteQuietly(String key) {
try {
data.deleteObject(DeleteObjectRequest.builder().bucket(bucket).key(key).build());
} catch (RuntimeException exception) {
log.warn("s3 delete failed status={}", exception.getClass().getSimpleName());
}
}
private void createBucket() {
try {
bootstrap.createBucket(CreateBucketRequest.builder().bucket(bucket).build());
} catch (BucketAlreadyExistsException | BucketAlreadyOwnedByYouException ignored) {
// another replica created the bucket
} catch (S3Exception exception) {
if (exception.statusCode() == 409) {
return;
}
throw storageFailure(exception);
} catch (RuntimeException exception) {
throw storageFailure(exception);
}
}
private static UnavailableException storageFailure(RuntimeException exception) {
log.warn("s3 failure type={}", exception.getClass().getSimpleName());
return new UnavailableException("Хранилище временно недоступно");
}
}
@@ -0,0 +1,81 @@
package ru.mtuci.messenger.storage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import ru.mtuci.messenger.repository.AttachmentRepository;
import ru.mtuci.messenger.repository.AttachmentVariantRepository;
import java.time.Duration;
import java.time.Instant;
import java.util.List;
import java.util.function.Predicate;
@Component
@Profile("!migrate")
@ConditionalOnProperty(name = "app.s3.sweep-enabled", havingValue = "true", matchIfMissing = true)
public class OrphanSweeper {
private static final Logger log = LoggerFactory.getLogger(OrphanSweeper.class);
private static final List<String> ACTIVE = List.of("queued", "processing");
private final ObjectStorage storage;
private final AttachmentRepository attachments;
private final AttachmentVariantRepository variants;
private final Duration grace;
private final Duration stagingRetention;
public OrphanSweeper(ObjectStorage storage, AttachmentRepository attachments, AttachmentVariantRepository variants,
@Value("${app.s3.orphan-grace:2m}") Duration grace,
@Value("${app.staging.retention:86400s}") Duration stagingRetention) {
this.storage = storage;
this.attachments = attachments;
this.variants = variants;
this.grace = grace;
this.stagingRetention = stagingRetention;
}
@Scheduled(fixedDelayString = "${app.s3.sweep-interval-ms:60000}",
initialDelayString = "${app.s3.sweep-initial-delay-ms:60000}")
public int sweep() {
Instant now = Instant.now();
int removed = sweepPrefix("originals/", now.minus(grace), attachments::existsByObjectKey);
removed += sweepPrefix("variants/", now.minus(grace), variants::existsByObjectKey);
removed += sweepPrefix("staging/", now.minus(stagingRetention), this::stagingActive);
if (removed > 0) {
log.info("removed {} unreferenced objects", removed);
}
return removed;
}
public int sweepOlderThan(Instant cutoff) {
int removed = sweepPrefix("originals/", cutoff, attachments::existsByObjectKey);
removed += sweepPrefix("variants/", cutoff, variants::existsByObjectKey);
removed += sweepPrefix("staging/", cutoff, this::stagingActive);
return removed;
}
private boolean stagingActive(String key) {
return attachments.existsByStagingKeyAndStatusIn(key, ACTIVE);
}
private int sweepPrefix(String prefix, Instant cutoff, Predicate<String> referenced) {
int removed = 0;
for (ObjectStorage.StoredObject object : storage.list(prefix)) {
if (!expiredUnreferenced(object.modified(), cutoff, referenced.test(object.key()))) {
continue;
}
storage.deleteQuietly(object.key());
removed++;
}
return removed;
}
static boolean expiredUnreferenced(Instant modified, Instant cutoff, boolean referenced) {
return !referenced && !modified.isAfter(cutoff);
}
}
@@ -0,0 +1,50 @@
package ru.mtuci.messenger.storage;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.http.apache.ApacheHttpClient;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.S3Configuration;
import java.net.URI;
import java.time.Duration;
@Configuration
public class S3Config {
@Bean(name = "s3Data", destroyMethod = "close")
S3Client s3Data(@Value("${app.s3.endpoint}") String endpoint,
@Value("${app.s3.region}") String region,
@Value("${app.s3.access-key}") String accessKey,
@Value("${app.s3.secret-key}") String secretKey) {
return client(endpoint, region, accessKey, secretKey);
}
@Bean(name = "s3Bootstrap", destroyMethod = "close")
S3Client s3Bootstrap(@Value("${app.s3.endpoint}") String endpoint,
@Value("${app.s3.region}") String region,
@Value("${app.s3.bootstrap-access-key}") String accessKey,
@Value("${app.s3.bootstrap-secret-key}") String secretKey) {
return client(endpoint, region, accessKey, secretKey);
}
private static S3Client client(String endpoint, String region, String accessKey, String secretKey) {
return S3Client.builder()
.endpointOverride(URI.create(endpoint))
.region(Region.of(region))
.credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(accessKey, secretKey)))
.serviceConfiguration(S3Configuration.builder().pathStyleAccessEnabled(true).build())
.httpClientBuilder(ApacheHttpClient.builder()
.connectionTimeout(Duration.ofSeconds(2))
.socketTimeout(Duration.ofSeconds(8)))
.overrideConfiguration(config -> config
.apiCallTimeout(Duration.ofSeconds(10))
.apiCallAttemptTimeout(Duration.ofSeconds(8)))
.build();
}
}
@@ -0,0 +1,30 @@
package ru.mtuci.messenger.storage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.health.contributor.Health;
import org.springframework.boot.health.contributor.HealthIndicator;
import org.springframework.stereotype.Component;
@Component
public class S3HealthIndicator implements HealthIndicator {
private static final Logger log = LoggerFactory.getLogger(S3HealthIndicator.class);
private final ObjectStorage storage;
public S3HealthIndicator(ObjectStorage storage) {
this.storage = storage;
}
@Override
public Health health() {
try {
storage.ensureBucket();
return Health.up().build();
} catch (RuntimeException exception) {
log.info("s3 readiness failed type={}", exception.getClass().getSimpleName());
return Health.down().build();
}
}
}
@@ -0,0 +1,11 @@
package ru.mtuci.messenger.storage;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.EnableScheduling;
@Configuration
@EnableScheduling
@Profile("!migrate")
public class SweepConfig {
}
@@ -0,0 +1,99 @@
package ru.mtuci.messenger.web;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import jakarta.validation.ConstraintViolationException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Profile;
import org.springframework.dao.DataAccessException;
import org.springframework.transaction.CannotCreateTransactionException;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.http.converter.HttpMessageNotReadableException;
import org.springframework.web.HttpMediaTypeNotSupportedException;
import org.springframework.web.HttpRequestMethodNotSupportedException;
import org.springframework.web.bind.MethodArgumentNotValidException;
import org.springframework.web.bind.MissingServletRequestParameterException;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import org.springframework.web.method.annotation.HandlerMethodValidationException;
import org.springframework.web.method.annotation.MethodArgumentTypeMismatchException;
import org.springframework.web.servlet.NoHandlerFoundException;
import org.springframework.web.servlet.resource.NoResourceFoundException;
import ru.mtuci.messenger.api.model.Error;
import ru.mtuci.messenger.api.model.ErrorError;
import ru.mtuci.messenger.exception.ApiException;
import ru.mtuci.messenger.exception.UnauthorizedException;
@RestControllerAdvice
@Profile("!worker")
public class ApiAdvice {
private static final Logger log = LoggerFactory.getLogger(ApiAdvice.class);
private final InstanceIdentity instanceId;
public ApiAdvice(InstanceIdentity instanceId) {
this.instanceId = instanceId;
}
@ExceptionHandler(ApiException.class)
public ResponseEntity<Error> handleApi(HttpServletRequest request, HttpServletResponse response, ApiException exception) {
log.info("api error code={}", exception.code().getValue());
String challenge = exception instanceof UnauthorizedException unauthorized ? unauthorized.challenge() : null;
return response(request, response, exception.status(), exception.code(), exception.getMessage(), challenge);
}
@ExceptionHandler({
HttpMessageNotReadableException.class,
MethodArgumentNotValidException.class,
ConstraintViolationException.class,
HandlerMethodValidationException.class,
MethodArgumentTypeMismatchException.class,
MissingServletRequestParameterException.class,
HttpRequestMethodNotSupportedException.class
})
public ResponseEntity<Error> handleInvalid(HttpServletRequest request, HttpServletResponse response, Exception exception) {
log.info("invalid request type={}", exception.getClass().getSimpleName());
return response(request, response, HttpStatus.BAD_REQUEST, ErrorError.CodeEnum.INVALID_REQUEST, "Некорректный запрос", null);
}
@ExceptionHandler(HttpMediaTypeNotSupportedException.class)
public ResponseEntity<Error> handleMedia(HttpServletRequest request, HttpServletResponse response) {
return response(request, response, HttpStatus.UNSUPPORTED_MEDIA_TYPE, ErrorError.CodeEnum.UNSUPPORTED_MEDIA_TYPE,
"Неподдерживаемый тип содержимого", null);
}
@ExceptionHandler({NoResourceFoundException.class, NoHandlerFoundException.class})
public ResponseEntity<Error> handleMissing(HttpServletRequest request, HttpServletResponse response) {
return response(request, response, HttpStatus.NOT_FOUND, ErrorError.CodeEnum.NOT_FOUND, "Объект не найден", null);
}
@ExceptionHandler({DataAccessException.class, CannotCreateTransactionException.class})
public ResponseEntity<Error> handleDataAccess(HttpServletRequest request, HttpServletResponse response, Exception exception) {
log.warn("data access failure type={}", exception.getClass().getSimpleName());
return response(request, response, HttpStatus.SERVICE_UNAVAILABLE, ErrorError.CodeEnum.UNAVAILABLE,
"Сервис временно недоступен", null);
}
@ExceptionHandler(Exception.class)
public ResponseEntity<Error> handleUnexpected(HttpServletRequest request, HttpServletResponse response, Exception exception) {
log.error("unexpected failure type={}", exception.getClass().getName(), exception);
return response(request, response, HttpStatus.INTERNAL_SERVER_ERROR, ErrorError.CodeEnum.INTERNAL_ERROR,
"Внутренняя ошибка", null);
}
private ResponseEntity<Error> response(HttpServletRequest request, HttpServletResponse response, HttpStatus status,
ErrorError.CodeEnum code, String message, String challenge) {
String requestId = RequestIdFilter.id(request);
response.setHeader(RequestIdFilter.HEADER, requestId);
response.setHeader(InstanceIdFilter.HEADER, instanceId.value());
HttpHeaders headers = new HttpHeaders();
if (challenge != null) {
headers.set(HttpHeaders.WWW_AUTHENTICATE, challenge);
}
return new ResponseEntity<>(new Error(new ErrorError(code, message, requestId)), headers, status);
}
}
@@ -0,0 +1,75 @@
package ru.mtuci.messenger.web;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Profile;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.http.HttpMethod;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
import org.springframework.web.servlet.HandlerExceptionResolver;
import ru.mtuci.messenger.exception.UnauthorizedException;
import ru.mtuci.messenger.service.UserService;
import java.io.IOException;
import java.util.UUID;
@Component
@Profile("!worker")
@Order(Ordered.HIGHEST_PRECEDENCE + 10)
public class BearerAuthFilter extends OncePerRequestFilter {
public static final String TOKEN = "token";
private final UserService users;
private final HandlerExceptionResolver errors;
public BearerAuthFilter(UserService users, @Qualifier("handlerExceptionResolver") HandlerExceptionResolver errors) {
this.users = users;
this.errors = errors;
}
public static UUID userId(HttpServletRequest request) {
return (UUID) request.getAttribute("userId");
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain)
throws ServletException, IOException {
if (isPublic(request)) {
chain.doFilter(request, response);
return;
}
try {
String header = request.getHeader("Authorization");
if (header == null || !header.regionMatches(true, 0, "Bearer ", 0, 7)) {
throw UnauthorizedException.bearer();
}
String token = header.substring(7).trim();
UUID userId = users.authenticate(token).orElseThrow(UnauthorizedException::bearer);
request.setAttribute("userId", userId);
request.setAttribute(TOKEN, token);
} catch (RuntimeException exception) {
errors.resolveException(request, response, null, exception);
return;
}
chain.doFilter(request, response);
}
private static boolean isPublic(HttpServletRequest request) {
String path = request.getRequestURI();
if (path.startsWith("/internal/demo/")) {
return true;
}
if (HttpMethod.GET.matches(request.getMethod()) && (path.equals("/health/live") || path.equals("/health/ready")
|| path.startsWith("/actuator/"))) {
return true;
}
return HttpMethod.POST.matches(request.getMethod())
&& (path.equals("/api/v1/users") || path.equals("/api/v1/auth/sessions"));
}
}
@@ -0,0 +1,34 @@
package ru.mtuci.messenger.web;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.springframework.context.annotation.Profile;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
import java.io.IOException;
@Component
@Profile("!worker")
@Order(Ordered.HIGHEST_PRECEDENCE)
public class InstanceIdFilter extends OncePerRequestFilter {
static final String HEADER = "X-Instance-Id";
private final InstanceIdentity instanceId;
public InstanceIdFilter(InstanceIdentity instanceId) {
this.instanceId = instanceId;
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
response.setHeader(HEADER, instanceId.value());
filterChain.doFilter(request, response);
}
}
@@ -0,0 +1,19 @@
package ru.mtuci.messenger.web;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
@Component
public class InstanceIdentity {
private final String value;
public InstanceIdentity(@Value("${app.instance-id:local}") String configured) {
String trimmed = configured == null ? "" : configured.trim();
this.value = trimmed.isEmpty() ? "local" : trimmed;
}
public String value() {
return value;
}
}
@@ -0,0 +1,47 @@
package ru.mtuci.messenger.web;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.slf4j.MDC;
import org.springframework.context.annotation.Profile;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
import java.io.IOException;
import java.util.UUID;
@Component
@Profile("!worker")
@Order(Ordered.HIGHEST_PRECEDENCE)
public class RequestIdFilter extends OncePerRequestFilter {
static final String HEADER = "X-Request-Id";
static String id(HttpServletRequest request) {
Object existing = request.getAttribute(HEADER);
if (existing != null) {
return existing.toString();
}
String header = request.getHeader(HEADER);
String id = header == null || header.isBlank() ? UUID.randomUUID().toString() : header;
request.setAttribute(HEADER, id);
return id;
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
String requestId = id(request);
response.setHeader(HEADER, requestId);
MDC.put("requestId", requestId);
try {
filterChain.doFilter(request, response);
} finally {
MDC.remove("requestId");
}
}
}
@@ -0,0 +1,25 @@
package ru.mtuci.messenger.web;
import com.fasterxml.jackson.annotation.JsonFormat;
import org.springframework.boot.jackson.autoconfigure.JsonMapperBuilderCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import java.time.OffsetDateTime;
@Configuration
public class TimeFormatConfig {
/**
* Python 3.9 {@code fromisoformat} принимает только 0, 3 или 6 знаков дробной секунды.
* Jackson по умолчанию отбрасывает хвостовые нули, и время вроде {@code .38078Z} ломает публичные тесты.
*/
@Bean
@Order(Ordered.LOWEST_PRECEDENCE)
JsonMapperBuilderCustomizer utcMicroseconds() {
JsonFormat.Value format = JsonFormat.Value.forPattern("yyyy-MM-dd'T'HH:mm:ss.SSSSSSXXX");
return builder -> builder.withConfigOverride(OffsetDateTime.class, override -> override.setFormat(format));
}
}
@@ -0,0 +1,14 @@
spring:
autoconfigure:
exclude:
- org.springframework.boot.kafka.autoconfigure.KafkaAutoConfiguration
app:
broker:
enabled: false
outbox:
enabled: false
worker:
consume: false
s3:
sweep-enabled: false
@@ -0,0 +1,9 @@
spring:
main:
web-application-type: none
app:
worker:
consume: true
s3:
sweep-enabled: false
+168
View File
@@ -0,0 +1,168 @@
server:
address: ${APP_HOST:0.0.0.0}
port: ${APP_PORT:8080}
shutdown: graceful
tomcat:
connection-timeout: 3s
max-http-form-post-size: 12MB
max-swallow-size: 12MB
error:
include-message: never
include-stacktrace: never
include-binding-errors: never
whitelabel:
enabled: false
spring:
application:
name: messenger
lifecycle:
timeout-per-shutdown-phase: 4s
threads:
virtual:
enabled: false
mvc:
problemdetails:
enabled: false
throw-exception-if-no-handler-found: true
web:
resources:
add-mappings: false
jackson:
use-jackson2-defaults: false
property-naming-strategy: SNAKE_CASE
time-zone: UTC
default-property-inclusion: always
deserialization:
fail-on-unknown-properties: true
datatype:
datetime:
write-dates-as-timestamps: false
write-utc-as-offset: false
datasource:
url: ${SPRING_DATASOURCE_URL:jdbc:postgresql://localhost:5432/messenger}
username: ${SPRING_DATASOURCE_USERNAME:messenger}
password: ${SPRING_DATASOURCE_PASSWORD:messenger}
hikari:
maximum-pool-size: 10
minimum-idle: 1
connection-timeout: 3000
validation-timeout: 1000
jpa:
open-in-view: false
hibernate:
ddl-auto: validate
properties:
hibernate.jdbc.time_zone: UTC
liquibase:
change-log: classpath:db/changelog/db.changelog-master.yaml
data:
redis:
host: ${SPRING_DATA_REDIS_HOST:localhost}
port: ${SPRING_DATA_REDIS_PORT:6379}
timeout: 1s
connect-timeout: 1s
repositories:
enabled: false
kafka:
bootstrap-servers: ${SPRING_KAFKA_BOOTSTRAP_SERVERS:localhost:9092}
producer:
acks: all
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: org.apache.kafka.common.serialization.StringSerializer
properties:
enable.idempotence: true
consumer:
group-id: attachment-worker
auto-offset-reset: earliest
enable-auto-commit: false
key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
properties:
max-poll-records: ${WORKER_CONCURRENCY:2}
listener:
ack-mode: manual
missing-topics-fatal: false
management:
endpoints:
access:
default: READ_ONLY
web:
exposure:
include:
- health
- info
- prometheus
endpoint:
health:
probes:
enabled: true
show-details: always
show-components: always
cache:
time-to-live: 0s
group:
readiness:
include:
- readinessState
- db
- s3
health:
redis:
enabled: false
kafka:
enabled: false
prometheus:
metrics:
export:
enabled: true
info:
app:
name: messenger
lab: 5
app:
session-ttl: ${SESSION_TTL_SECONDS:3600}s
idempotency-ttl: ${IDEMPOTENCY_TTL_SECONDS:86400}s
instance-id: ${HOSTNAME:local}
upload:
max-bytes: ${MAX_UPLOAD_BYTES:10485760}
max-pixels: ${MAX_IMAGE_PIXELS:40000000}
s3:
endpoint: ${S3_ENDPOINT:http://127.0.0.1:8333}
region: ${S3_REGION:us-east-1}
bucket: ${S3_BUCKET:messenger-attachments}
access-key: ${S3_ACCESS_KEY:messenger}
secret-key: ${S3_SECRET_KEY:messenger}
bootstrap-access-key: ${S3_BOOTSTRAP_ACCESS_KEY:messenger-bootstrap}
bootstrap-secret-key: ${S3_BOOTSTRAP_SECRET_KEY:messenger-bootstrap}
orphan-grace: ${S3_ORPHAN_GRACE:2m}
sweep-interval-ms: ${S3_SWEEP_INTERVAL_MS:60000}
sweep-initial-delay-ms: ${S3_SWEEP_INITIAL_DELAY_MS:60000}
broker:
enabled: true
outbox:
enabled: true
poll-ms: ${OUTBOX_POLL_MS:500}
worker:
consume: false
concurrency: ${WORKER_CONCURRENCY:2}
job:
max-attempts: ${JOB_MAX_ATTEMPTS:5}
timeout: ${JOB_TIMEOUT_SECONDS:60}s
backoff-base-ms: ${JOB_BACKOFF_BASE_MS:1000}
backoff-cap-ms: ${JOB_BACKOFF_CAP_MS:30000}
reaper-ms: ${JOB_REAPER_MS:5000}
demo:
token: ${DEMO_TOKEN:lab-demo}
image:
jpeg-quality: ${JPEG_QUALITY:0.85}
webp-quality: ${WEBP_QUALITY:0.8}
staging:
retention: ${STAGING_RETENTION_SECONDS:86400}s
logging:
pattern:
console: "%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} %-5level [%X{requestId}] %msg%n"
@@ -0,0 +1,41 @@
databaseChangeLog:
- changeSet:
id: 001-users
author: lab2
changes:
- createTable:
tableName: users
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: users_pkey
- column:
name: username
type: varchar(32)
constraints:
nullable: false
unique: true
uniqueConstraintName: users_username_key
- column:
name: display_name
type: varchar(100)
constraints:
nullable: false
- column:
name: password_hash
type: varchar(255)
constraints:
nullable: false
- column:
name: created_at
type: timestamp with time zone
constraints:
nullable: false
- sql:
sql: alter table users add constraint users_username_pattern check (username ~ '^[a-z0-9_]{3,32}$')
- sql:
sql: alter table users add constraint users_display_name_text check (char_length(btrim(display_name)) > 0)
@@ -0,0 +1,47 @@
databaseChangeLog:
- changeSet:
id: 002-sessions
author: lab2
changes:
- createTable:
tableName: sessions
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: sessions_pkey
- column:
name: token_hash
type: varchar(64)
constraints:
nullable: false
unique: true
uniqueConstraintName: sessions_token_hash_key
- column:
name: user_id
type: uuid
constraints:
nullable: false
- column:
name: expires_at
type: timestamp with time zone
constraints:
nullable: false
- column:
name: revoked_at
type: timestamp with time zone
- column:
name: created_at
type: timestamp with time zone
constraints:
nullable: false
- addForeignKeyConstraint:
baseTableName: sessions
baseColumnNames: user_id
referencedTableName: users
referencedColumnNames: id
constraintName: sessions_user_fk
onDelete: RESTRICT
@@ -0,0 +1,77 @@
databaseChangeLog:
- changeSet:
id: 003-chats
author: lab2
changes:
- createTable:
tableName: chats
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: chats_pkey
- column:
name: title
type: varchar(100)
constraints:
nullable: false
- column:
name: created_at
type: timestamp with time zone
constraints:
nullable: false
- sql:
sql: alter table chats add constraint chats_title_text check (char_length(btrim(title)) > 0)
- createTable:
tableName: chat_members
columns:
- column:
name: chat_id
type: uuid
constraints:
nullable: false
- column:
name: user_id
type: uuid
constraints:
nullable: false
- column:
name: ordinal
type: int
constraints:
nullable: false
- addPrimaryKey:
tableName: chat_members
columnNames: chat_id, user_id
constraintName: chat_members_pkey
- addForeignKeyConstraint:
baseTableName: chat_members
baseColumnNames: chat_id
referencedTableName: chats
referencedColumnNames: id
constraintName: chat_members_chat_fk
onDelete: RESTRICT
- addForeignKeyConstraint:
baseTableName: chat_members
baseColumnNames: user_id
referencedTableName: users
referencedColumnNames: id
constraintName: chat_members_user_fk
onDelete: RESTRICT
- createIndex:
indexName: chat_members_user_idx
tableName: chat_members
columns:
- column:
name: user_id
- createIndex:
indexName: chats_history_idx
tableName: chats
columns:
- column:
name: created_at
- column:
name: id
@@ -0,0 +1,61 @@
databaseChangeLog:
- changeSet:
id: 004-messages
author: lab2
changes:
- createTable:
tableName: messages
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: messages_pkey
- column:
name: chat_id
type: uuid
constraints:
nullable: false
- column:
name: sender_id
type: uuid
constraints:
nullable: false
- column:
name: text
type: varchar(4000)
constraints:
nullable: false
- column:
name: created_at
type: timestamp with time zone
constraints:
nullable: false
- addForeignKeyConstraint:
baseTableName: messages
baseColumnNames: chat_id
referencedTableName: chats
referencedColumnNames: id
constraintName: messages_chat_fk
onDelete: RESTRICT
- addForeignKeyConstraint:
baseTableName: messages
baseColumnNames: sender_id
referencedTableName: users
referencedColumnNames: id
constraintName: messages_sender_fk
onDelete: RESTRICT
- sql:
sql: alter table messages add constraint messages_text_not_blank check (char_length(btrim(text)) > 0 and char_length(text) <= 4000)
- createIndex:
indexName: messages_history_idx
tableName: messages
columns:
- column:
name: chat_id
- column:
name: created_at
- column:
name: id
@@ -0,0 +1,73 @@
databaseChangeLog:
- changeSet:
id: 005-idempotency
author: lab3
changes:
- createTable:
tableName: idempotency_records
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: idempotency_records_pkey
- column:
name: user_id
type: uuid
constraints:
nullable: false
- column:
name: http_method
type: varchar(8)
constraints:
nullable: false
- column:
name: path
type: varchar(256)
constraints:
nullable: false
- column:
name: idempotency_key
type: varchar(128)
constraints:
nullable: false
- column:
name: request_hash
type: varchar(64)
constraints:
nullable: false
- column:
name: message_id
type: uuid
constraints:
nullable: false
- column:
name: created_at
type: timestamp with time zone
constraints:
nullable: false
- column:
name: expires_at
type: timestamp with time zone
constraints:
nullable: false
- addUniqueConstraint:
tableName: idempotency_records
columnNames: user_id, http_method, path, idempotency_key
constraintName: idempotency_scope_uq
- addForeignKeyConstraint:
baseTableName: idempotency_records
baseColumnNames: user_id
referencedTableName: users
referencedColumnNames: id
constraintName: idempotency_user_fk
onDelete: RESTRICT
- addForeignKeyConstraint:
baseTableName: idempotency_records
baseColumnNames: message_id
referencedTableName: messages
referencedColumnNames: id
constraintName: idempotency_message_fk
onDelete: RESTRICT
@@ -0,0 +1,127 @@
databaseChangeLog:
- changeSet:
id: 006-attachments
author: lab4
changes:
- sql:
sql: alter table messages drop constraint messages_text_not_blank
- createTable:
tableName: attachments
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: attachments_pkey
- column:
name: chat_id
type: uuid
constraints:
nullable: false
- column:
name: owner_id
type: uuid
constraints:
nullable: false
- column:
name: filename
type: varchar(255)
constraints:
nullable: false
- column:
name: content_type
type: varchar(32)
constraints:
nullable: false
- column:
name: size_bytes
type: bigint
constraints:
nullable: false
- column:
name: sha256
type: varchar(64)
constraints:
nullable: false
- column:
name: object_key
type: varchar(80)
constraints:
nullable: false
- column:
name: status
type: varchar(16)
constraints:
nullable: false
- column:
name: created_at
type: timestamp with time zone
constraints:
nullable: false
- column:
name: updated_at
type: timestamp with time zone
constraints:
nullable: false
- addUniqueConstraint:
tableName: attachments
columnNames: object_key
constraintName: attachments_object_key_uq
- addForeignKeyConstraint:
baseTableName: attachments
baseColumnNames: chat_id
referencedTableName: chats
referencedColumnNames: id
constraintName: attachments_chat_fk
onDelete: RESTRICT
- addForeignKeyConstraint:
baseTableName: attachments
baseColumnNames: owner_id
referencedTableName: users
referencedColumnNames: id
constraintName: attachments_owner_fk
onDelete: RESTRICT
- createIndex:
indexName: attachments_chat_idx
tableName: attachments
columns:
- column:
name: chat_id
- createTable:
tableName: message_attachments
columns:
- column:
name: message_id
type: uuid
constraints:
nullable: false
- column:
name: sort_order
type: int
constraints:
nullable: false
- column:
name: attachment_id
type: uuid
constraints:
nullable: false
- addPrimaryKey:
tableName: message_attachments
columnNames: message_id, sort_order
constraintName: message_attachments_pkey
- addForeignKeyConstraint:
baseTableName: message_attachments
baseColumnNames: message_id
referencedTableName: messages
referencedColumnNames: id
constraintName: message_attachments_message_fk
onDelete: RESTRICT
- addForeignKeyConstraint:
baseTableName: message_attachments
baseColumnNames: attachment_id
referencedTableName: attachments
referencedColumnNames: id
constraintName: message_attachments_attachment_fk
onDelete: RESTRICT
@@ -0,0 +1,221 @@
databaseChangeLog:
- changeSet:
id: 007-jobs
author: lab5
changes:
- modifyDataType:
tableName: attachments
columnName: object_key
newDataType: varchar(120)
- addColumn:
tableName: attachments
columns:
- column:
name: error_code
type: varchar(64)
- column:
name: pipeline_version
type: int
defaultValueNumeric: 1
constraints:
nullable: false
- column:
name: attempt
type: int
defaultValueNumeric: 0
constraints:
nullable: false
- column:
name: job_id
type: uuid
- column:
name: staging_key
type: varchar(120)
- column:
name: lease_until
type: timestamp with time zone
- column:
name: lease_token
type: uuid
- createIndex:
indexName: attachments_lease_idx
tableName: attachments
columns:
- column:
name: status
- column:
name: lease_until
- createTable:
tableName: attachment_variants
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: attachment_variants_pkey
- column:
name: attachment_id
type: uuid
constraints:
nullable: false
- column:
name: pipeline_version
type: int
constraints:
nullable: false
- column:
name: name
type: varchar(16)
constraints:
nullable: false
- column:
name: content_type
type: varchar(32)
constraints:
nullable: false
- column:
name: size_bytes
type: bigint
constraints:
nullable: false
- column:
name: width
type: int
constraints:
nullable: false
- column:
name: height
type: int
constraints:
nullable: false
- column:
name: sha256
type: varchar(64)
constraints:
nullable: false
- column:
name: object_key
type: varchar(120)
constraints:
nullable: false
- addUniqueConstraint:
tableName: attachment_variants
columnNames: attachment_id, pipeline_version, name
constraintName: attachment_variants_identity_uq
- addUniqueConstraint:
tableName: attachment_variants
columnNames: object_key
constraintName: attachment_variants_object_key_uq
- addForeignKeyConstraint:
baseTableName: attachment_variants
baseColumnNames: attachment_id
referencedTableName: attachments
referencedColumnNames: id
constraintName: attachment_variants_attachment_fk
onDelete: RESTRICT
- createTable:
tableName: job_outbox
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: job_outbox_pkey
- column:
name: job_id
type: uuid
constraints:
nullable: false
- column:
name: attachment_id
type: uuid
constraints:
nullable: false
- column:
name: attempt
type: int
constraints:
nullable: false
- column:
name: payload
type: varchar(4000)
constraints:
nullable: false
- column:
name: available_at
type: timestamp with time zone
constraints:
nullable: false
- column:
name: published_at
type: timestamp with time zone
- column:
name: created_at
type: timestamp with time zone
constraints:
nullable: false
- createIndex:
indexName: job_outbox_due_idx
tableName: job_outbox
columns:
- column:
name: published_at
- column:
name: available_at
- createTable:
tableName: failed_jobs
columns:
- column:
name: id
type: uuid
constraints:
primaryKey: true
nullable: false
primaryKeyName: failed_jobs_pkey
- column:
name: job_id
type: uuid
constraints:
nullable: false
- column:
name: attachment_id
type: uuid
constraints:
nullable: false
- column:
name: error_code
type: varchar(64)
constraints:
nullable: false
- column:
name: attempt
type: int
constraints:
nullable: false
- column:
name: created_at
type: timestamp with time zone
constraints:
nullable: false
- column:
name: replayed_at
type: timestamp with time zone
- createTable:
tableName: demo_flags
columns:
- column:
name: name
type: varchar(64)
constraints:
primaryKey: true
nullable: false
primaryKeyName: demo_flags_pkey
- column:
name: enabled
type: boolean
constraints:
nullable: false
@@ -0,0 +1,22 @@
databaseChangeLog:
- include:
file: 001-users.yaml
relativeToChangelogFile: true
- include:
file: 002-sessions.yaml
relativeToChangelogFile: true
- include:
file: 003-chats.yaml
relativeToChangelogFile: true
- include:
file: 004-messages.yaml
relativeToChangelogFile: true
- include:
file: 005-idempotency.yaml
relativeToChangelogFile: true
- include:
file: 006-attachments.yaml
relativeToChangelogFile: true
- include:
file: 007-jobs.yaml
relativeToChangelogFile: true
@@ -0,0 +1,281 @@
package ru.mtuci.messenger;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import ru.mtuci.messenger.repository.ChatRepository;
import ru.mtuci.messenger.service.UserService;
import ru.mtuci.messenger.support.ApiClient;
import ru.mtuci.messenger.support.LabContainers;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.json.JsonMapper;
import java.util.Base64;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class ApiIntegrationTest {
@LocalServerPort
int port;
@Autowired
JsonMapper jsonMapper;
@Autowired
ChatRepository chatRepository;
@Autowired
StringRedisTemplate redis;
private ApiClient api;
@DynamicPropertySource
static void containers(DynamicPropertyRegistry registry) {
LabContainers.register(registry);
}
@BeforeEach
void client() {
api = new ApiClient(port, jsonMapper);
}
@Test
void healthAndActuatorReportProcessAndDatabase() {
ApiClient.Result live = api.get("/health/live", null);
assertEquals(200, live.status());
assertEquals("ok", live.body().get("status").asString());
assertFalse(live.requestId().isBlank());
ApiClient.Result ready = api.get("/health/ready", null);
assertEquals(200, ready.status());
assertEquals("ok", ready.body().get("status").asString());
ApiClient.Result actuator = api.get("/actuator/health/readiness", null);
assertEquals(200, actuator.status());
assertEquals("UP", actuator.body().get("status").asString());
}
@Test
void registrationConflictValidationAndRace() throws Exception {
String username = name("reg");
ApiClient.Result created = api.post("/api/v1/users", api.user(username, "Алиса", "Example-only-not-a-real-secret-123"));
assertEquals(201, created.status());
assertEquals(username, created.body().get("username").asString());
assertTrue(created.body().get("created_at").asString().endsWith("Z"));
assertFalse(created.raw().contains("password"));
assertEquals(created.requestId(), created.requestId());
ApiClient.Result conflict = api.post("/api/v1/users", api.user(username, "Алиса", "Example-only-not-a-real-secret-123"));
assertEquals(409, conflict.status());
assertEquals("conflict", conflict.body().get("error").get("code").asString());
assertEquals(conflict.requestId(), conflict.body().get("error").get("request_id").asString());
ApiClient.Result blank = api.post("/api/v1/users", api.user(name("blank"), " ", "Example-only-not-a-real-secret-123"));
assertEquals(400, blank.status());
ApiClient.Result extra = api.post("/api/v1/users", Map.of("username", name("extra"), "display_name", "Имя", "password", "Example-only-not-a-real-secret-123", "role", "admin"));
assertEquals(400, extra.status());
String raced = name("race");
CountDownLatch latch = new CountDownLatch(1);
try (ExecutorService pool = Executors.newFixedThreadPool(2)) {
List<Future<Integer>> results = List.of(
pool.submit(() -> once(latch, raced)),
pool.submit(() -> once(latch, raced))
);
latch.countDown();
int createdCount = 0;
int conflictCount = 0;
for (Future<Integer> result : results) {
int status = result.get();
if (status == 201) {
createdCount++;
} else if (status == 409) {
conflictCount++;
}
}
assertEquals(1, createdCount);
assertEquals(1, conflictCount);
}
}
@Test
void sessionLoginLogoutAndIgnoredUserHeader() {
String username = name("sess");
String password = "Example-only-not-a-real-secret-123";
JsonNode user = api.post("/api/v1/users", api.user(username, "Алиса", password)).body();
JsonNode other = api.post("/api/v1/users", api.user(name("oth"), "Боб", password)).body();
String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes());
ApiClient.Result bad = api.postBasic("/api/v1/auth/sessions", Base64.getEncoder().encodeToString((username + ":wrong-password-xx").getBytes()));
assertEquals(401, bad.status());
assertTrue(bad.authenticate().toLowerCase().startsWith("basic"));
ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic);
assertEquals(201, session.status());
assertEquals("Bearer", session.body().get("token_type").asString());
assertTrue(session.body().get("token").asString().length() >= 32);
assertNotEquals(user.get("id").asString(), session.body().get("token").asString());
assertEquals(user.get("id").asString(), session.body().get("user").get("id").asString());
String token = session.body().get("token").asString();
ApiClient.Result basicOnApi = api.postBasic("/api/v1/users/me", basic);
assertEquals(401, basicOnApi.status());
assertTrue(basicOnApi.authenticate().toLowerCase().startsWith("bearer"));
ApiClient.Result headerOnly = api.get("/api/v1/users/me", null, Map.of("X-User-Id", user.get("id").asString()));
assertEquals(401, headerOnly.status());
ApiClient.Result me = api.get("/api/v1/users/me", token, Map.of("X-User-Id", other.get("id").asString()));
assertEquals(200, me.status());
assertEquals(user.get("id").asString(), me.body().get("id").asString());
assertTrue(Boolean.TRUE.equals(redis.hasKey("session:" + UserService.tokenHash(token))));
ApiClient.Result logout = api.delete("/api/v1/auth/sessions/current", token);
assertEquals(204, logout.status());
assertTrue(logout.raw().isBlank());
assertFalse(Boolean.TRUE.equals(redis.hasKey("session:" + UserService.tokenHash(token))));
assertEquals(401, api.get("/api/v1/users/me", token).status());
assertEquals(401, api.delete("/api/v1/auth/sessions/current", token).status());
}
@Test
void chatsAreAtomicAndHiddenFromOutsiders() {
String password = "Example-only-not-a-real-secret-123";
JsonNode alice = register(name("alice"), "Алиса", password);
JsonNode bob = register(name("bob"), "Боб", password);
String aliceToken = login(alice.get("username").asString(), password);
String bobToken = login(bob.get("username").asString(), password);
String eveToken = login(register(name("eve"), "Ева", password).get("username").asString(), password);
String title = "rollback-" + UUID.randomUUID();
ApiClient.Result unknown = api.post("/api/v1/chats", aliceToken, Map.of(
"title", title,
"member_ids", List.of(alice.get("id").asString(), UUID.randomUUID().toString())
));
assertEquals(400, unknown.status());
assertEquals(0, chatRepository.countByTitle(title));
ApiClient.Result duplicates = api.post("/api/v1/chats", aliceToken, Map.of(
"title", "dup-" + UUID.randomUUID(),
"member_ids", List.of(alice.get("id").asString(), alice.get("id").asString())
));
assertEquals(400, duplicates.status());
ApiClient.Result withoutSelf = api.post("/api/v1/chats", aliceToken, Map.of(
"title", "self-" + UUID.randomUUID(),
"member_ids", List.of(bob.get("id").asString())
));
assertEquals(400, withoutSelf.status());
ApiClient.Result created = api.post("/api/v1/chats", aliceToken, Map.of(
"title", "Контрактный тест",
"member_ids", List.of(alice.get("id").asString(), bob.get("id").asString())
));
assertEquals(201, created.status());
String chatId = created.body().get("id").asString();
assertEquals(404, api.get("/api/v1/chats/" + chatId, eveToken).status());
JsonNode bobList = api.get("/api/v1/chats?limit=100", bobToken).body();
assertTrue(containsId(bobList.get("items"), chatId));
JsonNode eveList = api.get("/api/v1/chats?limit=100", eveToken).body();
assertFalse(containsId(eveList.get("items"), chatId));
ApiClient.Result second = api.post("/api/v1/chats", aliceToken, Map.of(
"title", "Второй",
"member_ids", List.of(alice.get("id").asString())
));
assertEquals(201, second.status());
JsonNode page = api.get("/api/v1/chats?limit=1", aliceToken).body();
assertEquals(1, page.get("items").size());
assertFalse(page.get("next_cursor").isNull());
JsonNode next = api.get("/api/v1/chats?limit=1&cursor=" + page.get("next_cursor").asString(), aliceToken).body();
assertEquals(1, next.get("items").size());
assertNotEquals(page.get("items").get(0).get("id").asString(), next.get("items").get(0).get("id").asString());
}
@Test
void messageHistoryUsesKeysetCursor() {
String password = "Example-only-not-a-real-secret-123";
JsonNode alice = register(name("ma"), "Алиса", password);
JsonNode bob = register(name("mb"), "Боб", password);
String aliceToken = login(alice.get("username").asString(), password);
String bobToken = login(bob.get("username").asString(), password);
String eveToken = login(register(name("me"), "Ева", password).get("username").asString(), password);
String chatId = api.post("/api/v1/chats", aliceToken, Map.of(
"title", "История",
"member_ids", List.of(alice.get("id").asString(), bob.get("id").asString())
)).body().get("id").asString();
String otherChat = api.post("/api/v1/chats", aliceToken, Map.of(
"title", "Другой",
"member_ids", List.of(alice.get("id").asString())
)).body().get("id").asString();
ApiClient.Result first = sendMessage(aliceToken, chatId, "Привет, мир");
ApiClient.Result second = sendMessage(bobToken, chatId, "Сообщение Боба");
assertEquals(201, first.status());
assertTrue(first.body().get("created_at").asString().matches("\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{6}Z"));
assertFalse(first.instanceId().isBlank());
assertEquals(first.instanceId(), api.get("/api/v1/users/me", aliceToken).instanceId());
assertEquals(alice.get("id").asString(), first.body().get("sender_id").asString());
assertEquals(201, second.status());
assertEquals(400, sendMessage(aliceToken, chatId, "").status());
assertEquals(400, sendMessage(aliceToken, chatId, " ").status());
assertEquals(404, sendMessage(eveToken, chatId, "чужое").status());
assertEquals(400, api.get("/api/v1/chats/" + chatId + "/messages?limit=0", aliceToken).status());
JsonNode page = api.get("/api/v1/chats/" + chatId + "/messages?limit=1", bobToken).body();
assertEquals(1, page.get("items").size());
assertEquals(first.body().get("id").asString(), page.get("items").get(0).get("id").asString());
String cursor = page.get("next_cursor").asString();
JsonNode rest = api.get("/api/v1/chats/" + chatId + "/messages?limit=1&cursor=" + cursor, aliceToken).body();
assertEquals(second.body().get("id").asString(), rest.get("items").get(0).get("id").asString());
assertTrue(rest.get("next_cursor").isNull());
assertEquals(400, api.get("/api/v1/chats/" + otherChat + "/messages?cursor=" + cursor, aliceToken).status());
}
private int once(CountDownLatch latch, String username) throws Exception {
latch.await();
return api.post("/api/v1/users", api.user(username, "Гонка", "Example-only-not-a-real-secret-123")).status();
}
private JsonNode register(String username, String displayName, String password) {
ApiClient.Result result = api.post("/api/v1/users", api.user(username, displayName, password));
assertEquals(201, result.status());
return result.body();
}
private String login(String username, String password) {
String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes());
ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic);
assertEquals(201, session.status());
return session.body().get("token").asString();
}
private static boolean containsId(JsonNode items, String id) {
for (JsonNode item : items) {
if (id.equals(item.get("id").asString())) {
return true;
}
}
return false;
}
private ApiClient.Result sendMessage(String token, String chatId, String text) {
return api.post("/api/v1/chats/" + chatId + "/messages", token, Map.of("text", text),
Map.of("Idempotency-Key", UUID.randomUUID().toString()));
}
private static String name(String prefix) {
return (prefix + UUID.randomUUID().toString().replace("-", "")).substring(0, 20);
}
}
@@ -0,0 +1,240 @@
package ru.mtuci.messenger;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import ru.mtuci.messenger.storage.ObjectStorage;
import ru.mtuci.messenger.storage.OrphanSweeper;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import ru.mtuci.messenger.support.ApiClient;
import ru.mtuci.messenger.support.LabContainers;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.json.JsonMapper;
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.MessageDigest;
import java.util.Base64;
import java.util.HexFormat;
import java.util.List;
import java.util.Map;
import java.time.Instant;
import java.util.UUID;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class AttachmentIntegrationTest {
@LocalServerPort
int port;
@Autowired
JsonMapper jsonMapper;
@Autowired
ObjectStorage storage;
@Autowired
OrphanSweeper sweeper;
private ApiClient api;
private final HttpClient http = HttpClient.newHttpClient();
@DynamicPropertySource
static void containers(DynamicPropertyRegistry registry) {
LabContainers.register(registry);
}
@BeforeEach
void client() {
api = new ApiClient(port, jsonMapper);
}
@Test
void uploadIsPrivateAndReadableByAnotherMember() throws Exception {
String password = "Example-only-not-a-real-secret-123";
JsonNode alice = register("at", password);
JsonNode bob = register("bt", password);
JsonNode eve = register("et", password);
String aliceToken = login(alice.get("username").asString(), password);
String bobToken = login(bob.get("username").asString(), password);
String eveToken = login(eve.get("username").asString(), password);
String chatId = chat(aliceToken, List.of(alice.get("id").asString(), bob.get("id").asString()));
String otherChat = chat(aliceToken, List.of(alice.get("id").asString()));
byte[] png = Files.readAllBytes(Path.of("../tests/fixtures/sample.png"));
HttpResponse<String> created = upload(aliceToken, chatId, "sample.png", "image/png", png);
assertEquals(202, created.statusCode());
JsonNode attachment = jsonMapper.readTree(created.body());
assertEquals("queued", attachment.get("status").asString());
assertEquals(sha256(png), attachment.get("sha256").asString());
assertEquals(png.length, attachment.get("size_bytes").asInt());
assertEquals(false, attachment.has("object_key"));
String attachmentId = attachment.get("id").asString();
ApiClient.Result message = api.post("/api/v1/chats/" + chatId + "/messages", aliceToken,
Map.of("text", "", "attachment_ids", List.of(attachmentId)),
Map.of("Idempotency-Key", UUID.randomUUID().toString()));
assertEquals(201, message.status());
assertEquals(attachmentId, message.body().get("attachment_ids").get(0).asString());
JsonNode ready = waitReady(bobToken, attachmentId);
JsonNode thumbnail = variant(ready, "thumbnail");
assertEquals(256, thumbnail.get("width").asInt());
assertEquals(160, thumbnail.get("height").asInt());
assertEquals("image/jpeg", thumbnail.get("content_type").asString());
JsonNode crop = variant(ready, "crop");
assertEquals(128, crop.get("width").asInt());
assertEquals(128, crop.get("height").asInt());
JsonNode optimized = variant(ready, "optimized");
assertEquals("image/webp", optimized.get("content_type").asString());
assertEquals(320, optimized.get("width").asInt());
assertEquals(200, optimized.get("height").asInt());
HttpResponse<byte[]> download = download(bobToken, attachmentId);
assertEquals(200, download.statusCode());
assertEquals("image/png", download.headers().firstValue("Content-Type").orElse(""));
assertArrayEquals(png, download.body());
assertEquals(404, api.get("/api/v1/attachments/" + attachmentId, eveToken).status());
assertEquals(404, download(eveToken, attachmentId).statusCode());
HttpResponse<String> badName = upload(aliceToken, chatId, "../secret.png", "image/png", png);
assertEquals(400, badName.statusCode());
HttpResponse<String> spoofed = upload(aliceToken, chatId, "photo.png", "image/png", new byte[] {1, 2, 3, 4});
assertEquals(415, spoofed.statusCode());
ApiClient.Result foreign = api.post("/api/v1/chats/" + otherChat + "/messages", aliceToken,
Map.of("text", "", "attachment_ids", List.of(attachmentId)),
Map.of("Idempotency-Key", UUID.randomUUID().toString()));
assertEquals(404, foreign.status());
assertEquals("not_found", foreign.body().get("error").get("code").asString());
ApiClient.Result missing = api.post("/api/v1/chats/" + chatId + "/messages", aliceToken,
Map.of("text", "", "attachment_ids", List.of(UUID.randomUUID().toString())),
Map.of("Idempotency-Key", UUID.randomUUID().toString()));
assertEquals(404, missing.status());
HttpResponse<String> corrupt = upload(aliceToken, chatId, "broken.png", "image/png",
new byte[] {(byte) 137, 80, 78, 71, 13, 10, 26, 10, 0, 1, 2});
assertEquals(415, corrupt.statusCode());
assertEquals("unsupported_media_type", jsonMapper.readTree(corrupt.body()).get("error").get("code").asString());
byte[] jpeg = new byte[] {
(byte) 0xff, (byte) 0xd8,
(byte) 0xff, (byte) 0xc0, 0x00, 0x0b, 0x08, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x11, 0x00,
(byte) 0xff, (byte) 0xd9
};
HttpResponse<String> mime = upload(aliceToken, chatId, "photo.png", "image/png", jpeg);
assertEquals(415, mime.statusCode());
HttpResponse<String> tooBig = upload(aliceToken, chatId, "big.png", "image/png", new byte[10_485_761]);
assertEquals(413, tooBig.statusCode());
assertEquals("payload_too_large", jsonMapper.readTree(tooBig.body()).get("error").get("code").asString());
assertTrue(tooBig.statusCode() < 500);
}
@Test
void sweepDropsOrphanAndKeepsReferencedObject() throws Exception {
String password = "Example-only-not-a-real-secret-123";
JsonNode alice = register("sw", password);
String token = login(alice.get("username").asString(), password);
String chatId = chat(token, List.of(alice.get("id").asString()));
byte[] png = Files.readAllBytes(Path.of("../tests/fixtures/sample.png"));
HttpResponse<String> created = upload(token, chatId, "sample.png", "image/png", png);
assertEquals(202, created.statusCode());
String id = jsonMapper.readTree(created.body()).get("id").asString();
waitReady(token, id);
storage.put("originals/orphan-" + UUID.randomUUID(), new byte[] {9, 9, 9}, "application/octet-stream");
int removed = sweeper.sweepOlderThan(Instant.now().plusSeconds(120));
assertTrue(removed >= 1);
assertEquals(200, download(token, id).statusCode());
assertTrue(storage.list("originals/").stream().noneMatch(object -> object.key().contains("orphan-")));
}
private JsonNode waitReady(String token, String id) throws Exception {
long deadline = System.nanoTime() + java.time.Duration.ofSeconds(40).toNanos();
JsonNode last = null;
while (System.nanoTime() < deadline) {
ApiClient.Result result = api.get("/api/v1/attachments/" + id, token);
assertEquals(200, result.status());
last = result.body();
String status = last.get("status").asString();
if ("ready".equals(status)) {
return last;
}
if ("failed".equals(status)) {
fail("attachment failed: " + last.get("error_code"));
}
Thread.sleep(200);
}
fail("attachment did not become ready: " + last);
return last;
}
private static JsonNode variant(JsonNode attachment, String name) {
for (JsonNode variant : attachment.get("variants")) {
if (name.equals(variant.get("name").asString())) {
return variant;
}
}
fail("missing variant " + name);
return null;
}
private HttpResponse<String> upload(String token, String chatId, String filename, String type, byte[] body) throws Exception {
String name = URLEncoder.encode(filename, StandardCharsets.UTF_8);
HttpRequest request = HttpRequest.newBuilder(URI.create(base() + "/api/v1/chats/" + chatId + "/attachments?filename=" + name))
.header("Authorization", "Bearer " + token)
.header("Content-Type", type)
.POST(HttpRequest.BodyPublishers.ofByteArray(body))
.build();
return http.send(request, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
}
private HttpResponse<byte[]> download(String token, String attachmentId) throws Exception {
HttpRequest request = HttpRequest.newBuilder(URI.create(base() + "/api/v1/attachments/" + attachmentId + "/content"))
.header("Authorization", "Bearer " + token)
.GET()
.build();
return http.send(request, HttpResponse.BodyHandlers.ofByteArray());
}
private String base() {
return "http://127.0.0.1:" + port;
}
private String chat(String token, List<String> members) {
return api.post("/api/v1/chats", token, Map.of("title", "Вложения", "member_ids", members)).body().get("id").asString();
}
private JsonNode register(String prefix, String password) {
String username = (prefix + UUID.randomUUID().toString().replace("-", "")).substring(0, 20);
ApiClient.Result result = api.post("/api/v1/users", api.user(username, "Алиса", password));
assertEquals(201, result.status());
return result.body();
}
private String login(String username, String password) {
String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes(StandardCharsets.UTF_8));
ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic);
assertEquals(201, session.status());
return session.body().get("token").asString();
}
private static String sha256(byte[] bytes) throws Exception {
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes));
}
}
@@ -0,0 +1,75 @@
package ru.mtuci.messenger;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Timeout;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.postgresql.PostgreSQLContainer;
import ru.mtuci.messenger.support.ApiClient;
import ru.mtuci.messenger.support.LabContainers;
import tools.jackson.databind.json.JsonMapper;
import java.util.UUID;
import static org.junit.jupiter.api.Assertions.assertEquals;
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = {
"spring.data.redis.host=127.0.0.1",
"spring.data.redis.port=1",
"spring.data.redis.timeout=200ms",
"spring.data.redis.connect-timeout=200ms",
"spring.datasource.hikari.connection-timeout=1000",
"spring.datasource.hikari.validation-timeout=500",
"spring.datasource.hikari.maximum-pool-size=2"
})
@DirtiesContext
class DatabaseOutageIntegrationTest {
private static final PostgreSQLContainer POSTGRES = new PostgreSQLContainer("postgres:16.10");
static {
POSTGRES.start();
}
@LocalServerPort
int port;
@Autowired
JsonMapper jsonMapper;
private ApiClient api;
@DynamicPropertySource
static void database(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", POSTGRES::getJdbcUrl);
registry.add("spring.datasource.username", POSTGRES::getUsername);
registry.add("spring.datasource.password", POSTGRES::getPassword);
LabContainers.registerS3(registry);
}
@BeforeEach
void client() {
api = new ApiClient(port, jsonMapper);
}
@Test
@Timeout(40)
void readinessAndWritesFailWhenDatabaseStops() {
assertEquals(200, api.get("/health/ready", null).status());
POSTGRES.stop();
ApiClient.Result ready = api.get("/health/ready", null);
assertEquals(503, ready.status());
assertEquals("unavailable", ready.body().get("error").get("code").asString());
ApiClient.Result register = api.post("/api/v1/users", api.user(
("db" + UUID.randomUUID().toString().replace("-", "")).substring(0, 16),
"База",
"Example-only-not-a-real-secret-123"
));
assertEquals(503, register.status());
}
}
@@ -0,0 +1,65 @@
package ru.mtuci.messenger;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import ru.mtuci.messenger.support.LabContainers;
import java.nio.file.Files;
import java.nio.file.Path;
import java.sql.Timestamp;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
@SpringBootTest
class HistoryIndexIntegrationTest {
@Autowired
JdbcTemplate jdbc;
@DynamicPropertySource
static void containers(DynamicPropertyRegistry registry) {
LabContainers.register(registry);
}
@Test
void historyPageUsesCompositeIndex() throws Exception {
UUID userId = UUID.randomUUID();
UUID chatId = UUID.randomUUID();
Instant created = Instant.parse("2026-01-01T00:00:00Z");
jdbc.update("insert into users (id, username, display_name, password_hash, created_at) values (?, ?, ?, ?, ?)",
userId, ("idx" + userId.toString().replace("-", "")).substring(0, 16), "Индекс", "hash", Timestamp.from(created));
jdbc.update("insert into chats (id, title, created_at) values (?, ?, ?)", chatId, "Индекс", Timestamp.from(created));
jdbc.update("insert into chat_members (chat_id, user_id, ordinal) values (?, ?, 0)", chatId, userId);
List<Object[]> batch = new ArrayList<>();
for (int index = 0; index < 10_000; index++) {
batch.add(new Object[]{UUID.randomUUID(), chatId, userId, "m" + index, Timestamp.from(created.plusMillis(index))});
}
jdbc.batchUpdate("insert into messages (id, chat_id, sender_id, text, created_at) values (?, ?, ?, ?, ?)", batch);
jdbc.execute("analyze messages");
String plan = String.join("\n", jdbc.query("""
explain (analyze, buffers)
select id from messages
where chat_id = ?
and (created_at, id) > (?, ?)
order by created_at asc, id asc
limit 50
""", (rs, row) -> rs.getString(1), chatId, Timestamp.from(created.plusMillis(5000)), UUID.randomUUID()));
Path evidence = Files.isDirectory(Path.of("evidence")) ? Path.of("evidence") : Path.of("..", "evidence");
Files.createDirectories(evidence);
Files.writeString(evidence.resolve("explain-messages.txt"), plan + "\n");
assertTrue(plan.contains("messages_history_idx"), plan);
assertTrue(plan.contains("Index Cond"), plan);
assertFalse(plan.contains("Seq Scan on messages"), plan);
assertFalse(plan.contains("Rows Removed by Filter"), plan);
}
}
@@ -0,0 +1,164 @@
package ru.mtuci.messenger;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import ru.mtuci.messenger.repository.MessageRepository;
import ru.mtuci.messenger.support.ApiClient;
import ru.mtuci.messenger.support.LabContainers;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.json.JsonMapper;
import java.util.ArrayList;
import java.util.Base64;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class IdempotencyIntegrationTest {
@LocalServerPort
int port;
@Autowired
JsonMapper jsonMapper;
@Autowired
MessageRepository messages;
private ApiClient api;
@DynamicPropertySource
static void containers(DynamicPropertyRegistry registry) {
LabContainers.register(registry);
}
@BeforeEach
void client() {
api = new ApiClient(port, jsonMapper);
}
@Test
void replayConflictAndRejectedKey() {
String password = "Example-only-not-a-real-secret-123";
JsonNode alice = register(name("ia"), password);
String token = login(alice.get("username").asString(), password);
String chatId = chat(token, List.of(alice.get("id").asString()));
String key = "same-" + UUID.randomUUID();
ApiClient.Result rejected = send(token, chatId, "ещё не записано", key.replace("same", "bad key"));
assertEquals(400, rejected.status());
assertFalse(rejected.instanceId().isBlank());
assertEquals(400, api.post("/api/v1/chats/" + chatId + "/messages", token, Map.of("text", "без ключа")).status());
String recover = "recover-" + UUID.randomUUID();
assertEquals(400, send(token, chatId, "", recover).status());
ApiClient.Result first = send(token, chatId, "Привет, мир", recover);
assertEquals(201, first.status());
ApiClient.Result replay = send(token, chatId, "Привет, мир", recover);
assertEquals(201, replay.status());
assertEquals(first.body(), replay.body());
assertEquals(409, send(token, chatId, "другой body", recover).status());
assertEquals(1, messages.countByChatId(UUID.fromString(chatId)));
ApiClient.Result fresh = send(token, chatId, "Привет, мир", key);
assertEquals(201, fresh.status());
assertNotEquals(first.body().get("id").asString(), fresh.body().get("id").asString());
}
@Test
void sameKeyIsIndependentAcrossUsersAndChats() {
String password = "Example-only-not-a-real-secret-123";
JsonNode alice = register(name("ib"), password);
JsonNode bob = register(name("ic"), password);
String aliceToken = login(alice.get("username").asString(), password);
String bobToken = login(bob.get("username").asString(), password);
String shared = chat(aliceToken, List.of(alice.get("id").asString(), bob.get("id").asString()));
String other = chat(aliceToken, List.of(alice.get("id").asString()));
String key = "shared-" + UUID.randomUUID();
ApiClient.Result aliceMessage = send(aliceToken, shared, "общий ключ", key);
ApiClient.Result bobMessage = send(bobToken, shared, "общий ключ", key);
ApiClient.Result otherChat = send(aliceToken, other, "общий ключ", key);
assertEquals(201, aliceMessage.status());
assertEquals(201, bobMessage.status());
assertEquals(201, otherChat.status());
assertNotEquals(aliceMessage.body().get("id").asString(), bobMessage.body().get("id").asString());
assertNotEquals(aliceMessage.body().get("id").asString(), otherChat.body().get("id").asString());
}
@Test
void parallelSameKeyCreatesOneMessage() throws Exception {
String password = "Example-only-not-a-real-secret-123";
JsonNode alice = register(name("ip"), password);
String token = login(alice.get("username").asString(), password);
String chatId = chat(token, List.of(alice.get("id").asString()));
String key = "parallel-" + UUID.randomUUID();
int threads = 8;
ExecutorService pool = Executors.newFixedThreadPool(threads);
CountDownLatch start = new CountDownLatch(1);
try {
List<Future<ApiClient.Result>> futures = new ArrayList<>();
for (int i = 0; i < threads; i++) {
futures.add(pool.submit(() -> {
start.await();
return send(token, chatId, "одно сообщение", key);
}));
}
start.countDown();
JsonNode body = null;
for (Future<ApiClient.Result> future : futures) {
ApiClient.Result result = future.get();
assertEquals(201, result.status());
if (body == null) {
body = result.body();
} else {
assertEquals(body, result.body());
}
}
} finally {
pool.shutdownNow();
}
assertEquals(1, messages.countByChatId(UUID.fromString(chatId)));
}
private ApiClient.Result send(String token, String chatId, String text, String key) {
return api.post("/api/v1/chats/" + chatId + "/messages", token, Map.of("text", text),
Map.of("Idempotency-Key", key));
}
private String chat(String token, List<String> members) {
return api.post("/api/v1/chats", token, Map.of("title", "Идемпотентность", "member_ids", members))
.body().get("id").asString();
}
private JsonNode register(String username, String password) {
ApiClient.Result result = api.post("/api/v1/users", api.user(username, "Алиса", password));
assertEquals(201, result.status());
return result.body();
}
private String login(String username, String password) {
String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes());
ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic);
assertEquals(201, session.status());
return session.body().get("token").asString();
}
private static String name(String prefix) {
return (prefix + UUID.randomUUID().toString().replace("-", "")).substring(0, 20);
}
}
@@ -0,0 +1,62 @@
package ru.mtuci.messenger;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import ru.mtuci.messenger.support.ApiClient;
import ru.mtuci.messenger.support.LabContainers;
import tools.jackson.databind.json.JsonMapper;
import java.util.Base64;
import java.util.UUID;
import static org.junit.jupiter.api.Assertions.assertEquals;
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = {
"spring.data.redis.host=127.0.0.1",
"spring.data.redis.port=1",
"spring.data.redis.timeout=200ms",
"spring.data.redis.connect-timeout=200ms"
})
class RedisOutageIntegrationTest {
@LocalServerPort
int port;
@Autowired
JsonMapper jsonMapper;
private ApiClient api;
@DynamicPropertySource
static void database(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", LabContainers.POSTGRES::getJdbcUrl);
registry.add("spring.datasource.username", LabContainers.POSTGRES::getUsername);
registry.add("spring.datasource.password", LabContainers.POSTGRES::getPassword);
LabContainers.registerS3(registry);
}
@BeforeEach
void client() {
api = new ApiClient(port, jsonMapper);
}
@Test
void databaseSessionWorksWhenRedisIsDown() {
assertEquals(200, api.get("/health/ready", null).status());
String username = ("red" + UUID.randomUUID().toString().replace("-", "")).substring(0, 16);
String password = "Example-only-not-a-real-secret-123";
assertEquals(201, api.post("/api/v1/users", api.user(username, "Кэш", password)).status());
String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes());
ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic);
assertEquals(201, session.status());
ApiClient.Result me = api.get("/api/v1/users/me", session.body().get("token").asString());
assertEquals(200, me.status());
assertEquals(username, me.body().get("username").asString());
assertEquals(200, api.get("/actuator/health", null).status());
}
}
@@ -0,0 +1,53 @@
package ru.mtuci.messenger;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import ru.mtuci.messenger.support.ApiClient;
import ru.mtuci.messenger.support.LabContainers;
import tools.jackson.databind.json.JsonMapper;
import java.util.Base64;
import java.util.UUID;
import static org.junit.jupiter.api.Assertions.assertEquals;
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = "app.session-ttl=2s")
class SessionExpiryIntegrationTest {
@LocalServerPort
int port;
@Autowired
JsonMapper jsonMapper;
private ApiClient api;
@DynamicPropertySource
static void containers(DynamicPropertyRegistry registry) {
LabContainers.register(registry);
}
@BeforeEach
void client() {
api = new ApiClient(port, jsonMapper);
}
@Test
void expiredTokenIsRejected() throws Exception {
String username = ("exp" + UUID.randomUUID().toString().replace("-", "")).substring(0, 16);
String password = "Example-only-not-a-real-secret-123";
assertEquals(201, api.post("/api/v1/users", api.user(username, "Срок", password)).status());
String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes());
String token = api.postBasic("/api/v1/auth/sessions", basic).body().get("token").asString();
assertEquals(200, api.get("/api/v1/users/me", token).status());
Thread.sleep(2500);
ApiClient.Result expired = api.get("/api/v1/users/me", token);
assertEquals(401, expired.status());
assertEquals("unauthorized", expired.body().get("error").get("code").asString());
}
}
@@ -0,0 +1,68 @@
package ru.mtuci.messenger.service;
import org.junit.jupiter.api.Test;
import ru.mtuci.messenger.exception.PayloadTooLargeException;
import ru.mtuci.messenger.exception.UnsupportedMediaException;
import java.nio.file.Files;
import java.nio.file.Path;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
class ImageFilesTest {
@Test
void readsFixtureWithoutDecodingPixels() throws Exception {
byte[] png = Files.readAllBytes(Path.of("../tests/fixtures/sample.png"));
ImageFiles.Image image = ImageFiles.inspect(png, "image/png", 40_000_000);
assertEquals("image/png", image.contentType());
assertEquals(320, image.width());
assertEquals(200, image.height());
assertArrayEquals(png, ImageFiles.readLimited(new java.io.ByteArrayInputStream(png), 10_485_760));
}
@Test
void rejectsSpoofedMimeCorruptAndPixelBomb() {
byte[] jpeg = new byte[] {
(byte) 0xff, (byte) 0xd8,
(byte) 0xff, (byte) 0xc0, 0x00, 0x0b, 0x08, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x11, 0x00,
(byte) 0xff, (byte) 0xd9
};
assertEquals(1, ImageFiles.inspect(jpeg, "image/jpeg", 40_000_000).width());
assertThrows(UnsupportedMediaException.class, () -> ImageFiles.inspect(jpeg, "image/png", 40_000_000));
assertThrows(UnsupportedMediaException.class, () -> ImageFiles.inspect(new byte[] {1, 2, 3}, "image/png", 40_000_000));
byte[] bomb = png(8_000, 8_000);
assertThrows(PayloadTooLargeException.class, () -> ImageFiles.inspect(bomb, "image/png", 40_000_000));
byte[] huge = new byte[8];
assertThrows(PayloadTooLargeException.class, () -> ImageFiles.readLimited(new java.io.ByteArrayInputStream(huge), 7));
}
private static byte[] png(int width, int height) {
byte[] data = new byte[8 + 12 + 13 + 12];
byte[] signature = new byte[] {(byte) 137, 80, 78, 71, 13, 10, 26, 10};
System.arraycopy(signature, 0, data, 0, 8);
data[11] = 13;
data[12] = 'I';
data[13] = 'H';
data[14] = 'D';
data[15] = 'R';
putInt(data, 16, width);
putInt(data, 20, height);
data[24] = 8;
data[25] = 2;
data[37] = 'I';
data[38] = 'E';
data[39] = 'N';
data[40] = 'D';
return data;
}
private static void putInt(byte[] data, int offset, int value) {
data[offset] = (byte) (value >>> 24);
data[offset + 1] = (byte) (value >>> 16);
data[offset + 2] = (byte) (value >>> 8);
data[offset + 3] = (byte) value;
}
}
@@ -0,0 +1,75 @@
package ru.mtuci.messenger.service;
import org.junit.jupiter.api.Test;
import javax.imageio.ImageIO;
import java.awt.Color;
import java.awt.image.BufferedImage;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
class ImageVariantsTest {
@Test
void fixtureSizeBecomesContractDerivatives() throws Exception {
byte[] png = png(320, 200);
Map<String, ImageVariants.VariantBytes> variants = ImageVariants.render(png, 0.85f, 0.8f).stream()
.collect(Collectors.toMap(ImageVariants.VariantBytes::name, Function.identity()));
ImageVariants.VariantBytes thumbnail = variants.get("thumbnail");
assertEquals("image/jpeg", thumbnail.contentType());
assertEquals(256, thumbnail.width());
assertEquals(160, thumbnail.height());
assertTrue(thumbnail.bytes()[0] == (byte) 0xff && thumbnail.bytes()[1] == (byte) 0xd8);
BufferedImage decoded = ImageIO.read(new ByteArrayInputStream(thumbnail.bytes()));
assertEquals(256, decoded.getWidth());
assertEquals(160, decoded.getHeight());
assertEquals(128, variants.get("crop").width());
assertEquals(128, variants.get("crop").height());
ImageVariants.VariantBytes optimized = variants.get("optimized");
assertEquals("image/webp", optimized.contentType());
assertEquals(320, optimized.width());
assertEquals(200, optimized.height());
BufferedImage webp = ImageIO.read(new ByteArrayInputStream(optimized.bytes()));
assertEquals(320, webp.getWidth());
assertEquals(200, webp.getHeight());
}
@Test
void smallImageIsNotUpscaledExceptCenteredCrop() throws Exception {
Map<String, ImageVariants.VariantBytes> variants = ImageVariants.render(png(20, 10), 0.85f, 0.8f).stream()
.collect(Collectors.toMap(ImageVariants.VariantBytes::name, Function.identity()));
assertEquals(20, variants.get("thumbnail").width());
assertEquals(10, variants.get("thumbnail").height());
assertEquals(20, variants.get("optimized").width());
assertEquals(128, variants.get("crop").width());
assertEquals(128, variants.get("crop").height());
}
@Test
void fitUsesFloorAndDoesNotEnlarge() {
assertEquals(256, ImageVariants.fitted(320, 200, 256, false)[0]);
assertEquals(160, ImageVariants.fitted(320, 200, 256, false)[1]);
assertEquals(320, ImageVariants.fitted(320, 200, 1280, false)[0]);
assertEquals(10, ImageVariants.fitted(10, 10, 256, false)[0]);
}
private static byte[] png(int width, int height) throws Exception {
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
image.setRGB(x, y, new Color(x * 255 / width, y * 255 / height, 80).getRGB());
}
}
ByteArrayOutputStream out = new ByteArrayOutputStream();
ImageIO.write(image, "png", out);
return out.toByteArray();
}
}
@@ -0,0 +1,24 @@
package ru.mtuci.messenger.service;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
class RetryDelayTest {
@Test
void backoffGrowsThenStaysInsideHalfToFullWindow() {
assertEquals(1000, RetryDelay.exponentialMillis(1, 1000, 30_000));
assertEquals(2000, RetryDelay.exponentialMillis(2, 1000, 30_000));
assertEquals(16_000, RetryDelay.exponentialMillis(5, 1000, 30_000));
assertEquals(30_000, RetryDelay.exponentialMillis(20, 1000, 30_000));
for (int attempt = 1; attempt <= 8; attempt++) {
long exp = RetryDelay.exponentialMillis(attempt, 1000, 30_000);
for (int i = 0; i < 20; i++) {
long delay = RetryDelay.millis(attempt, 1000, 30_000);
assertTrue(delay >= exp / 2 && delay <= exp, "attempt " + attempt + " delay " + delay);
}
}
}
}

Some files were not shown because too many files have changed in this diff Show More