From 805785282300450a6202536cca53e20df60d838e Mon Sep 17 00:00:00 2001 From: ivanstorozhev Date: Thu, 12 Nov 2026 19:33:05 +0300 Subject: [PATCH] =?UTF-8?q?=D0=9E=D1=80=D0=B8=D0=B3=D0=B8=D0=BD=D0=B0?= =?UTF-8?q?=D0=BB=20=D0=BF=D1=80=D0=B8=D0=BD=D0=B8=D0=BC=D0=B0=D0=B5=D1=82?= =?UTF-8?q?=D1=81=D1=8F=20=D0=B2=20S3=20=D0=B8=20=D0=B2=20=D0=BE=D0=B4?= =?UTF-8?q?=D0=BD=D1=83=20=D1=82=D1=80=D0=B0=D0=BD=D0=B7=D0=B0=D0=BA=D1=86?= =?UTF-8?q?=D0=B8=D1=8E=20=D1=81=20outbox,=20=D0=BF=D0=BE=D1=8D=D1=82?= =?UTF-8?q?=D0=BE=D0=BC=D1=83=20=D0=BE=D1=82=D0=B2=D0=B5=D1=82=20202=20?= =?UTF-8?q?=D1=83=D0=B6=D0=B5=20=D0=BD=D0=B5=20=D0=B7=D0=B0=D0=B2=D0=B8?= =?UTF-8?q?=D1=81=D0=B8=D1=82=20=D0=BE=D1=82=20=D0=B1=D1=80=D0=BE=D0=BA?= =?UTF-8?q?=D0=B5=D1=80=D0=B0.=20Worker=20=D0=BA=D0=BE=D0=BC=D0=BC=D0=B8?= =?UTF-8?q?=D1=82=D0=B8=D1=82=20=D1=81=D0=BC=D0=B5=D1=89=D0=B5=D0=BD=D0=B8?= =?UTF-8?q?=D0=B5=20Kafka=20=D1=82=D0=BE=D0=BB=D1=8C=D0=BA=D0=BE=20=D0=BF?= =?UTF-8?q?=D0=BE=D1=81=D0=BB=D0=B5=20=D0=B7=D0=B0=D0=BF=D0=B8=D1=81=D0=B8?= =?UTF-8?q?=20=D0=B2=D0=B0=D1=80=D0=B8=D0=B0=D0=BD=D1=82=D0=BE=D0=B2=20?= =?UTF-8?q?=D0=B2=20=D1=84=D0=B8=D0=BA=D1=81=D0=B8=D1=80=D0=BE=D0=B2=D0=B0?= =?UTF-8?q?=D0=BD=D0=BD=D1=8B=D0=B5=20=D0=BA=D0=BB=D1=8E=D1=87=D0=B8,=20?= =?UTF-8?q?=D1=82=D0=B0=D0=BA=20=D1=87=D1=82=D0=BE=20=D0=BF=D0=BE=D0=B2?= =?UTF-8?q?=D1=82=D0=BE=D1=80=D0=BD=D0=B0=D1=8F=20=D0=B4=D0=BE=D1=81=D1=82?= =?UTF-8?q?=D0=B0=D0=B2=D0=BA=D0=B0=20=D0=BD=D0=B5=20=D1=81=D0=BE=D0=B7?= =?UTF-8?q?=D0=B4=D0=B0=D1=91=D1=82=20=D0=B2=D1=82=D0=BE=D1=80=D0=BE=D0=B9?= =?UTF-8?q?=20=D0=BD=D0=B0=D0=B1=D0=BE=D1=80.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Автор / группа: ivanstorozhev, БВТ2304 Целевая оценка (3/4/5): 4 Трек 5, если заявлен: не заявлен Что реализовано и что перенесено: из lab4 перенесены Spring Boot API, PostgreSQL, Redis-кэш сессий, Argon2id, Liquibase через одноразовый сервис migrate, HAProxy без sticky session, две реплики, X-Instance-Id, обязательный Idempotency-Key, scripts/stop-one.sh и scripts/restart.sh, приватный SeaweedFS, таблица attachments и скачивание оригинала через API. Ключ объекта назначает сервер. У API отдельный ключ S3 с правами Read/Write/List; Admin используется только чтобы создать bucket. Объект без строки в базе старше двух минут удаляет фоновая очистка. Добавлены ответ 202, staging, transactional outbox, отдельный worker, Kafka 3.9.1 (KRaft, volume kafka-data), состояния queued/processing/ready/failed, JPEG thumbnail и crop, lossy WebP, retry с backoff и jitter, топик attachments.failed, явный replay и lease. Сообщение брокера содержит только идентификаторы. Подтверждение записи брокером (acks=all) и commit смещения потребителя разделены. Команды запуска из чистого clone: cp .env.example .env docker compose --env-file .env config --quiet docker compose --env-file .env up -d --build curl -fsS http://127.0.0.1:8080/health/ready make test make test-persistence ACTION_SCRIPT=scripts/restart.sh make test-failover ACTION_SCRIPT=scripts/stop-one.sh docker compose --env-file .env start api python3 scripts/grade4-evidence.py Результаты проверок и ссылка на REPORT.md: подробности и числа в REPORT.md make check — PASS: offline skeleton structure, local refs and Python syntax. make test — PASS: lab5 public API scenario. make test-persistence ACTION_SCRIPT=scripts/restart.sh — PASS: persistence preserved confirmed data and existing session. Хук перезапускает и kafka. make test-failover ACTION_SCRIPT=scripts/stop-one.sh — PASS: две реплики до отказа, восстановление записи за 1.795 с, 117 успешных записей и 1 ошибка перехода. python3 scripts/grade4-evidence.py — PASS. Worker остановлен: после рестарта API вложение осталось queued, затем стало ready; thumbnail JPEG 256×160, crop JPEG 128×128, optimized WebP 320×200, оригинал 111817 байт совпал. Конец топика attachments.process вырос 2→3 и после restart kafka остался 3. Пауза outbox пережила kill API, unpublished=1, затем тот же id стал ready. System.exit после записи объектов: RestartCount 0→1, один набор вариантов. Постоянный сбой processing_failed, топик attachments.failed, replay доводит то же вложение до ready, второй replay остаётся ready. mvn -q test в backend/ — код выхода 0, 22 теста: 202 и варианты, слишком большой файл 413, порча и подмена MIME 415, чужой id и другой чат 404, сирота в S3 удаляется. Анонимный GET объекта в сети стенда — HTTP 403. Ограничения / известные ошибки: трек 5 не делался. Один Docker-хост, единственные PostgreSQL, Redis, SeaweedFS, Kafka и HAProxy остаются точками отказа. Replication factor топиков равен 1. Доставка at least once: повтор перезаписывает те же ключи, но это не exactly-once на стороне S3. Если процесс умер после PUT staging и до COMMIT, объект без строки живёт не дольше grace period, затем удаляется. Staging failed-задания хранится сутки; позже replay уже не находит байты. Пока HAProxy не снял backend, часть запросов на останавливаемую реплику завершается ошибкой соединения; повтор с тем же Idempotency-Key безопасен для отправки сообщения. Источники и использование ИИ: контракт contracts/openapi.json, contracts/README.md, contracts/attachment-states.md, contracts/jobs.schema.json; код lab4; Spring Boot 4.1.1, Spring Data JPA, spring-security-crypto, Spring Data Redis, Spring Kafka, Liquibase, Actuator, MapStruct 1.6.3, OpenAPI Generator 7.17.0, AWS SDK for Java 2.31.50, Thumbnailator 0.4.20, webp-imageio 0.11.0, Bouncy Castle, PostgreSQL 16.10, Redis 7.4.6, HAProxy 3.2.6, SeaweedFS 4.52, Apache Kafka 3.9.1, Testcontainers. Реализация собрана в Cursor. Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами. Контракт и публичные тесты не ослаблены. make check и make test выполнены; приложен реальный вывод. Сценарии отказов своего уровня воспроизведены. Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens. README задания сохранён, REPORT.md содержит инструкцию именно моего решения. Co-authored-by: Cursor --- .env.example | 36 +- Dockerfile | 22 + REPORT.md | 117 +++++- backend/pom.xml | 218 ++++++++++ .../mtuci/messenger/MessengerApplication.java | 20 + .../api/attachment/AttachmentController.java | 52 +++ .../messenger/api/chat/ChatController.java | 51 +++ .../messenger/api/demo/DemoController.java | 77 ++++ .../api/health/HealthController.java | 46 ++ .../api/message/MessageController.java | 50 +++ .../messenger/api/user/UserController.java | 47 +++ .../mtuci/messenger/cache/SessionCache.java | 61 +++ .../messenger/domain/AttachmentEntity.java | 210 ++++++++++ .../domain/AttachmentVariantEntity.java | 100 +++++ .../ru/mtuci/messenger/domain/ChatEntity.java | 45 ++ .../messenger/domain/ChatMemberEntity.java | 47 +++ .../mtuci/messenger/domain/ChatMemberId.java | 34 ++ .../ru/mtuci/messenger/domain/DemoFlag.java | 34 ++ .../messenger/domain/FailedJobEntity.java | 51 +++ .../messenger/domain/IdempotencyRecord.java | 69 +++ .../mtuci/messenger/domain/MessageEntity.java | 79 ++++ .../mtuci/messenger/domain/OutboxMessage.java | 72 ++++ .../mtuci/messenger/domain/SessionEntity.java | 53 +++ .../mtuci/messenger/domain/UserAccount.java | 61 +++ .../messenger/exception/ApiException.java | 24 ++ .../exception/ConflictException.java | 11 + .../exception/InvalidRequestException.java | 11 + .../exception/NotFoundException.java | 11 + .../exception/PayloadTooLargeException.java | 11 + .../exception/UnauthorizedException.java | 26 ++ .../exception/UnavailableException.java | 11 + .../exception/UnsupportedMediaException.java | 11 + .../mtuci/messenger/job/AttachmentJobs.java | 393 ++++++++++++++++++ .../mtuci/messenger/job/AttachmentWorker.java | 35 ++ .../ru/mtuci/messenger/job/BrokerConfig.java | 45 ++ .../mtuci/messenger/job/BrokerPublisher.java | 51 +++ .../ru/mtuci/messenger/job/JobPayload.java | 19 + .../ru/mtuci/messenger/job/LeaseReaper.java | 21 + .../mtuci/messenger/job/OutboxPublisher.java | 70 ++++ .../messenger/job/PermanentJobException.java | 15 + .../messenger/job/PoisonJobException.java | 8 + .../messenger/job/TransientJobException.java | 12 + .../ru/mtuci/messenger/mapper/ChatMapper.java | 20 + .../mtuci/messenger/mapper/MessageMapper.java | 12 + .../ru/mtuci/messenger/mapper/TimeMapper.java | 15 + .../ru/mtuci/messenger/mapper/UserMapper.java | 19 + .../mtuci/messenger/migrate/MigrateExit.java | 34 ++ .../repository/AttachmentRepository.java | 31 ++ .../AttachmentVariantRepository.java | 28 ++ .../repository/ChatMemberRepository.java | 18 + .../messenger/repository/ChatRepository.java | 30 ++ .../repository/DemoFlagRepository.java | 7 + .../repository/FailedJobRepository.java | 14 + .../repository/IdempotencyRepository.java | 13 + .../repository/MessageRepository.java | 31 ++ .../repository/OutboxRepository.java | 24 ++ .../repository/SessionRepository.java | 28 ++ .../messenger/repository/UserRepository.java | 15 + .../messenger/service/AttachmentService.java | 230 ++++++++++ .../mtuci/messenger/service/ChatService.java | 99 +++++ .../mtuci/messenger/service/CursorCodec.java | 48 +++ .../ru/mtuci/messenger/service/DemoFlags.java | 31 ++ .../mtuci/messenger/service/ImageFiles.java | 171 ++++++++ .../messenger/service/ImageVariants.java | 118 ++++++ .../messenger/service/MessageService.java | 202 +++++++++ .../mtuci/messenger/service/RetryDelay.java | 30 ++ .../ru/mtuci/messenger/service/TextRules.java | 35 ++ .../mtuci/messenger/service/UserService.java | 147 +++++++ .../messenger/storage/ObjectStorage.java | 149 +++++++ .../messenger/storage/OrphanSweeper.java | 81 ++++ .../ru/mtuci/messenger/storage/S3Config.java | 50 +++ .../messenger/storage/S3HealthIndicator.java | 30 ++ .../mtuci/messenger/storage/SweepConfig.java | 11 + .../ru/mtuci/messenger/web/ApiAdvice.java | 99 +++++ .../mtuci/messenger/web/BearerAuthFilter.java | 75 ++++ .../mtuci/messenger/web/InstanceIdFilter.java | 34 ++ .../mtuci/messenger/web/InstanceIdentity.java | 19 + .../mtuci/messenger/web/RequestIdFilter.java | 47 +++ .../mtuci/messenger/web/TimeFormatConfig.java | 25 ++ .../main/resources/application-migrate.yaml | 14 + .../main/resources/application-worker.yaml | 9 + backend/src/main/resources/application.yaml | 168 ++++++++ .../resources/db/changelog/001-users.yaml | 41 ++ .../resources/db/changelog/002-sessions.yaml | 47 +++ .../resources/db/changelog/003-chats.yaml | 77 ++++ .../resources/db/changelog/004-messages.yaml | 61 +++ .../db/changelog/005-idempotency.yaml | 73 ++++ .../db/changelog/006-attachments.yaml | 127 ++++++ .../main/resources/db/changelog/007-jobs.yaml | 221 ++++++++++ .../db/changelog/db.changelog-master.yaml | 22 + .../mtuci/messenger/ApiIntegrationTest.java | 281 +++++++++++++ .../messenger/AttachmentIntegrationTest.java | 240 +++++++++++ .../DatabaseOutageIntegrationTest.java | 75 ++++ .../HistoryIndexIntegrationTest.java | 65 +++ .../messenger/IdempotencyIntegrationTest.java | 164 ++++++++ .../messenger/RedisOutageIntegrationTest.java | 62 +++ .../SessionExpiryIntegrationTest.java | 53 +++ .../messenger/service/ImageFilesTest.java | 68 +++ .../messenger/service/ImageVariantsTest.java | 75 ++++ .../messenger/service/RetryDelayTest.java | 24 ++ .../messenger/service/TextRulesTest.java | 16 + .../messenger/storage/OrphanSweeperTest.java | 19 + .../ru/mtuci/messenger/support/ApiClient.java | 93 +++++ .../messenger/support/LabContainers.java | 64 +++ backend/src/test/resources/s3.json | 24 ++ compose.yaml | 245 +++++++++++ evidence/explain-messages.txt | 10 + evidence/grade4.txt | 48 +++ evidence/results.txt | 33 ++ infra/haproxy.cfg | 39 ++ infra/s3.json | 24 ++ scripts/grade4-evidence.py | 278 +++++++++++++ scripts/restart.sh | 22 + scripts/stop-one.sh | 24 ++ 114 files changed, 7390 insertions(+), 23 deletions(-) create mode 100644 Dockerfile create mode 100644 backend/pom.xml create mode 100644 backend/src/main/java/ru/mtuci/messenger/MessengerApplication.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/api/attachment/AttachmentController.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/api/chat/ChatController.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/api/demo/DemoController.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/api/health/HealthController.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/api/message/MessageController.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/api/user/UserController.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/cache/SessionCache.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/AttachmentEntity.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/AttachmentVariantEntity.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/ChatEntity.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/ChatMemberEntity.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/ChatMemberId.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/DemoFlag.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/FailedJobEntity.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/IdempotencyRecord.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/MessageEntity.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/OutboxMessage.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/SessionEntity.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/domain/UserAccount.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/exception/ApiException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/exception/ConflictException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/exception/InvalidRequestException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/exception/NotFoundException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/exception/PayloadTooLargeException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/exception/UnauthorizedException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/exception/UnavailableException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/exception/UnsupportedMediaException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/AttachmentJobs.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/AttachmentWorker.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/BrokerConfig.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/BrokerPublisher.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/JobPayload.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/LeaseReaper.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/OutboxPublisher.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/PermanentJobException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/PoisonJobException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/job/TransientJobException.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/mapper/ChatMapper.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/mapper/MessageMapper.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/mapper/TimeMapper.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/mapper/UserMapper.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/migrate/MigrateExit.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/AttachmentRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/AttachmentVariantRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/ChatMemberRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/ChatRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/DemoFlagRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/FailedJobRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/IdempotencyRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/MessageRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/OutboxRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/SessionRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/repository/UserRepository.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/AttachmentService.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/ChatService.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/CursorCodec.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/DemoFlags.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/ImageFiles.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/ImageVariants.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/MessageService.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/RetryDelay.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/TextRules.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/service/UserService.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/storage/ObjectStorage.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/storage/OrphanSweeper.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/storage/S3Config.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/storage/S3HealthIndicator.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/storage/SweepConfig.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/web/ApiAdvice.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/web/BearerAuthFilter.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/web/InstanceIdFilter.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/web/InstanceIdentity.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/web/RequestIdFilter.java create mode 100644 backend/src/main/java/ru/mtuci/messenger/web/TimeFormatConfig.java create mode 100644 backend/src/main/resources/application-migrate.yaml create mode 100644 backend/src/main/resources/application-worker.yaml create mode 100644 backend/src/main/resources/application.yaml create mode 100644 backend/src/main/resources/db/changelog/001-users.yaml create mode 100644 backend/src/main/resources/db/changelog/002-sessions.yaml create mode 100644 backend/src/main/resources/db/changelog/003-chats.yaml create mode 100644 backend/src/main/resources/db/changelog/004-messages.yaml create mode 100644 backend/src/main/resources/db/changelog/005-idempotency.yaml create mode 100644 backend/src/main/resources/db/changelog/006-attachments.yaml create mode 100644 backend/src/main/resources/db/changelog/007-jobs.yaml create mode 100644 backend/src/main/resources/db/changelog/db.changelog-master.yaml create mode 100644 backend/src/test/java/ru/mtuci/messenger/ApiIntegrationTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/AttachmentIntegrationTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/DatabaseOutageIntegrationTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/HistoryIndexIntegrationTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/IdempotencyIntegrationTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/RedisOutageIntegrationTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/SessionExpiryIntegrationTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/service/ImageFilesTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/service/ImageVariantsTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/service/RetryDelayTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/service/TextRulesTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/storage/OrphanSweeperTest.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/support/ApiClient.java create mode 100644 backend/src/test/java/ru/mtuci/messenger/support/LabContainers.java create mode 100644 backend/src/test/resources/s3.json create mode 100644 compose.yaml create mode 100644 evidence/explain-messages.txt create mode 100644 evidence/grade4.txt create mode 100644 evidence/results.txt create mode 100644 infra/haproxy.cfg create mode 100644 infra/s3.json create mode 100644 scripts/grade4-evidence.py create mode 100755 scripts/restart.sh create mode 100755 scripts/stop-one.sh diff --git a/.env.example b/.env.example index 119384b5..32f060cc 100644 --- a/.env.example +++ b/.env.example @@ -1,25 +1,43 @@ # Copy to .env for your LOCAL synthetic test environment; never commit .env. -# These are logical interface suggestions; map actual framework names in REPORT.md. APP_HOST=0.0.0.0 APP_PORT=8080 APP_ENV=lab SESSION_TTL_SECONDS=3600 -# Choose credentials locally. File paths contain no secret values. -DATABASE_URL_FILE=/run/secrets/database_url -# Optional on 3, required use on 4 in lab2; Redis OR Valkey: SESSION_STORE=postgres -CACHE_URL_FILE=/run/secrets/cache_url API_REPLICAS=2 IDEMPOTENCY_TTL_SECONDS=86400 -S3_ENDPOINT=http://s3:9000 +POSTGRES_USER=messenger +POSTGRES_PASSWORD=messenger +POSTGRES_DB=messenger +SPRING_DATASOURCE_URL=jdbc:postgresql://postgres:5432/messenger +SPRING_DATASOURCE_USERNAME=messenger +SPRING_DATASOURCE_PASSWORD=messenger +SPRING_DATA_REDIS_HOST=redis +SPRING_DATA_REDIS_PORT=6379 +S3_ENDPOINT=http://s3:8333 S3_REGION=us-east-1 S3_BUCKET=messenger-attachments -S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id -S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key +S3_ACCESS_KEY=messenger +S3_SECRET_KEY=messenger-lab-secret +S3_BOOTSTRAP_ACCESS_KEY=messenger-bootstrap +S3_BOOTSTRAP_SECRET_KEY=messenger-bootstrap-secret +S3_ORPHAN_GRACE=2m MAX_UPLOAD_BYTES=10485760 MAX_IMAGE_PIXELS=40000000 -BROKER_URL_FILE=/run/secrets/broker_url +# Kafka слушает только сеть стенда, SASL нет: порт наружу не опубликован. +SPRING_KAFKA_BOOTSTRAP_SERVERS=kafka:9092 WORKER_CONCURRENCY=2 JOB_MAX_ATTEMPTS=5 JOB_TIMEOUT_SECONDS=60 +JOB_BACKOFF_BASE_MS=1000 +JOB_BACKOFF_CAP_MS=30000 STAGING_RETENTION_SECONDS=86400 +DEMO_TOKEN=lab-demo +JPEG_QUALITY=0.85 +WEBP_QUALITY=0.8 +# Course names kept for the report mapping. This stand passes credentials as env vars, not files. +DATABASE_URL_FILE=/run/secrets/database_url +CACHE_URL_FILE=/run/secrets/cache_url +S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id +S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key +BROKER_URL_FILE=/run/secrets/broker_url diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 00000000..37f9bb49 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,22 @@ +FROM maven:3.9.11-eclipse-temurin-21 AS build +WORKDIR /src +COPY backend/pom.xml backend/pom.xml +COPY contracts contracts +COPY backend/src backend/src +RUN --mount=type=cache,target=/root/.m2 \ + mvn -f backend/pom.xml -DskipTests -Dmaven.wagon.http.retryHandler.count=5 package \ + || mvn -f backend/pom.xml -DskipTests -Dmaven.wagon.http.retryHandler.count=5 package + +FROM eclipse-temurin:21.0.8_9-jre-jammy +RUN apt-get update \ + && apt-get install -y --no-install-recommends curl \ + && rm -rf /var/lib/apt/lists/* \ + && useradd --uid 10001 --create-home --shell /usr/sbin/nologin app +WORKDIR /app +COPY --from=build /src/backend/target/messenger.jar /app/messenger.jar +USER app +EXPOSE 8080 +ENV LIQUIBASE_ANALYTICS_ENABLED=false +HEALTHCHECK --interval=5s --timeout=3s --start-period=60s --retries=3 \ + CMD curl -fsS http://127.0.0.1:8080/health/ready || exit 1 +ENTRYPOINT ["java", "-jar", "/app/messenger.jar"] diff --git a/REPORT.md b/REPORT.md index e98f75e9..c748af43 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,37 +1,126 @@ # Отчёт — лабораторная 5 -- Автор, группа: TODO -- Целевая оценка; выбранный трек 5, если нужен: TODO -- Commit/PR: TODO после публикации -- Использованные источники, библиотеки и помощь ИИ: TODO +- Автор, группа: ivanstorozhev, БВТ2304 +- Целевая оценка: 4 +- Трек 5: не выбран +- Commit/PR: ветка `solution/bvt2304-storozhev` +- Использованные источники, библиотеки и помощь ИИ: контракт `contracts/openapi.json`, `contracts/README.md`, `contracts/attachment-states.md`, `contracts/jobs.schema.json`. Код lab4 перенесён как основа (Spring Boot 4.1.1, Spring Data JPA, Liquibase, Redis-кэш сессий, HAProxy, Argon2id, OpenAPI Generator 7.17.0, AWS SDK for Java 2.31.50, SeaweedFS 4.52). Новое: Spring Kafka, Apache Kafka 3.9.1 (KRaft), Thumbnailator 0.4.20, webp-imageio 0.11.0. Помощь ИИ: перенос lab4, обработка вложений и отчёт сделаны в Cursor. ## Запуск из чистого clone -TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных. +Нужны Docker Desktop (проверено 28.5.1), Docker Compose и Python 3.9+. JDK 21 и Maven 3.9 нужны только для `mvn test` вне образа. На этой машине JDK — `/opt/homebrew/opt/openjdk@21`. + +```sh +cp .env.example .env +# Перед общим стендом замените учебные пароли PostgreSQL и ключи S3 в .env. +docker compose --env-file .env config --quiet +docker compose --env-file .env up -d --build +curl -fsS http://127.0.0.1:8080/health/ready +make test +make test-persistence ACTION_SCRIPT=scripts/restart.sh +make test-failover ACTION_SCRIPT=scripts/stop-one.sh +# Поднять остановленную реплику: +docker compose --env-file .env start api +python3 scripts/grade4-evidence.py +``` + +Образ приложения собирает сервис `migrate` (`messenger-lab5:1.0.0`). У `api` и `worker` тот же образ и `pull_policy: never`. Миграции — одноразовый `migrate`, профиль `migrate`, Liquibase до changelog `007-jobs`. Реплики `api` и `worker` стартуют после него с `SPRING_LIQUIBASE_ENABLED=false`. Bucket `messenger-attachments` создаёт API при readiness. + +Остановка без удаления данных: `docker compose --env-file .env stop`. Снятие контейнеров с сохранением volumes: `docker compose --env-file .env down`. Намеренное удаление учебных данных: `docker compose --env-file .env down -v`. Файл `.env` в Git не входит. + +Свои проверки: `JAVA_HOME=/opt/homebrew/opt/openjdk@21 mvn -f backend/pom.xml test`. Testcontainers поднимает PostgreSQL, Redis, Kafka и SeaweedFS. ## Архитектура и решения -TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab4, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации. +Клиент ходит только в HAProxy на `127.0.0.1:8080`. Две реплики `api` портов наружу не публикуют. Пользователи, чаты, сообщения, сессии и идемпотентность перенесены из lab4: PostgreSQL — источник истины, Redis — только кэш разбора Bearer. + +Загрузка больше не делает производные в запросе. `POST /api/v1/chats/{chat_id}/attachments` проверяет тип, размер и заголовки картинки, пишет байты в ключ `staging/{id}` и в одной транзакции сохраняет строку `attachments` (`queued`) и строку `job_outbox`. После COMMIT API отвечает 202. В брокер на этом шаге ещё ничего не уходит. Если транзакция не прошла, тот же запрос удаляет staging-объект. + +Публикацию делает отдельный цикл outbox и в API, и в worker. Строка берётся `FOR UPDATE SKIP LOCKED`, запись уходит в топик `attachments.process` с `acks=all`, и только потом ставится `published_at`. Пока брокер не подтвердил запись, строка остаётся и будет отправлена снова. Пауза публикатора для демонстрации лежит в таблице `demo_flags`, поэтому переживает `kill` API. Сообщение — JSON `attachment.process.v1`: идентификаторы, номер попытки и версия конвейера. Байтов и ключей S3 в нём нет. Ключ Kafka-записи — `attachment_id`. Топик лежит на volume `kafka-data`. + +Worker — отдельный процесс (`--spring.profiles.active=worker`, без HTTP). Он коммитит смещение Kafka только после записи исхода в PostgreSQL. Оригинал копируется в `originals/{id}` без изменения байтов. Производные пишутся в фиксированные ключи `variants/{id}/thumbnail`, `crop`, `optimized`. Повторная доставка перезаписывает те же ключи и заново вставляет тот же набор строк `attachment_variants`. Ключ эффекта — `(attachment_id, pipeline_version, variant)`, сейчас `pipeline_version=1`. + +Состояния в API: `queued`, `processing`, `ready`, `failed`. Пока статус не `ready` и не успешный `original` после ready, `GET .../content` отвечает 409. Failed-вложение к сообщению не прикрепляется (409). Queued и processing прикрепить можно. `sha256` и `size_bytes` описывают оригинал. + +Производные, качество записано в `ImageVariants` и в `.env.example` (`JPEG_QUALITY=0.85`, `WEBP_QUALITY=0.8`): + +- thumbnail — JPEG, вписать в 256×256 без увеличения, размеры вниз, минимум 1 px. Фикстура 320×200 даёт 256×160; +- crop — JPEG 128×128 из центра, увеличение разрешено; +- optimized — lossy WebP, вписать в 1280×1280 без увеличения. Та же фикстура остаётся 320×200. + +Перед изменением размера применяется EXIF orientation. Альфа кладётся на белый. В JPEG и WebP метаданные исходника не копируются. WebP quality 0.8 — это качество libwebp 80. + +Лимиты: тело не больше 10 MiB (`MAX_UPLOAD_BYTES`), пиксели не больше 40 млн (`MAX_IMAGE_PIXELS`, проверка по заголовку до полного decode). Воркер обрывает обработку по `JOB_TIMEOUT_SECONDS` (60 с). Контейнер worker — 1 CPU и 1024 MiB, JVM `-XX:MaxRAMPercentage=60 -XX:+ExitOnOutOfMemoryError`. Одновременно воркер держит `WORKER_CONCURRENCY` потребителей (2) и такой же prefetch. + +Повтор: не больше `JOB_MAX_ATTEMPTS` (5). Постоянные ошибки `invalid_image` и `processing_failed` сразу дают `failed`. Временные (S3, обрыв, неожиданный runtime) возвращают задание в `queued` с задержкой. База задержки — 1 с, на каждую следующую попытку она удваивается и упирается в 30 с; фактическая пауза выбирается случайно от половины этой базы до всей базы. После исчерпания попыток код `retry_exhausted`. Исход `failed` пишется в `failed_jobs` и best-effort публикуется в топик `attachments.failed`. Битый JSON пишется туда же, и только потом коммитится смещение, иначе запись навсегда заняла бы партицию. + +Явный replay: `POST /internal/demo/attachments/{id}/replay` с заголовком `X-Demo-Token` (`DEMO_TOKEN`). Эти маршруты вне публичного контракта. Для `ready` вызов ничего не ставит в очередь. Для `failed` тот же `attachment_id` снова становится `queued`, счётчик попыток обнуляется, в outbox появляется одна новая строка. Staging failed-задания хранится до `STAGING_RETENTION_SECONDS` (86400), поэтому replay ещё может прочитать байты. После ready staging удаляется только если транзакция исхода закоммитилась. + +На время `processing` ставится lease на `JOB_TIMEOUT_SECONDS`. Пока этот процесс сам держит `lease_token`, вторая запись того же задания пропускается. Если процесс умер до `acknowledge()`, новый процесс токена не помнит и забирает `processing` себе. Если запись брокера так и не вернулась, reaper раз в 5 с возвращает просроченный lease в outbox либо закрывает его как `retry_exhausted`. + +`/health/ready` смотрит PostgreSQL и S3. Kafka в readiness не входит: брокер может лежать, а upload всё равно отвечает 202 за счёт outbox. Таймауты HAProxy подняты (клиент и разбор запроса 60 с, сервер 30 с), чтобы загрузка не обрывалась на старых 5 с. + +| Роль | Сервис | Образ | Готовность | Volume | Сеть | Credentials | +| --- | --- | --- | --- | --- | --- | --- | +| API | `api` × `API_REPLICAS` | `messenger-lab5:1.0.0` | `GET /health/ready` | нет файла оригинала | порт наружу не опубликован | env Spring, S3, Kafka | +| Worker | `worker` | тот же образ, профиль `worker` | `kill -0 1`, restart unless-stopped | нет | порт наружу не опубликован | те же env, `JAVA_TOOL_OPTIONS` | +| Балансировщик | `lb` | `haproxy:3.2.6` | процесс слушает 8080 | нет | только `127.0.0.1:8080` | нет | +| Метаданные | `postgres` | `postgres:16.10` | `pg_isready` | `postgres-data` | 5432 только внутри сети | `POSTGRES_*` из `.env` | +| Кэш сессии | `redis` | `redis:7.4.6` | `redis-cli ping` | `redis-data` | 6379 только внутри сети | без пароля, сеть стенда | +| Брокер | `kafka` | `apache/kafka:3.9.1` | `kafka-broker-api-versions` | `kafka-data` | 9092 только внутри сети | без SASL, порт наружу не опубликован | +| Объекты | `s3` | `chrislusf/seaweedfs:4.52` | `GET :9333/cluster/status` | `s3-data` | 8333/9333 только внутри сети | API: Read/Write/List. Bootstrap: Admin, только создание bucket | +| Миграции | `migrate` | тот же образ | код 0 | нет | нет | тот же datasource | + +Лимиты: API и worker по 1 CPU и 1024 MiB, PostgreSQL 1 CPU и 512 MiB, S3 0.5 CPU и 512 MiB, Kafka 0.5 CPU и 768 MiB (heap 256 MiB), Redis и HAProxy по 0.25 CPU и 128 MiB. + +Соответствие `.env.example`: переменные lab4 те же. Добавлены `SPRING_KAFKA_BOOTSTRAP_SERVERS`, `WORKER_CONCURRENCY`, `JOB_MAX_ATTEMPTS`, `JOB_TIMEOUT_SECONDS`, `JOB_BACKOFF_BASE_MS`, `JOB_BACKOFF_CAP_MS`, `STAGING_RETENTION_SECONDS`, `DEMO_TOKEN`, `JPEG_QUALITY`, `WEBP_QUALITY`. Имена `*_FILE` из курса оставлены комментарием: стенд передаёт доступ переменными окружения, не файлами секретов. Kafka без пароля, потому что слушатель не опубликован наружу. ## Доказательства критериев +Стенд: Mac, arm64, 12 CPU, 24 GiB RAM, Docker 28.5.1. Прогон 12 ноября 2026. Команды и цифры — `evidence/results.txt` и `evidence/grade4.txt`. Токены туда не записывались. + | Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства | | --- | --- | --- | --- | -| Публичный smoke | make test | TODO | TODO | -| Остальные пункты уровня 3 | TODO | TODO | TODO | -| Все пункты уровня 4, если заявлен | TODO | TODO | TODO | -| Выбранный трек 5, если заявлен | TODO | TODO | TODO | +| Публичный smoke: 202, ready, оригинал и JPEG thumbnail | `make test` | `PASS: lab5 public API scenario` | `evidence/results.txt` | +| Оригинал и метаданные переживают рестарт всего стенда, включая worker и Kafka | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | После смены брокера: `PASS`. Хук перезапускает и `kafka`, volumes не удаляет | `evidence/results.txt`, `scripts/restart.sh` | +| Другая реплика API отдаёт тот же оригинал | `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` на прогретом стенде | 2 instance id до отказа; 117 успешных записей, 1 ошибка на переходе, первая успешная запись через 1.795 с. `PASS`. Реплика поднята `start api` | `evidence/results.txt` | +| Worker остановлен, upload 202, сообщение с вложением, рестарт API, затем worker | `python3 scripts/grade4-evidence.py` | `9c4ab448-af4d-4acf-883d-4a8929a8b5e1`: после рестарта API всё ещё `queued`, content 409 `conflict`. После старта worker — `ready`. Оригинал 111817 байт совпал. Thumbnail JPEG 256×160, crop JPEG 128×128, optimized WebP 320×200 | `evidence/grade4.txt` | +| Задание в брокере переживает рестарт Kafka с volume | worker остановлен, upload, `compose restart kafka` | Конец топика `attachments.process` вырос 2→3 и после рестарта остался 3. Затем то же вложение `45a5da06-a3c3-4d12-ad25-3d5a9154d531` стало `ready` | `evidence/grade4.txt` | +| Crash между COMMIT и publish | outbox на паузе, upload, `compose kill api`, `start api`, снять паузу | До и после kill: `paused=true`, `unpublished=1`, статус `queued`, id `26bd80e5-7074-44b8-b2d5-1a28b14343fe`. После resume тот же id `ready` с тремя вариантами | `evidence/grade4.txt` | +| Падение worker после записи объектов и до commit смещения | файл `/tmp/messenger-crash-after-write` в контейнере worker, один upload | `RestartCount` 0→1. Id `9e1ed645-d5b7-4006-a84c-75ac547eb165` стал `ready`. В БД по одной строке crop, optimized и thumbnail. Replay ready вернул `ready` и не добавил вариант | `evidence/grade4.txt` | +| Постоянная ошибка, DLQ/реестр, явный replay | файл `/tmp/messenger-fail-permanent`, затем replay дважды | Id `05e76bf9-a6d0-4dc9-9ea0-ee1950a1218b`: `failed`, `error_code=processing_failed`. Сообщение с ним — 409 `conflict`. Реестр failed 1→2. Топик `attachments.failed` дошёл до смещения 1. Replay вернул `queued`, затем тот же id `ready`, в БД снова по одной строке варианта. Второй replay остался `ready` | `evidence/grade4.txt` | +| Мусор вместо картинки | POST 415 в том же сценарии | `unsupported_media_type`. Лимиты 10 MiB и 40 млн пикселей закрывает `AttachmentIntegrationTest` | `evidence/grade4.txt`, `mvn test` | +| Приватный bucket и порты | `docker ps` / `inspect` в том же сценарии, анонимный GET из сети стенда | Наружу только `lb` на `127.0.0.1:8080`. У api, worker, postgres, redis, kafka и s3 host-порт пустой. Kafka 768 MiB. Анонимный GET объекта — 403 | `evidence/grade4.txt` | +| Конкуренция worker | лог старта worker | Два потребителя группы `attachment-worker`, партиции `attachments.process-0` и `attachments.process-1` | лог `messenger-lab5-worker-1` | +| Свои тесты | `mvn -f backend/pom.xml -q test` | код выхода 0, 22 теста | `evidence/results.txt` | +| Трек 5 | не заявлен | — | — | -TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены. +Живой прогон ждал 60 с истёкшего lease не делал: фикстура обрабатывается быстрее таймаута. Проверен соседний путь — повторная доставка при ещё живом lease после `System.exit` между PUT и COMMIT. Reaper для просроченного lease в коде worker есть (`JOB_TIMEOUT_SECONDS`, период 5 с). ## Эксперименты и отказы -TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения. +Отдельного замера RPS нет: оценка 4 этого не требует. Наблюдения ниже сняты тем же стендом. | Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных | | --- | --- | --- | --- | -| TODO | TODO | TODO | TODO | +| `scripts/restart.sh` сразу после серии kill/crash, пауза хука 5 с | тот же Bearer читает оригинал | `TimeoutError` на GET content: клиент теста ждёт 2 с | подтверждённые данные на месте; хук увеличен до 12 с | +| Тот же хук с паузой 12 с | `/health/ready`, затем чтение оригинала | `PASS` | метаданные, сессия и байты на месте | +| `stop-one.sh` сразу после холодного рестарта | загрузка и остановка одной реплики | `TimeoutError` на POST вложения ещё до hook | повтор на прогретом стенде ниже | +| `stop-one.sh` на прогретом стенде | другая реплика принимает запись, остановленная сама не поднимается | 2 instance id, восстановление 1.795 с, 117 записей и 1 ошибка, `PASS` | подтверждённые сообщения и оригинал на месте; реплика поднята `start api` | +| Worker остановлен, затем убит API | 202 и строка задания не пропадают | статус остался `queued`, после старта worker — `ready` | работа не потеряна | +| Рестарт Kafka при записи в топике | запись остаётся на volume | конец топика 3 до и после рестарта, затем `ready` | задание не потеряно | +| Пауза outbox и `kill` обеих реплик API | строка outbox переживает процесс | `unpublished=1` до и после kill, затем тот же id `ready` | новое задание вручную не создавалось | +| `System.exit` после PUT и до ack | повторная доставка заканчивает тот же результат | контейнер сам поднялся, один набор вариантов | дубля вариантов нет | +| Постоянный сбой до записи вариантов | `failed`, replay, повторный replay ready | `processing_failed`, потом `ready`, второй replay остался `ready` | второй набор вариантов не создан | ## Ограничения и следующие шаги -TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза. +Один Docker-host не переживает потерю этой машины. PostgreSQL, Redis, SeaweedFS, Kafka и HAProxy в одном экземпляре. Replication factor топиков равен 1. + +Доставка at least once. Подтверждение брокера (`acks=all`) и commit смещения потребителя — разные границы. Повтор перезаписывает те же ключи и те же строки вариантов, но это не exactly-once на стороне S3: объект может быть записан дважды одинаковыми байтами. + +Если процесс умирает после PUT staging и до COMMIT строки, объект без строки забирает sweeper после grace. Если строка `failed` живёт дольше `STAGING_RETENTION_SECONDS`, replay уже не находит байты и завершается `processing_failed`. + +Пока HAProxy не снял backend, часть запросов попадает в останавливаемую реплику. Балансировщик POST не повторяет. Клиент может повторить тот же `Idempotency-Key`. Клиент публичных тестов отказов обрывает запрос через 2 с, поэтому хук рестарта ждёт не только `/health/ready`, но и 12 с, пока SeaweedFS начинает отдавать объекты быстрее этого таймаута. + +Внутренние demo-маршруты закрыты общим токеном из окружения и не входят в OpenAPI. Для учебной демонстрации этого достаточно; отдельной роли оператора нет. diff --git a/backend/pom.xml b/backend/pom.xml new file mode 100644 index 00000000..a3b6e713 --- /dev/null +++ b/backend/pom.xml @@ -0,0 +1,218 @@ + + + 4.0.0 + + org.springframework.boot + spring-boot-starter-parent + 4.1.1 + + + ru.mtuci + messenger + 1.0.0-SNAPSHOT + messenger + MTUSI messenger lab 5 + + 21 + 1.6.3 + 7.17.0 + 2.2.36 + 2.31.50 + + + + org.springframework.boot + spring-boot-starter-webmvc + + + org.springframework.boot + spring-boot-starter-validation + + + org.springframework.boot + spring-boot-starter-data-jpa + + + org.springframework.boot + spring-boot-starter-liquibase + + + org.springframework.boot + spring-boot-starter-data-redis + + + org.springframework.security + spring-security-crypto + + + org.bouncycastle + bcprov-jdk18on + 1.86 + + + org.springframework.boot + spring-boot-starter-actuator + + + io.micrometer + micrometer-registry-prometheus + + + org.postgresql + postgresql + runtime + + + org.mapstruct + mapstruct + ${mapstruct.version} + + + io.swagger.core.v3 + swagger-annotations-jakarta + ${swagger-annotations.version} + + + software.amazon.awssdk + s3 + ${aws.sdk.version} + + + software.amazon.awssdk + apache-client + ${aws.sdk.version} + + + org.springframework.boot + spring-boot-starter-kafka + + + net.coobird + thumbnailator + 0.4.20 + + + com.github.usefulness + webp-imageio + 0.11.0 + + + org.springframework.boot + spring-boot-starter-webmvc-test + test + + + org.springframework.boot + spring-boot-starter-data-jpa-test + test + + + org.springframework.boot + spring-boot-testcontainers + test + + + org.testcontainers + testcontainers + test + + + org.testcontainers + testcontainers-postgresql + test + + + org.testcontainers + testcontainers-kafka + test + + + + messenger + + + org.openapitools + openapi-generator-maven-plugin + ${openapi-generator.version} + + + + generate + + + ${project.basedir}/../contracts/openapi.json + spring + ${project.build.directory}/generated-sources/openapi + ru.mtuci.messenger.api.model + ru.mtuci.messenger.api.generated + false + true + false + + set=List + + + set=ArrayList + + + true + true + true + java8 + false + jackson + true + @com.fasterxml.jackson.annotation.JsonInclude(com.fasterxml.jackson.annotation.JsonInclude.Include.ALWAYS) + + + + + + + org.apache.maven.plugins + maven-antrun-plugin + + + jackson3-model-annotations + generate-sources + + run + + + + + + + + + + + + + + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + org.apache.maven.plugins + maven-compiler-plugin + + + + org.mapstruct + mapstruct-processor + ${mapstruct.version} + + + + + + + diff --git a/backend/src/main/java/ru/mtuci/messenger/MessengerApplication.java b/backend/src/main/java/ru/mtuci/messenger/MessengerApplication.java new file mode 100644 index 00000000..639754ee --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/MessengerApplication.java @@ -0,0 +1,20 @@ +package ru.mtuci.messenger; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; +import org.springframework.context.annotation.Bean; +import org.springframework.security.crypto.argon2.Argon2PasswordEncoder; +import org.springframework.security.crypto.password.PasswordEncoder; + +@SpringBootApplication +public class MessengerApplication { + + public static void main(String[] args) { + SpringApplication.run(MessengerApplication.class, args); + } + + @Bean + PasswordEncoder passwordEncoder() { + return Argon2PasswordEncoder.defaultsForSpringSecurity_v5_8(); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/api/attachment/AttachmentController.java b/backend/src/main/java/ru/mtuci/messenger/api/attachment/AttachmentController.java new file mode 100644 index 00000000..49c17da1 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/api/attachment/AttachmentController.java @@ -0,0 +1,52 @@ +package ru.mtuci.messenger.api.attachment; + +import jakarta.servlet.http.HttpServletRequest; +import org.springframework.core.io.InputStreamResource; +import org.springframework.http.HttpStatus; +import org.springframework.http.MediaType; +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.RestController; +import ru.mtuci.messenger.api.model.Attachment; +import ru.mtuci.messenger.service.AttachmentService; +import ru.mtuci.messenger.web.BearerAuthFilter; + +import java.io.IOException; +import java.util.UUID; + +@RestController +public class AttachmentController { + + private final AttachmentService attachments; + + public AttachmentController(AttachmentService attachments) { + this.attachments = attachments; + } + + @PostMapping(value = "/api/v1/chats/{chatId}/attachments", consumes = {"image/png", "image/jpeg"}) + public ResponseEntity upload(HttpServletRequest request, @PathVariable UUID chatId, + @RequestParam("filename") String filename) throws IOException { + long length = request.getContentLengthLong(); + Attachment created = attachments.upload( + BearerAuthFilter.userId(request), chatId, filename, request.getContentType(), length, request.getInputStream()); + return ResponseEntity.status(HttpStatus.ACCEPTED).body(created); + } + + @GetMapping("/api/v1/attachments/{attachmentId}") + public Attachment get(HttpServletRequest request, @PathVariable UUID attachmentId) { + return attachments.get(BearerAuthFilter.userId(request), attachmentId); + } + + @GetMapping("/api/v1/attachments/{attachmentId}/content") + public ResponseEntity content(HttpServletRequest request, @PathVariable UUID attachmentId, + @RequestParam(required = false) String variant) { + var download = attachments.content(BearerAuthFilter.userId(request), attachmentId, variant); + return ResponseEntity.ok() + .contentType(MediaType.parseMediaType(download.contentType())) + .contentLength(download.size()) + .body(new InputStreamResource(download.body())); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/api/chat/ChatController.java b/backend/src/main/java/ru/mtuci/messenger/api/chat/ChatController.java new file mode 100644 index 00000000..1a936db2 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/api/chat/ChatController.java @@ -0,0 +1,51 @@ +package ru.mtuci.messenger.api.chat; + +import jakarta.servlet.http.HttpServletRequest; +import jakarta.validation.Valid; +import jakarta.validation.constraints.Max; +import jakarta.validation.constraints.Min; +import jakarta.validation.constraints.Size; +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; +import org.springframework.validation.annotation.Validated; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestBody; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.RestController; +import ru.mtuci.messenger.api.model.Chat; +import ru.mtuci.messenger.api.model.ChatPage; +import ru.mtuci.messenger.api.model.CreateChat; +import ru.mtuci.messenger.service.ChatService; +import ru.mtuci.messenger.web.BearerAuthFilter; + +import java.util.UUID; + +@Validated +@RestController +public class ChatController { + + private final ChatService chats; + + public ChatController(ChatService chats) { + this.chats = chats; + } + + @PostMapping("/api/v1/chats") + public ResponseEntity create(HttpServletRequest request, @Valid @RequestBody CreateChat body) { + return ResponseEntity.status(HttpStatus.CREATED).body(chats.create(BearerAuthFilter.userId(request), body)); + } + + @GetMapping("/api/v1/chats") + public ChatPage list(HttpServletRequest request, + @RequestParam(required = false) @Min(1) @Max(100) Integer limit, + @RequestParam(required = false) @Size(min = 1) String cursor) { + return chats.list(BearerAuthFilter.userId(request), limit, cursor); + } + + @GetMapping("/api/v1/chats/{chatId}") + public Chat get(HttpServletRequest request, @PathVariable UUID chatId) { + return chats.get(BearerAuthFilter.userId(request), chatId); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/api/demo/DemoController.java b/backend/src/main/java/ru/mtuci/messenger/api/demo/DemoController.java new file mode 100644 index 00000000..ecb11da2 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/api/demo/DemoController.java @@ -0,0 +1,77 @@ +package ru.mtuci.messenger.api.demo; + +import jakarta.servlet.http.HttpServletRequest; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Profile; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RestController; +import ru.mtuci.messenger.api.model.Attachment; +import ru.mtuci.messenger.exception.UnauthorizedException; +import ru.mtuci.messenger.repository.FailedJobRepository; +import ru.mtuci.messenger.repository.OutboxRepository; +import ru.mtuci.messenger.service.AttachmentService; +import ru.mtuci.messenger.service.DemoFlags; + +import java.util.Map; +import java.util.UUID; + +@RestController +@Profile("!worker") +public class DemoController { + + private final DemoFlags flags; + private final OutboxRepository outbox; + private final FailedJobRepository failedJobs; + private final AttachmentService attachments; + private final String token; + + public DemoController(DemoFlags flags, OutboxRepository outbox, FailedJobRepository failedJobs, + AttachmentService attachments, @Value("${app.demo.token:lab-demo}") String token) { + this.flags = flags; + this.outbox = outbox; + this.failedJobs = failedJobs; + this.attachments = attachments; + this.token = token; + } + + @GetMapping("/internal/demo/outbox") + public Map outbox(HttpServletRequest request) { + authorize(request); + return Map.of("paused", flags.outboxPaused(), "unpublished", outbox.countByPublishedAtIsNull()); + } + + @PostMapping("/internal/demo/outbox-pause") + public Map pause(HttpServletRequest request) { + authorize(request); + flags.setOutboxPaused(true); + return Map.of("paused", true); + } + + @PostMapping("/internal/demo/outbox-resume") + public Map resume(HttpServletRequest request) { + authorize(request); + flags.setOutboxPaused(false); + return Map.of("paused", false); + } + + @GetMapping("/internal/demo/failed") + public Map failed(HttpServletRequest request) { + authorize(request); + return Map.of("count", failedJobs.countByReplayedAtIsNull()); + } + + @PostMapping("/internal/demo/attachments/{attachmentId}/replay") + public Attachment replay(HttpServletRequest request, @PathVariable UUID attachmentId) { + authorize(request); + return attachments.replay(attachmentId); + } + + private void authorize(HttpServletRequest request) { + String header = request.getHeader("X-Demo-Token"); + if (header == null || !header.equals(token)) { + throw UnauthorizedException.bearer(); + } + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/api/health/HealthController.java b/backend/src/main/java/ru/mtuci/messenger/api/health/HealthController.java new file mode 100644 index 00000000..1fac6410 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/api/health/HealthController.java @@ -0,0 +1,46 @@ +package ru.mtuci.messenger.api.health; + +import org.springframework.boot.health.actuate.endpoint.HealthDescriptor; +import org.springframework.context.annotation.Profile; +import org.springframework.boot.health.actuate.endpoint.HealthEndpoint; +import org.springframework.boot.health.contributor.Status; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RestController; +import ru.mtuci.messenger.api.model.Health; +import ru.mtuci.messenger.exception.UnavailableException; + +@RestController +@Profile("!worker") +public class HealthController { + + private final HealthEndpoint healthEndpoint; + + public HealthController(HealthEndpoint healthEndpoint) { + this.healthEndpoint = healthEndpoint; + } + + @GetMapping("/health/live") + public Health live() { + if (!up("liveness")) { + throw new IllegalStateException("liveness"); + } + return ok(); + } + + @GetMapping("/health/ready") + public Health ready() { + if (!up("readiness")) { + throw new UnavailableException("Сервис не готов"); + } + return ok(); + } + + private boolean up(String path) { + HealthDescriptor descriptor = healthEndpoint.healthForPath(path); + return descriptor != null && Status.UP.equals(descriptor.getStatus()); + } + + private Health ok() { + return new Health(Health.StatusEnum.OK); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/api/message/MessageController.java b/backend/src/main/java/ru/mtuci/messenger/api/message/MessageController.java new file mode 100644 index 00000000..015ad13b --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/api/message/MessageController.java @@ -0,0 +1,50 @@ +package ru.mtuci.messenger.api.message; + +import jakarta.validation.Valid; +import jakarta.validation.constraints.Max; +import jakarta.validation.constraints.Min; +import jakarta.validation.constraints.Size; +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; +import org.springframework.validation.annotation.Validated; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestBody; +import org.springframework.web.bind.annotation.RequestHeader; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.RestController; +import ru.mtuci.messenger.api.model.CreateMessage; +import ru.mtuci.messenger.api.model.Message; +import ru.mtuci.messenger.api.model.MessagePage; +import jakarta.servlet.http.HttpServletRequest; +import ru.mtuci.messenger.service.MessageService; +import ru.mtuci.messenger.web.BearerAuthFilter; + +import java.util.UUID; + +@Validated +@RestController +public class MessageController { + + private final MessageService messageService; + + public MessageController(MessageService messageService) { + this.messageService = messageService; + } + + @PostMapping("/api/v1/chats/{chatId}/messages") + public ResponseEntity create(HttpServletRequest request, @PathVariable UUID chatId, + @RequestHeader(value = "Idempotency-Key", required = false) String idempotencyKey, + @Valid @RequestBody CreateMessage body) { + return ResponseEntity.status(HttpStatus.CREATED) + .body(messageService.create(BearerAuthFilter.userId(request), chatId, idempotencyKey, body)); + } + + @GetMapping("/api/v1/chats/{chatId}/messages") + public MessagePage list(HttpServletRequest request, @PathVariable UUID chatId, + @RequestParam(required = false) @Min(1) @Max(100) Integer limit, + @RequestParam(required = false) @Size(min = 1) String cursor) { + return messageService.list(BearerAuthFilter.userId(request), chatId, limit, cursor); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/api/user/UserController.java b/backend/src/main/java/ru/mtuci/messenger/api/user/UserController.java new file mode 100644 index 00000000..5d180b59 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/api/user/UserController.java @@ -0,0 +1,47 @@ +package ru.mtuci.messenger.api.user; + +import jakarta.servlet.http.HttpServletRequest; +import jakarta.validation.Valid; +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.DeleteMapping; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestBody; +import org.springframework.web.bind.annotation.RestController; +import ru.mtuci.messenger.api.model.CreateUser; +import ru.mtuci.messenger.api.model.Session; +import ru.mtuci.messenger.api.model.User; +import ru.mtuci.messenger.service.UserService; +import ru.mtuci.messenger.web.BearerAuthFilter; + +@RestController +public class UserController { + + private final UserService users; + + public UserController(UserService users) { + this.users = users; + } + + @PostMapping("/api/v1/users") + public ResponseEntity register(@Valid @RequestBody CreateUser request) { + return ResponseEntity.status(HttpStatus.CREATED).body(users.register(request)); + } + + @GetMapping("/api/v1/users/me") + public User me(HttpServletRequest request) { + return users.current(BearerAuthFilter.userId(request)); + } + + @PostMapping("/api/v1/auth/sessions") + public ResponseEntity login(HttpServletRequest request) { + return ResponseEntity.status(HttpStatus.CREATED).body(users.login(request.getHeader("Authorization"))); + } + + @DeleteMapping("/api/v1/auth/sessions/current") + public ResponseEntity logout(HttpServletRequest request) { + users.revoke((String) request.getAttribute(BearerAuthFilter.TOKEN)); + return ResponseEntity.noContent().build(); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/cache/SessionCache.java b/backend/src/main/java/ru/mtuci/messenger/cache/SessionCache.java new file mode 100644 index 00000000..836fe1ac --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/cache/SessionCache.java @@ -0,0 +1,61 @@ +package ru.mtuci.messenger.cache; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.dao.DataAccessException; +import org.springframework.data.redis.core.StringRedisTemplate; +import org.springframework.stereotype.Component; + +import java.time.Duration; +import java.time.Instant; +import java.util.Optional; +import java.util.UUID; + +/** + * Кэш Bearer-сессий: session:{sha256(token)} -> userId, TTL = остаток жизни сессии. + * Источник истины — таблица sessions; при ошибке Redis вызывающий идёт в PostgreSQL. + */ +@Component +public class SessionCache { + + private static final Logger log = LoggerFactory.getLogger(SessionCache.class); + + private final StringRedisTemplate redis; + + public SessionCache(StringRedisTemplate redis) { + this.redis = redis; + } + + public Optional find(String tokenHash) { + try { + return Optional.ofNullable(redis.opsForValue().get(key(tokenHash))).map(UUID::fromString); + } catch (DataAccessException exception) { + log.warn("session cache bypass: {}", exception.getClass().getSimpleName()); + return Optional.empty(); + } + } + + public void put(String tokenHash, UUID userId, Instant expiresAt) { + Duration ttl = Duration.between(Instant.now(), expiresAt); + if (ttl.isNegative() || ttl.isZero()) { + return; + } + try { + redis.opsForValue().set(key(tokenHash), userId.toString(), ttl); + } catch (DataAccessException exception) { + log.warn("session cache write skipped: {}", exception.getClass().getSimpleName()); + } + } + + public void evict(String tokenHash) { + try { + redis.delete(key(tokenHash)); + } catch (DataAccessException exception) { + log.warn("session cache evict skipped: {}", exception.getClass().getSimpleName()); + } + } + + private static String key(String tokenHash) { + return "session:" + tokenHash; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/AttachmentEntity.java b/backend/src/main/java/ru/mtuci/messenger/domain/AttachmentEntity.java new file mode 100644 index 00000000..7418f2be --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/AttachmentEntity.java @@ -0,0 +1,210 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.UUID; + +@Entity +@Table(name = "attachments") +public class AttachmentEntity { + + @Id + private UUID id; + + @Column(name = "chat_id", nullable = false) + private UUID chatId; + + @Column(name = "owner_id", nullable = false) + private UUID ownerId; + + @Column(nullable = false, length = 255) + private String filename; + + @Column(name = "content_type", nullable = false, length = 32) + private String contentType; + + @Column(name = "size_bytes", nullable = false) + private long sizeBytes; + + @Column(nullable = false, length = 64) + private String sha256; + + @Column(name = "object_key", nullable = false, length = 120) + private String objectKey; + + @Column(nullable = false, length = 16) + private String status; + + @Column(name = "created_at", nullable = false) + private Instant createdAt; + + @Column(name = "updated_at", nullable = false) + private Instant updatedAt; + + @Column(name = "error_code", length = 64) + private String errorCode; + + @Column(name = "pipeline_version", nullable = false) + private int pipelineVersion; + + @Column(nullable = false) + private int attempt; + + @Column(name = "job_id") + private UUID jobId; + + @Column(name = "staging_key", length = 120) + private String stagingKey; + + @Column(name = "lease_until") + private Instant leaseUntil; + + @Column(name = "lease_token") + private UUID leaseToken; + + protected AttachmentEntity() { + } + + public AttachmentEntity(UUID id, UUID chatId, UUID ownerId, String filename, String contentType, long sizeBytes, + String sha256, String objectKey, String stagingKey, UUID jobId) { + Instant now = Instant.now().truncatedTo(ChronoUnit.MICROS); + this.id = id; + this.chatId = chatId; + this.ownerId = ownerId; + this.filename = filename; + this.contentType = contentType; + this.sizeBytes = sizeBytes; + this.sha256 = sha256; + this.objectKey = objectKey; + this.stagingKey = stagingKey; + this.jobId = jobId; + this.status = "queued"; + this.pipelineVersion = 1; + this.attempt = 0; + this.createdAt = now; + this.updatedAt = now; + } + + public boolean leaseHeld(Instant now) { + return leaseUntil != null && leaseUntil.isAfter(now); + } + + public void markQueued(Instant now) { + this.status = "queued"; + this.errorCode = null; + this.leaseUntil = null; + this.leaseToken = null; + this.updatedAt = now; + } + + public void markProcessing(int attempt, UUID leaseToken, Instant leaseUntil, Instant now) { + this.status = "processing"; + this.attempt = attempt; + this.leaseToken = leaseToken; + this.leaseUntil = leaseUntil; + this.errorCode = null; + this.updatedAt = now; + } + + public void markReady(Instant now) { + this.status = "ready"; + this.errorCode = null; + this.leaseUntil = null; + this.leaseToken = null; + this.updatedAt = now; + } + + public void markFailed(String errorCode, Instant now) { + this.status = "failed"; + this.errorCode = errorCode; + this.leaseUntil = null; + this.leaseToken = null; + this.updatedAt = now; + } + + public void prepareReplay(Instant now) { + this.status = "queued"; + this.errorCode = null; + this.attempt = 0; + this.leaseUntil = null; + this.leaseToken = null; + this.updatedAt = now; + } + + public UUID getId() { + return id; + } + + public UUID getChatId() { + return chatId; + } + + public UUID getOwnerId() { + return ownerId; + } + + public String getFilename() { + return filename; + } + + public String getContentType() { + return contentType; + } + + public long getSizeBytes() { + return sizeBytes; + } + + public String getSha256() { + return sha256; + } + + public String getObjectKey() { + return objectKey; + } + + public String getStatus() { + return status; + } + + public Instant getCreatedAt() { + return createdAt; + } + + public Instant getUpdatedAt() { + return updatedAt; + } + + public String getErrorCode() { + return errorCode; + } + + public int getPipelineVersion() { + return pipelineVersion; + } + + public int getAttempt() { + return attempt; + } + + public UUID getJobId() { + return jobId; + } + + public String getStagingKey() { + return stagingKey; + } + + public Instant getLeaseUntil() { + return leaseUntil; + } + + public UUID getLeaseToken() { + return leaseToken; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/AttachmentVariantEntity.java b/backend/src/main/java/ru/mtuci/messenger/domain/AttachmentVariantEntity.java new file mode 100644 index 00000000..09bff3b4 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/AttachmentVariantEntity.java @@ -0,0 +1,100 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +import java.util.UUID; + +@Entity +@Table(name = "attachment_variants") +public class AttachmentVariantEntity { + + @Id + private UUID id; + + @Column(name = "attachment_id", nullable = false) + private UUID attachmentId; + + @Column(name = "pipeline_version", nullable = false) + private int pipelineVersion; + + @Column(nullable = false, length = 16) + private String name; + + @Column(name = "content_type", nullable = false, length = 32) + private String contentType; + + @Column(name = "size_bytes", nullable = false) + private long sizeBytes; + + @Column(nullable = false) + private int width; + + @Column(nullable = false) + private int height; + + @Column(nullable = false, length = 64) + private String sha256; + + @Column(name = "object_key", nullable = false, length = 120) + private String objectKey; + + protected AttachmentVariantEntity() { + } + + public AttachmentVariantEntity(UUID id, UUID attachmentId, int pipelineVersion, String name, String contentType, + long sizeBytes, int width, int height, String sha256, String objectKey) { + this.id = id; + this.attachmentId = attachmentId; + this.pipelineVersion = pipelineVersion; + this.name = name; + this.contentType = contentType; + this.sizeBytes = sizeBytes; + this.width = width; + this.height = height; + this.sha256 = sha256; + this.objectKey = objectKey; + } + + public UUID getId() { + return id; + } + + public UUID getAttachmentId() { + return attachmentId; + } + + public int getPipelineVersion() { + return pipelineVersion; + } + + public String getName() { + return name; + } + + public String getContentType() { + return contentType; + } + + public long getSizeBytes() { + return sizeBytes; + } + + public int getWidth() { + return width; + } + + public int getHeight() { + return height; + } + + public String getSha256() { + return sha256; + } + + public String getObjectKey() { + return objectKey; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/ChatEntity.java b/backend/src/main/java/ru/mtuci/messenger/domain/ChatEntity.java new file mode 100644 index 00000000..ff6169d0 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/ChatEntity.java @@ -0,0 +1,45 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.UUID; + +@Entity +@Table(name = "chats") +public class ChatEntity { + + @Id + private UUID id; + + @Column(nullable = false, length = 100) + private String title; + + @Column(name = "created_at", nullable = false) + private Instant createdAt; + + protected ChatEntity() { + } + + public ChatEntity(String title) { + this.id = UUID.randomUUID(); + this.title = title; + this.createdAt = Instant.now().truncatedTo(ChronoUnit.MICROS); + } + + public UUID getId() { + return id; + } + + public String getTitle() { + return title; + } + + public Instant getCreatedAt() { + return createdAt; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/ChatMemberEntity.java b/backend/src/main/java/ru/mtuci/messenger/domain/ChatMemberEntity.java new file mode 100644 index 00000000..1a929a5b --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/ChatMemberEntity.java @@ -0,0 +1,47 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.IdClass; +import jakarta.persistence.Table; + +import java.util.UUID; + +@Entity +@Table(name = "chat_members") +@IdClass(ChatMemberId.class) +public class ChatMemberEntity { + + @Id + @Column(name = "chat_id", nullable = false) + private UUID chatId; + + @Id + @Column(name = "user_id", nullable = false) + private UUID userId; + + @Column(name = "ordinal", nullable = false) + private int ordinal; + + protected ChatMemberEntity() { + } + + public ChatMemberEntity(UUID chatId, UUID userId, int ordinal) { + this.chatId = chatId; + this.userId = userId; + this.ordinal = ordinal; + } + + public UUID getChatId() { + return chatId; + } + + public UUID getUserId() { + return userId; + } + + public int getOrdinal() { + return ordinal; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/ChatMemberId.java b/backend/src/main/java/ru/mtuci/messenger/domain/ChatMemberId.java new file mode 100644 index 00000000..5caf3970 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/ChatMemberId.java @@ -0,0 +1,34 @@ +package ru.mtuci.messenger.domain; + +import java.io.Serializable; +import java.util.UUID; + +public class ChatMemberId implements Serializable { + + private UUID chatId; + private UUID userId; + + public ChatMemberId() { + } + + public ChatMemberId(UUID chatId, UUID userId) { + this.chatId = chatId; + this.userId = userId; + } + + @Override + public boolean equals(Object other) { + if (this == other) { + return true; + } + if (!(other instanceof ChatMemberId that)) { + return false; + } + return chatId.equals(that.chatId) && userId.equals(that.userId); + } + + @Override + public int hashCode() { + return chatId.hashCode() * 31 + userId.hashCode(); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/DemoFlag.java b/backend/src/main/java/ru/mtuci/messenger/domain/DemoFlag.java new file mode 100644 index 00000000..b4d1d6cb --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/DemoFlag.java @@ -0,0 +1,34 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +@Entity +@Table(name = "demo_flags") +public class DemoFlag { + + @Id + @Column(nullable = false, length = 64) + private String name; + + @Column(nullable = false) + private boolean enabled; + + protected DemoFlag() { + } + + public DemoFlag(String name, boolean enabled) { + this.name = name; + this.enabled = enabled; + } + + public boolean isEnabled() { + return enabled; + } + + public void setEnabled(boolean enabled) { + this.enabled = enabled; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/FailedJobEntity.java b/backend/src/main/java/ru/mtuci/messenger/domain/FailedJobEntity.java new file mode 100644 index 00000000..7e2cff9c --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/FailedJobEntity.java @@ -0,0 +1,51 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +import java.time.Instant; +import java.util.UUID; + +@Entity +@Table(name = "failed_jobs") +public class FailedJobEntity { + + @Id + private UUID id; + + @Column(name = "job_id", nullable = false) + private UUID jobId; + + @Column(name = "attachment_id", nullable = false) + private UUID attachmentId; + + @Column(name = "error_code", nullable = false, length = 64) + private String errorCode; + + @Column(nullable = false) + private int attempt; + + @Column(name = "created_at", nullable = false) + private Instant createdAt; + + @Column(name = "replayed_at") + private Instant replayedAt; + + protected FailedJobEntity() { + } + + public FailedJobEntity(UUID id, UUID jobId, UUID attachmentId, String errorCode, int attempt, Instant createdAt) { + this.id = id; + this.jobId = jobId; + this.attachmentId = attachmentId; + this.errorCode = errorCode; + this.attempt = attempt; + this.createdAt = createdAt; + } + + public void markReplayed(Instant replayedAt) { + this.replayedAt = replayedAt; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/IdempotencyRecord.java b/backend/src/main/java/ru/mtuci/messenger/domain/IdempotencyRecord.java new file mode 100644 index 00000000..76ddfdc5 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/IdempotencyRecord.java @@ -0,0 +1,69 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +import java.time.Instant; +import java.util.UUID; + +@Entity +@Table(name = "idempotency_records") +public class IdempotencyRecord { + + @Id + private UUID id; + + @Column(name = "user_id", nullable = false) + private UUID userId; + + @Column(name = "http_method", nullable = false, length = 8) + private String httpMethod; + + @Column(nullable = false, length = 256) + private String path; + + @Column(name = "idempotency_key", nullable = false, length = 128) + private String idempotencyKey; + + @Column(name = "request_hash", nullable = false, length = 64) + private String requestHash; + + @Column(name = "message_id", nullable = false) + private UUID messageId; + + @Column(name = "created_at", nullable = false) + private Instant createdAt; + + @Column(name = "expires_at", nullable = false) + private Instant expiresAt; + + protected IdempotencyRecord() { + } + + public IdempotencyRecord(UUID userId, String httpMethod, String path, String idempotencyKey, String requestHash, + UUID messageId, Instant createdAt, Instant expiresAt) { + this.id = UUID.randomUUID(); + this.userId = userId; + this.httpMethod = httpMethod; + this.path = path; + this.idempotencyKey = idempotencyKey; + this.requestHash = requestHash; + this.messageId = messageId; + this.createdAt = createdAt; + this.expiresAt = expiresAt; + } + + public UUID getMessageId() { + return messageId; + } + + public String getRequestHash() { + return requestHash; + } + + public Instant getExpiresAt() { + return expiresAt; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/MessageEntity.java b/backend/src/main/java/ru/mtuci/messenger/domain/MessageEntity.java new file mode 100644 index 00000000..56783ee3 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/MessageEntity.java @@ -0,0 +1,79 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.CollectionTable; +import jakarta.persistence.Column; +import jakarta.persistence.ElementCollection; +import jakarta.persistence.Entity; +import jakarta.persistence.FetchType; +import jakarta.persistence.Id; +import jakarta.persistence.JoinColumn; +import jakarta.persistence.OrderColumn; +import jakarta.persistence.Table; + +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.ArrayList; +import java.util.List; +import java.util.UUID; + +@Entity +@Table(name = "messages") +public class MessageEntity { + + @Id + private UUID id; + + @Column(name = "chat_id", nullable = false) + private UUID chatId; + + @Column(name = "sender_id", nullable = false) + private UUID senderId; + + @Column(nullable = false, length = 4000) + private String text; + + @Column(name = "created_at", nullable = false) + private Instant createdAt; + + @ElementCollection(fetch = FetchType.EAGER) + @CollectionTable(name = "message_attachments", joinColumns = @JoinColumn(name = "message_id")) + @OrderColumn(name = "sort_order") + @Column(name = "attachment_id", nullable = false) + private List attachmentIds = new ArrayList<>(); + + protected MessageEntity() { + } + + public MessageEntity(UUID chatId, UUID senderId, String text, List attachmentIds) { + this.id = UUID.randomUUID(); + this.chatId = chatId; + this.senderId = senderId; + this.text = text; + this.createdAt = Instant.now().truncatedTo(ChronoUnit.MICROS); + this.attachmentIds = new ArrayList<>(attachmentIds); + } + + public UUID getId() { + return id; + } + + public UUID getChatId() { + return chatId; + } + + public UUID getSenderId() { + return senderId; + } + + public String getText() { + return text; + } + + public Instant getCreatedAt() { + return createdAt; + } + + public List getAttachmentIds() { + return attachmentIds; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/OutboxMessage.java b/backend/src/main/java/ru/mtuci/messenger/domain/OutboxMessage.java new file mode 100644 index 00000000..0375e8e6 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/OutboxMessage.java @@ -0,0 +1,72 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +import java.time.Instant; +import java.util.UUID; + +@Entity +@Table(name = "job_outbox") +public class OutboxMessage { + + @Id + private UUID id; + + @Column(name = "job_id", nullable = false) + private UUID jobId; + + @Column(name = "attachment_id", nullable = false) + private UUID attachmentId; + + @Column(nullable = false) + private int attempt; + + @Column(nullable = false, length = 4000) + private String payload; + + @Column(name = "available_at", nullable = false) + private Instant availableAt; + + @Column(name = "published_at") + private Instant publishedAt; + + @Column(name = "created_at", nullable = false) + private Instant createdAt; + + protected OutboxMessage() { + } + + public OutboxMessage(UUID id, UUID jobId, UUID attachmentId, int attempt, String payload, Instant availableAt, + Instant createdAt) { + this.id = id; + this.jobId = jobId; + this.attachmentId = attachmentId; + this.attempt = attempt; + this.payload = payload; + this.availableAt = availableAt; + this.createdAt = createdAt; + } + + public UUID getId() { + return id; + } + + public UUID getAttachmentId() { + return attachmentId; + } + + public String getPayload() { + return payload; + } + + public Instant getPublishedAt() { + return publishedAt; + } + + public void setPublishedAt(Instant publishedAt) { + this.publishedAt = publishedAt; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/SessionEntity.java b/backend/src/main/java/ru/mtuci/messenger/domain/SessionEntity.java new file mode 100644 index 00000000..409cda91 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/SessionEntity.java @@ -0,0 +1,53 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +import java.time.Duration; +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.UUID; + +@Entity +@Table(name = "sessions") +public class SessionEntity { + + @Id + private UUID id; + + @Column(name = "token_hash", nullable = false, unique = true, length = 64) + private String tokenHash; + + @Column(name = "user_id", nullable = false) + private UUID userId; + + @Column(name = "expires_at", nullable = false) + private Instant expiresAt; + + @Column(name = "revoked_at") + private Instant revokedAt; + + @Column(name = "created_at", nullable = false) + private Instant createdAt; + + protected SessionEntity() { + } + + public SessionEntity(String tokenHash, UUID userId, Duration ttl) { + this.id = UUID.randomUUID(); + this.tokenHash = tokenHash; + this.userId = userId; + this.createdAt = Instant.now().truncatedTo(ChronoUnit.MICROS); + this.expiresAt = createdAt.plus(ttl); + } + + public UUID getUserId() { + return userId; + } + + public Instant getExpiresAt() { + return expiresAt; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/domain/UserAccount.java b/backend/src/main/java/ru/mtuci/messenger/domain/UserAccount.java new file mode 100644 index 00000000..d5fac265 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/domain/UserAccount.java @@ -0,0 +1,61 @@ +package ru.mtuci.messenger.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.UUID; + +@Entity +@Table(name = "users") +public class UserAccount { + + @Id + private UUID id; + + @Column(nullable = false, unique = true, length = 32) + private String username; + + @Column(name = "display_name", nullable = false, length = 100) + private String displayName; + + @Column(name = "password_hash", nullable = false, length = 255) + private String passwordHash; + + @Column(name = "created_at", nullable = false) + private Instant createdAt; + + protected UserAccount() { + } + + public UserAccount(String username, String displayName, String passwordHash) { + this.id = UUID.randomUUID(); + this.username = username; + this.displayName = displayName; + this.passwordHash = passwordHash; + this.createdAt = Instant.now().truncatedTo(ChronoUnit.MICROS); + } + + public UUID getId() { + return id; + } + + public String getUsername() { + return username; + } + + public String getDisplayName() { + return displayName; + } + + public String getPasswordHash() { + return passwordHash; + } + + public Instant getCreatedAt() { + return createdAt; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/exception/ApiException.java b/backend/src/main/java/ru/mtuci/messenger/exception/ApiException.java new file mode 100644 index 00000000..810796a4 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/exception/ApiException.java @@ -0,0 +1,24 @@ +package ru.mtuci.messenger.exception; + +import org.springframework.http.HttpStatus; +import ru.mtuci.messenger.api.model.ErrorError; + +public class ApiException extends RuntimeException { + + private final HttpStatus status; + private final ErrorError.CodeEnum code; + + public ApiException(HttpStatus status, ErrorError.CodeEnum code, String message) { + super(message); + this.status = status; + this.code = code; + } + + public HttpStatus status() { + return status; + } + + public ErrorError.CodeEnum code() { + return code; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/exception/ConflictException.java b/backend/src/main/java/ru/mtuci/messenger/exception/ConflictException.java new file mode 100644 index 00000000..e3388b6e --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/exception/ConflictException.java @@ -0,0 +1,11 @@ +package ru.mtuci.messenger.exception; + +import org.springframework.http.HttpStatus; +import ru.mtuci.messenger.api.model.ErrorError; + +public class ConflictException extends ApiException { + + public ConflictException(String message) { + super(HttpStatus.CONFLICT, ErrorError.CodeEnum.CONFLICT, message); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/exception/InvalidRequestException.java b/backend/src/main/java/ru/mtuci/messenger/exception/InvalidRequestException.java new file mode 100644 index 00000000..6e203a1e --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/exception/InvalidRequestException.java @@ -0,0 +1,11 @@ +package ru.mtuci.messenger.exception; + +import org.springframework.http.HttpStatus; +import ru.mtuci.messenger.api.model.ErrorError; + +public class InvalidRequestException extends ApiException { + + public InvalidRequestException(String message) { + super(HttpStatus.BAD_REQUEST, ErrorError.CodeEnum.INVALID_REQUEST, message); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/exception/NotFoundException.java b/backend/src/main/java/ru/mtuci/messenger/exception/NotFoundException.java new file mode 100644 index 00000000..96a728b5 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/exception/NotFoundException.java @@ -0,0 +1,11 @@ +package ru.mtuci.messenger.exception; + +import org.springframework.http.HttpStatus; +import ru.mtuci.messenger.api.model.ErrorError; + +public class NotFoundException extends ApiException { + + public NotFoundException() { + super(HttpStatus.NOT_FOUND, ErrorError.CodeEnum.NOT_FOUND, "Объект не найден"); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/exception/PayloadTooLargeException.java b/backend/src/main/java/ru/mtuci/messenger/exception/PayloadTooLargeException.java new file mode 100644 index 00000000..2a2eb3a3 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/exception/PayloadTooLargeException.java @@ -0,0 +1,11 @@ +package ru.mtuci.messenger.exception; + +import org.springframework.http.HttpStatus; +import ru.mtuci.messenger.api.model.ErrorError; + +public class PayloadTooLargeException extends ApiException { + + public PayloadTooLargeException() { + super(HttpStatus.PAYLOAD_TOO_LARGE, ErrorError.CodeEnum.PAYLOAD_TOO_LARGE, "Файл слишком большой"); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/exception/UnauthorizedException.java b/backend/src/main/java/ru/mtuci/messenger/exception/UnauthorizedException.java new file mode 100644 index 00000000..0950ac38 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/exception/UnauthorizedException.java @@ -0,0 +1,26 @@ +package ru.mtuci.messenger.exception; + +import org.springframework.http.HttpStatus; +import ru.mtuci.messenger.api.model.ErrorError; + +public class UnauthorizedException extends ApiException { + + private final String challenge; + + private UnauthorizedException(String challenge) { + super(HttpStatus.UNAUTHORIZED, ErrorError.CodeEnum.UNAUTHORIZED, "Требуется аутентификация"); + this.challenge = challenge; + } + + public static UnauthorizedException basic() { + return new UnauthorizedException("Basic realm=\"messenger\", charset=\"UTF-8\""); + } + + public static UnauthorizedException bearer() { + return new UnauthorizedException("Bearer realm=\"messenger\""); + } + + public String challenge() { + return challenge; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/exception/UnavailableException.java b/backend/src/main/java/ru/mtuci/messenger/exception/UnavailableException.java new file mode 100644 index 00000000..e3b72d84 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/exception/UnavailableException.java @@ -0,0 +1,11 @@ +package ru.mtuci.messenger.exception; + +import org.springframework.http.HttpStatus; +import ru.mtuci.messenger.api.model.ErrorError; + +public class UnavailableException extends ApiException { + + public UnavailableException(String message) { + super(HttpStatus.SERVICE_UNAVAILABLE, ErrorError.CodeEnum.UNAVAILABLE, message); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/exception/UnsupportedMediaException.java b/backend/src/main/java/ru/mtuci/messenger/exception/UnsupportedMediaException.java new file mode 100644 index 00000000..44bd67c5 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/exception/UnsupportedMediaException.java @@ -0,0 +1,11 @@ +package ru.mtuci.messenger.exception; + +import org.springframework.http.HttpStatus; +import ru.mtuci.messenger.api.model.ErrorError; + +public class UnsupportedMediaException extends ApiException { + + public UnsupportedMediaException() { + super(HttpStatus.UNSUPPORTED_MEDIA_TYPE, ErrorError.CodeEnum.UNSUPPORTED_MEDIA_TYPE, "Неподдерживаемый тип содержимого"); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/AttachmentJobs.java b/backend/src/main/java/ru/mtuci/messenger/job/AttachmentJobs.java new file mode 100644 index 00000000..1e831136 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/AttachmentJobs.java @@ -0,0 +1,393 @@ +package ru.mtuci.messenger.job; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Service; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.support.TransactionTemplate; +import ru.mtuci.messenger.domain.AttachmentEntity; +import ru.mtuci.messenger.domain.AttachmentVariantEntity; +import ru.mtuci.messenger.domain.FailedJobEntity; +import ru.mtuci.messenger.domain.OutboxMessage; +import ru.mtuci.messenger.exception.ConflictException; +import ru.mtuci.messenger.exception.NotFoundException; +import ru.mtuci.messenger.exception.PayloadTooLargeException; +import ru.mtuci.messenger.exception.UnavailableException; +import ru.mtuci.messenger.exception.UnsupportedMediaException; +import ru.mtuci.messenger.repository.AttachmentRepository; +import ru.mtuci.messenger.repository.AttachmentVariantRepository; +import ru.mtuci.messenger.repository.FailedJobRepository; +import ru.mtuci.messenger.repository.OutboxRepository; +import ru.mtuci.messenger.service.ImageFiles; +import ru.mtuci.messenger.service.ImageVariants; +import ru.mtuci.messenger.service.RetryDelay; +import ru.mtuci.messenger.storage.ObjectStorage; +import tools.jackson.databind.json.JsonMapper; + +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Path; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.time.Duration; +import java.time.Instant; +import java.time.temporal.ChronoUnit; +import java.util.HexFormat; +import java.util.Set; +import java.util.UUID; +import java.util.concurrent.ConcurrentHashMap; + +@Service +public class AttachmentJobs { + + private static final Logger log = LoggerFactory.getLogger(AttachmentJobs.class); + private static final Path CRASH_AFTER_WRITE = Path.of("/tmp/messenger-crash-after-write"); + private static final Path FAIL_PERMANENT = Path.of("/tmp/messenger-fail-permanent"); + + private final AttachmentRepository attachments; + private final AttachmentVariantRepository variants; + private final OutboxRepository outbox; + private final FailedJobRepository failedJobs; + private final ObjectStorage storage; + private final JsonMapper json; + private final TransactionTemplate transactions; + private final ObjectProvider brokers; + private final int maxAttempts; + private final Duration timeout; + private final long backoffBaseMs; + private final long backoffCapMs; + private final int maxBytes; + private final long maxPixels; + private final float jpegQuality; + private final float webpQuality; + private final Set localLeases = ConcurrentHashMap.newKeySet(); + + public AttachmentJobs(AttachmentRepository attachments, AttachmentVariantRepository variants, OutboxRepository outbox, + FailedJobRepository failedJobs, ObjectStorage storage, JsonMapper json, + PlatformTransactionManager transactionManager, ObjectProvider brokers, + @Value("${app.job.max-attempts:5}") int maxAttempts, + @Value("${app.job.timeout:60s}") Duration timeout, + @Value("${app.job.backoff-base-ms:1000}") long backoffBaseMs, + @Value("${app.job.backoff-cap-ms:30000}") long backoffCapMs, + @Value("${app.upload.max-bytes:10485760}") int maxBytes, + @Value("${app.upload.max-pixels:40000000}") long maxPixels, + @Value("${app.image.jpeg-quality:0.85}") float jpegQuality, + @Value("${app.image.webp-quality:0.8}") float webpQuality) { + this.attachments = attachments; + this.variants = variants; + this.outbox = outbox; + this.failedJobs = failedJobs; + this.storage = storage; + this.json = json; + this.transactions = new TransactionTemplate(transactionManager); + this.brokers = brokers; + this.maxAttempts = maxAttempts; + this.timeout = timeout; + this.backoffBaseMs = backoffBaseMs; + this.backoffCapMs = backoffCapMs; + this.maxBytes = maxBytes; + this.maxPixels = maxPixels; + this.jpegQuality = jpegQuality; + this.webpQuality = webpQuality; + } + + public void enqueue(AttachmentEntity attachment, int attempt, Instant availableAt) { + JobPayload payload = new JobPayload(1, attachment.getJobId(), "attachment.process.v1", attachment.getId(), + attachment.getPipelineVersion(), attempt, attachment.getCreatedAt()); + outbox.save(new OutboxMessage(UUID.randomUUID(), attachment.getJobId(), attachment.getId(), attempt, + json.writeValueAsString(payload), availableAt, now())); + } + + public void handle(String body, boolean redelivered) { + JobPayload job = parse(body); + Claim claim = transactions.execute(status -> claim(job, redelivered)); + if (claim == null || !claim.process()) { + if (claim != null && claim.dlq()) { + publishFailed(body); + } + return; + } + try { + try { + Processed processed = process(claim); + Boolean saved = transactions.execute(status -> finishReady(claim, processed)); + if (Boolean.TRUE.equals(saved)) { + storage.deleteQuietly(claim.stagingKey()); + } + } catch (PermanentJobException exception) { + Boolean saved = transactions.execute(status -> finishFailed(claim, exception.errorCode())); + if (Boolean.TRUE.equals(saved)) { + publishFailed(body); + } + } catch (TransientJobException exception) { + log.info("transient job failure attachment={} type={}", claim.attachmentId(), exception.getClass().getSimpleName()); + transactions.executeWithoutResult(status -> finishRetry(claim)); + } + } finally { + localLeases.remove(claim.leaseToken()); + } + } + + public void rejectPoison(String body) { + log.warn("poison job"); + publishFailed(body); + } + + public AttachmentEntity replay(UUID attachmentId) { + return transactions.execute(status -> { + AttachmentEntity attachment = attachments.lockById(attachmentId).orElseThrow(NotFoundException::new); + if ("ready".equals(attachment.getStatus())) { + return attachment; + } + if (!"failed".equals(attachment.getStatus())) { + throw new ConflictException("Задание ещё выполняется"); + } + Instant now = now(); + attachment.prepareReplay(now); + for (FailedJobEntity failed : failedJobs.findByAttachmentIdAndReplayedAtIsNull(attachmentId)) { + failed.markReplayed(now); + } + enqueue(attachment, 1, now); + log.info("replay attachment={}", attachmentId); + return attachment; + }); + } + + public int reapExpired() { + int recovered = 0; + for (UUID id : attachments.findExpiredLeaseIds(Instant.now())) { + Boolean changed = transactions.execute(status -> recoverOne(id)); + if (Boolean.TRUE.equals(changed)) { + recovered++; + } + } + return recovered; + } + + private boolean recoverOne(UUID id) { + AttachmentEntity attachment = attachments.lockById(id).orElse(null); + Instant now = now(); + if (attachment == null || !"processing".equals(attachment.getStatus()) || attachment.leaseHeld(now)) { + return false; + } + if (attachment.getAttempt() >= maxAttempts) { + markFailed(attachment, "retry_exhausted", now); + return true; + } + attachment.markQueued(now); + enqueue(attachment, attachment.getAttempt() + 1, now); + log.info("lease expired attachment={} attempt={}", id, attachment.getAttempt()); + return true; + } + + private Claim claim(JobPayload job, boolean redelivered) { + AttachmentEntity attachment = attachments.lockById(job.attachmentId()).orElse(null); + if (attachment == null || attachment.getPipelineVersion() > job.pipelineVersion()) { + return Claim.done(false); + } + if ("ready".equals(attachment.getStatus()) || "failed".equals(attachment.getStatus())) { + return Claim.done(false); + } + Instant now = now(); + boolean ownedHere = attachment.getLeaseToken() != null && localLeases.contains(attachment.getLeaseToken()); + if ("processing".equals(attachment.getStatus()) && attachment.leaseHeld(now) && ownedHere && !redelivered) { + return Claim.done(false); + } + int next = attachment.getAttempt() + 1; + if (next > maxAttempts) { + markFailed(attachment, "retry_exhausted", now); + return Claim.done(true); + } + if (attachment.getStagingKey() == null || attachment.getJobId() == null) { + markFailed(attachment, "processing_failed", now); + return Claim.done(true); + } + UUID token = UUID.randomUUID(); + attachment.markProcessing(next, token, now.plus(timeout), now); + localLeases.add(token); + return new Claim(true, false, attachment.getId(), attachment.getJobId(), token, next, attachment.getStagingKey(), + attachment.getObjectKey(), attachment.getContentType(), attachment.getSizeBytes(), attachment.getSha256(), + attachment.getPipelineVersion()); + } + + private Processed process(Claim claim) { + long started = System.nanoTime(); + byte[] original = readStaging(claim); + checkTime(started); + if (original.length != claim.sizeBytes() || !sha256(original).equals(claim.sha256())) { + throw new PermanentJobException("invalid_image"); + } + try { + ImageFiles.inspect(original, claim.contentType(), maxPixels); + } catch (UnsupportedMediaException | PayloadTooLargeException exception) { + throw new PermanentJobException("invalid_image"); + } + if (failPermanentRequested()) { + throw new PermanentJobException("processing_failed"); + } + try { + checkTime(started); + storage.put(claim.objectKey(), original, claim.contentType()); + var rendered = ImageVariants.render(original, jpegQuality, webpQuality); + checkTime(started); + for (ImageVariants.VariantBytes variant : rendered) { + storage.put(variantKey(claim.attachmentId(), variant.name()), variant.bytes(), variant.contentType()); + } + if (crashAfterWriteRequested()) { + log.warn("fault injection: exit after object write before result commit"); + System.exit(1); + } + return new Processed(rendered); + } catch (PermanentJobException | TransientJobException exception) { + throw exception; + } catch (UnavailableException exception) { + throw new TransientJobException(exception); + } catch (IOException exception) { + throw new PermanentJobException("invalid_image"); + } catch (RuntimeException exception) { + log.warn("transient processing failure type={}", exception.getClass().getSimpleName()); + throw new TransientJobException(exception); + } + } + + private byte[] readStaging(Claim claim) { + try (InputStream input = storage.get(claim.stagingKey())) { + return ImageFiles.readLimited(input, maxBytes); + } catch (PayloadTooLargeException exception) { + throw new PermanentJobException("invalid_image"); + } catch (NotFoundException exception) { + throw new PermanentJobException("processing_failed"); + } catch (UnavailableException exception) { + throw new TransientJobException(exception); + } catch (IOException exception) { + throw new TransientJobException(exception); + } + } + + private boolean finishReady(Claim claim, Processed processed) { + AttachmentEntity attachment = attachments.lockById(claim.attachmentId()).orElse(null); + if (attachment == null || lost(claim, attachment) || "ready".equals(attachment.getStatus())) { + return false; + } + variants.deleteForPipeline(attachment.getId(), attachment.getPipelineVersion()); + for (ImageVariants.VariantBytes variant : processed.variants()) { + variants.save(new AttachmentVariantEntity(UUID.randomUUID(), attachment.getId(), attachment.getPipelineVersion(), + variant.name(), variant.contentType(), variant.bytes().length, variant.width(), variant.height(), + sha256(variant.bytes()), variantKey(attachment.getId(), variant.name()))); + } + attachment.markReady(now()); + log.info("attachment ready id={}", attachment.getId()); + return true; + } + + private boolean finishFailed(Claim claim, String code) { + AttachmentEntity attachment = attachments.lockById(claim.attachmentId()).orElse(null); + if (attachment == null || lost(claim, attachment) || "ready".equals(attachment.getStatus()) + || "failed".equals(attachment.getStatus())) { + return false; + } + markFailed(attachment, code, now()); + return true; + } + + private void publishFailed(String body) { + BrokerPublisher publisher = brokers.getIfAvailable(); + if (publisher != null) { + publisher.failed(body); + } + } + + private void finishRetry(Claim claim) { + AttachmentEntity attachment = attachments.lockById(claim.attachmentId()).orElse(null); + if (attachment == null || lost(claim, attachment) || "ready".equals(attachment.getStatus()) + || "failed".equals(attachment.getStatus())) { + return; + } + Instant now = now(); + if (attachment.getAttempt() >= maxAttempts) { + markFailed(attachment, "retry_exhausted", now); + return; + } + attachment.markQueued(now); + long delay = RetryDelay.millis(attachment.getAttempt(), backoffBaseMs, backoffCapMs); + enqueue(attachment, attachment.getAttempt() + 1, now.plusMillis(delay)); + } + + private void markFailed(AttachmentEntity attachment, String code, Instant now) { + attachment.markFailed(code, now); + failedJobs.save(new FailedJobEntity(UUID.randomUUID(), attachment.getJobId(), attachment.getId(), code, + attachment.getAttempt(), now)); + log.info("attachment failed id={} code={}", attachment.getId(), code); + } + + private static boolean lost(Claim claim, AttachmentEntity attachment) { + return attachment.getLeaseToken() == null || !claim.leaseToken().equals(attachment.getLeaseToken()); + } + + private void checkTime(long started) { + if (System.nanoTime() - started > timeout.toNanos()) { + throw new TransientJobException("timeout"); + } + } + + private JobPayload parse(String body) { + try { + JobPayload job = json.readValue(body, JobPayload.class); + if (job == null || job.schemaVersion() != 1 || job.attachmentId() == null || job.jobId() == null + || !"attachment.process.v1".equals(job.type()) || job.pipelineVersion() < 1 || job.attempt() < 1) { + throw new PoisonJobException(); + } + return job; + } catch (PoisonJobException exception) { + throw exception; + } catch (RuntimeException exception) { + throw new PoisonJobException(); + } + } + + private static boolean crashAfterWriteRequested() { + try { + return Files.deleteIfExists(CRASH_AFTER_WRITE); + } catch (IOException exception) { + return false; + } + } + + private static boolean failPermanentRequested() { + try { + return Files.deleteIfExists(FAIL_PERMANENT); + } catch (IOException exception) { + return false; + } + } + + static String variantKey(UUID attachmentId, String name) { + return "variants/" + attachmentId + "/" + name; + } + + private static String sha256(byte[] bytes) { + try { + return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes)); + } catch (NoSuchAlgorithmException exception) { + throw new IllegalStateException(exception); + } + } + + private static Instant now() { + return Instant.now().truncatedTo(ChronoUnit.MICROS); + } + + private record Claim(boolean process, boolean dlq, UUID attachmentId, UUID jobId, UUID leaseToken, int attempt, + String stagingKey, String objectKey, String contentType, long sizeBytes, String sha256, + int pipelineVersion) { + static Claim done(boolean dlq) { + return new Claim(false, dlq, null, null, null, 0, null, null, null, 0, null, 0); + } + } + + private record Processed(java.util.List variants) { + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/AttachmentWorker.java b/backend/src/main/java/ru/mtuci/messenger/job/AttachmentWorker.java new file mode 100644 index 00000000..19c55313 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/AttachmentWorker.java @@ -0,0 +1,35 @@ +package ru.mtuci.messenger.job; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.kafka.annotation.KafkaListener; +import org.springframework.kafka.support.Acknowledgment; +import org.springframework.stereotype.Component; + +@Component +@ConditionalOnProperty(name = "app.worker.consume", havingValue = "true") +public class AttachmentWorker { + + private static final Logger log = LoggerFactory.getLogger(AttachmentWorker.class); + + private final AttachmentJobs jobs; + + public AttachmentWorker(AttachmentJobs jobs) { + this.jobs = jobs; + } + + @KafkaListener(topics = BrokerConfig.PROCESS, groupId = "attachment-worker") + public void onMessage(String body, Acknowledgment acknowledgment) { + try { + jobs.handle(body, false); + acknowledgment.acknowledge(); + } catch (PoisonJobException exception) { + jobs.rejectPoison(body); + acknowledgment.acknowledge(); + } catch (RuntimeException exception) { + log.warn("job not acked type={}", exception.getClass().getSimpleName()); + throw exception; + } + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/BrokerConfig.java b/backend/src/main/java/ru/mtuci/messenger/job/BrokerConfig.java new file mode 100644 index 00000000..5153853f --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/BrokerConfig.java @@ -0,0 +1,45 @@ +package ru.mtuci.messenger.job; + +import org.apache.kafka.clients.admin.NewTopic; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.kafka.annotation.EnableKafka; +import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory; +import org.springframework.kafka.config.TopicBuilder; +import org.springframework.kafka.core.ConsumerFactory; +import org.springframework.kafka.listener.ContainerProperties; +import org.springframework.kafka.listener.DefaultErrorHandler; +import org.springframework.util.backoff.FixedBackOff; + +@Configuration +@EnableKafka +@ConditionalOnProperty(name = "app.broker.enabled", havingValue = "true") +public class BrokerConfig { + + public static final String PROCESS = "attachments.process"; + public static final String FAILED = "attachments.failed"; + + @Bean + NewTopic processTopic(@Value("${app.worker.concurrency:2}") int concurrency) { + return TopicBuilder.name(PROCESS).partitions(concurrency).replicas(1).build(); + } + + @Bean + NewTopic failedTopic() { + return TopicBuilder.name(FAILED).partitions(1).replicas(1).build(); + } + + @Bean + ConcurrentKafkaListenerContainerFactory kafkaListenerContainerFactory( + ConsumerFactory consumerFactory, + @Value("${app.worker.concurrency:2}") int concurrency) { + ConcurrentKafkaListenerContainerFactory factory = new ConcurrentKafkaListenerContainerFactory<>(); + factory.setConsumerFactory(consumerFactory); + factory.setConcurrency(concurrency); + factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL); + factory.setCommonErrorHandler(new DefaultErrorHandler(new FixedBackOff(1000L, FixedBackOff.UNLIMITED_ATTEMPTS))); + return factory; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/BrokerPublisher.java b/backend/src/main/java/ru/mtuci/messenger/job/BrokerPublisher.java new file mode 100644 index 00000000..f4c15661 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/BrokerPublisher.java @@ -0,0 +1,51 @@ +package ru.mtuci.messenger.job; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.kafka.core.KafkaTemplate; +import org.springframework.kafka.support.SendResult; +import org.springframework.stereotype.Component; + +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; + +@Component +@ConditionalOnProperty(name = "app.broker.enabled", havingValue = "true") +public class BrokerPublisher { + + private static final Logger log = LoggerFactory.getLogger(BrokerPublisher.class); + + private final KafkaTemplate kafka; + + public BrokerPublisher(KafkaTemplate kafka) { + this.kafka = kafka; + } + + public void confirm(String key, String payload) { + send(BrokerConfig.PROCESS, key, payload); + } + + public void failed(String payload) { + try { + send(BrokerConfig.FAILED, null, payload); + } catch (RuntimeException exception) { + log.warn("dlq publish failed type={}", exception.getClass().getSimpleName()); + } + } + + private void send(String topic, String key, String payload) { + try { + SendResult result = kafka.send(topic, key, payload).get(5, TimeUnit.SECONDS); + if (result == null || result.getRecordMetadata() == null) { + throw new IllegalStateException("broker nack"); + } + } catch (InterruptedException exception) { + Thread.currentThread().interrupt(); + throw new IllegalStateException("broker confirm interrupted"); + } catch (TimeoutException | ExecutionException exception) { + throw new IllegalStateException("broker confirm failed"); + } + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/JobPayload.java b/backend/src/main/java/ru/mtuci/messenger/job/JobPayload.java new file mode 100644 index 00000000..946e27f6 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/JobPayload.java @@ -0,0 +1,19 @@ +package ru.mtuci.messenger.job; + +import com.fasterxml.jackson.annotation.JsonIgnoreProperties; +import com.fasterxml.jackson.annotation.JsonProperty; + +import java.time.Instant; +import java.util.UUID; + +@JsonIgnoreProperties(ignoreUnknown = true) +public record JobPayload( + @JsonProperty("schema_version") int schemaVersion, + @JsonProperty("job_id") UUID jobId, + @JsonProperty("type") String type, + @JsonProperty("attachment_id") UUID attachmentId, + @JsonProperty("pipeline_version") int pipelineVersion, + @JsonProperty("attempt") int attempt, + @JsonProperty("created_at") Instant createdAt +) { +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/LeaseReaper.java b/backend/src/main/java/ru/mtuci/messenger/job/LeaseReaper.java new file mode 100644 index 00000000..b6de1b51 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/LeaseReaper.java @@ -0,0 +1,21 @@ +package ru.mtuci.messenger.job; + +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.scheduling.annotation.Scheduled; +import org.springframework.stereotype.Component; + +@Component +@ConditionalOnProperty(name = "app.worker.consume", havingValue = "true") +public class LeaseReaper { + + private final AttachmentJobs jobs; + + public LeaseReaper(AttachmentJobs jobs) { + this.jobs = jobs; + } + + @Scheduled(fixedDelayString = "${app.job.reaper-ms:5000}") + public void reap() { + jobs.reapExpired(); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/OutboxPublisher.java b/backend/src/main/java/ru/mtuci/messenger/job/OutboxPublisher.java new file mode 100644 index 00000000..4e396ceb --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/OutboxPublisher.java @@ -0,0 +1,70 @@ +package ru.mtuci.messenger.job; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.scheduling.annotation.Scheduled; +import org.springframework.stereotype.Component; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.support.TransactionTemplate; +import ru.mtuci.messenger.domain.OutboxMessage; +import ru.mtuci.messenger.repository.OutboxRepository; +import ru.mtuci.messenger.service.DemoFlags; + +import java.time.Instant; +import java.util.List; +import java.util.UUID; + +@Component +@ConditionalOnProperty(name = "app.outbox.enabled", havingValue = "true", matchIfMissing = true) +public class OutboxPublisher { + + private static final Logger log = LoggerFactory.getLogger(OutboxPublisher.class); + + private final OutboxRepository outbox; + private final DemoFlags flags; + private final BrokerPublisher broker; + private final TransactionTemplate transactions; + + public OutboxPublisher(OutboxRepository outbox, DemoFlags flags, BrokerPublisher broker, + PlatformTransactionManager transactionManager) { + this.outbox = outbox; + this.flags = flags; + this.broker = broker; + this.transactions = new TransactionTemplate(transactionManager); + } + + @Scheduled(fixedDelayString = "${app.outbox.poll-ms:500}") + public void publishDue() { + if (flags.outboxPaused()) { + return; + } + for (int i = 0; i < 10; i++) { + if (flags.outboxPaused()) { + return; + } + Boolean sent = transactions.execute(status -> publishOne()); + if (!Boolean.TRUE.equals(sent)) { + return; + } + } + } + + private boolean publishOne() { + if (flags.outboxPaused()) { + return false; + } + List ids = outbox.lockDueIds(Instant.now(), 1); + if (ids.isEmpty()) { + return false; + } + OutboxMessage row = outbox.findById(ids.getFirst()).orElseThrow(); + if (row.getPublishedAt() != null) { + return false; + } + broker.confirm(row.getAttachmentId().toString(), row.getPayload()); + row.setPublishedAt(Instant.now()); + log.info("outbox published id={}", row.getId()); + return true; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/PermanentJobException.java b/backend/src/main/java/ru/mtuci/messenger/job/PermanentJobException.java new file mode 100644 index 00000000..86801d86 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/PermanentJobException.java @@ -0,0 +1,15 @@ +package ru.mtuci.messenger.job; + +public class PermanentJobException extends RuntimeException { + + private final String errorCode; + + public PermanentJobException(String errorCode) { + super(errorCode); + this.errorCode = errorCode; + } + + public String errorCode() { + return errorCode; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/PoisonJobException.java b/backend/src/main/java/ru/mtuci/messenger/job/PoisonJobException.java new file mode 100644 index 00000000..66f02a72 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/PoisonJobException.java @@ -0,0 +1,8 @@ +package ru.mtuci.messenger.job; + +public class PoisonJobException extends RuntimeException { + + public PoisonJobException() { + super("bad job"); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/job/TransientJobException.java b/backend/src/main/java/ru/mtuci/messenger/job/TransientJobException.java new file mode 100644 index 00000000..583c5a64 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/job/TransientJobException.java @@ -0,0 +1,12 @@ +package ru.mtuci.messenger.job; + +public class TransientJobException extends RuntimeException { + + public TransientJobException(String message) { + super(message); + } + + public TransientJobException(Throwable cause) { + super(cause); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/mapper/ChatMapper.java b/backend/src/main/java/ru/mtuci/messenger/mapper/ChatMapper.java new file mode 100644 index 00000000..ca8d7a7a --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/mapper/ChatMapper.java @@ -0,0 +1,20 @@ +package ru.mtuci.messenger.mapper; + +import org.mapstruct.Mapper; +import org.mapstruct.Mapping; +import org.mapstruct.ReportingPolicy; +import ru.mtuci.messenger.api.model.Chat; +import ru.mtuci.messenger.domain.ChatEntity; + +import java.util.List; +import java.util.UUID; + +@Mapper(componentModel = "spring", uses = TimeMapper.class, unmappedTargetPolicy = ReportingPolicy.ERROR) +public interface ChatMapper { + + @Mapping(target = "id", source = "chat.id") + @Mapping(target = "title", source = "chat.title") + @Mapping(target = "createdAt", source = "chat.createdAt") + @Mapping(target = "memberIds", source = "memberIds") + Chat toModel(ChatEntity chat, List memberIds); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/mapper/MessageMapper.java b/backend/src/main/java/ru/mtuci/messenger/mapper/MessageMapper.java new file mode 100644 index 00000000..8ff896e0 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/mapper/MessageMapper.java @@ -0,0 +1,12 @@ +package ru.mtuci.messenger.mapper; + +import org.mapstruct.Mapper; +import org.mapstruct.ReportingPolicy; +import ru.mtuci.messenger.api.model.Message; +import ru.mtuci.messenger.domain.MessageEntity; + +@Mapper(componentModel = "spring", uses = TimeMapper.class, unmappedTargetPolicy = ReportingPolicy.ERROR) +public interface MessageMapper { + + Message toModel(MessageEntity message); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/mapper/TimeMapper.java b/backend/src/main/java/ru/mtuci/messenger/mapper/TimeMapper.java new file mode 100644 index 00000000..33f52233 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/mapper/TimeMapper.java @@ -0,0 +1,15 @@ +package ru.mtuci.messenger.mapper; + +import org.mapstruct.Mapper; + +import java.time.Instant; +import java.time.OffsetDateTime; +import java.time.ZoneOffset; + +@Mapper(componentModel = "spring") +public interface TimeMapper { + + default OffsetDateTime map(Instant instant) { + return instant == null ? null : instant.atOffset(ZoneOffset.UTC); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/mapper/UserMapper.java b/backend/src/main/java/ru/mtuci/messenger/mapper/UserMapper.java new file mode 100644 index 00000000..f775cc2e --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/mapper/UserMapper.java @@ -0,0 +1,19 @@ +package ru.mtuci.messenger.mapper; + +import org.mapstruct.Mapper; +import org.mapstruct.Mapping; +import org.mapstruct.ReportingPolicy; +import ru.mtuci.messenger.api.model.Session; +import ru.mtuci.messenger.api.model.User; +import ru.mtuci.messenger.domain.UserAccount; + +import java.time.Instant; + +@Mapper(componentModel = "spring", uses = TimeMapper.class, unmappedTargetPolicy = ReportingPolicy.ERROR) +public interface UserMapper { + + User toModel(UserAccount user); + + @Mapping(target = "tokenType", expression = "java(ru.mtuci.messenger.api.model.Session.TokenTypeEnum.BEARER)") + Session toSession(String token, Instant expiresAt, UserAccount user); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/migrate/MigrateExit.java b/backend/src/main/java/ru/mtuci/messenger/migrate/MigrateExit.java new file mode 100644 index 00000000..0508a583 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/migrate/MigrateExit.java @@ -0,0 +1,34 @@ +package ru.mtuci.messenger.migrate; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.ApplicationArguments; +import org.springframework.boot.ApplicationRunner; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.context.annotation.Profile; +import org.springframework.stereotype.Component; + +@Component +@Profile("migrate") +public class MigrateExit implements ApplicationRunner { + + private final ConfigurableApplicationContext context; + + public MigrateExit(ConfigurableApplicationContext context) { + this.context = context; + } + + @Override + public void run(ApplicationArguments args) { + Thread exit = new Thread(() -> { + try { + Thread.sleep(500); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + } + int code = SpringApplication.exit(context, () -> 0); + System.exit(code); + }, "migrate-exit"); + exit.setDaemon(false); + exit.start(); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/AttachmentRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/AttachmentRepository.java new file mode 100644 index 00000000..4bb50f99 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/AttachmentRepository.java @@ -0,0 +1,31 @@ +package ru.mtuci.messenger.repository; + +import jakarta.persistence.LockModeType; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Lock; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; +import ru.mtuci.messenger.domain.AttachmentEntity; + +import java.time.Instant; +import java.util.Collection; +import java.util.List; +import java.util.Optional; +import java.util.UUID; + +public interface AttachmentRepository extends JpaRepository { + + boolean existsByObjectKey(String objectKey); + + boolean existsByStagingKeyAndStatusIn(String stagingKey, Collection statuses); + + @Lock(LockModeType.PESSIMISTIC_WRITE) + @Query("select a from AttachmentEntity a where a.id = :id") + Optional lockById(@Param("id") UUID id); + + @Query(""" + select a.id from AttachmentEntity a + where a.status = 'processing' and a.leaseUntil is not null and a.leaseUntil < :now + """) + List findExpiredLeaseIds(@Param("now") Instant now); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/AttachmentVariantRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/AttachmentVariantRepository.java new file mode 100644 index 00000000..60add695 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/AttachmentVariantRepository.java @@ -0,0 +1,28 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Modifying; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; +import ru.mtuci.messenger.domain.AttachmentVariantEntity; + +import java.util.List; +import java.util.Optional; +import java.util.UUID; + +public interface AttachmentVariantRepository extends JpaRepository { + + List findByAttachmentIdAndPipelineVersion(UUID attachmentId, int pipelineVersion); + + Optional findByAttachmentIdAndPipelineVersionAndName(UUID attachmentId, int pipelineVersion, + String name); + + boolean existsByObjectKey(String objectKey); + + @Modifying(flushAutomatically = true) + @Query(""" + delete from AttachmentVariantEntity v + where v.attachmentId = :attachmentId and v.pipelineVersion = :pipelineVersion + """) + void deleteForPipeline(@Param("attachmentId") UUID attachmentId, @Param("pipelineVersion") int pipelineVersion); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/ChatMemberRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/ChatMemberRepository.java new file mode 100644 index 00000000..4af44b28 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/ChatMemberRepository.java @@ -0,0 +1,18 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.jpa.repository.JpaRepository; +import ru.mtuci.messenger.domain.ChatMemberEntity; +import ru.mtuci.messenger.domain.ChatMemberId; + +import java.util.Collection; +import java.util.List; +import java.util.UUID; + +public interface ChatMemberRepository extends JpaRepository { + + boolean existsByChatIdAndUserId(UUID chatId, UUID userId); + + List findByChatIdOrderByOrdinalAsc(UUID chatId); + + List findByChatIdInOrderByOrdinalAsc(Collection chatIds); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/ChatRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/ChatRepository.java new file mode 100644 index 00000000..a9b6f30e --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/ChatRepository.java @@ -0,0 +1,30 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.domain.Pageable; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import ru.mtuci.messenger.domain.ChatEntity; + +import java.time.Instant; +import java.util.List; +import java.util.UUID; + +public interface ChatRepository extends JpaRepository { + + @Query(""" + select c from ChatEntity c + where c.id in (select m.chatId from ChatMemberEntity m where m.userId = :userId) + order by c.createdAt asc, c.id asc + """) + List findFirst(UUID userId, Pageable pageable); + + @Query(value = """ + select c.* from chats c + where c.id in (select m.chat_id from chat_members m where m.user_id = :userId) + and (c.created_at, c.id) > (:createdAt, :id) + order by c.created_at asc, c.id asc + """, nativeQuery = true) + List findAfter(UUID userId, Instant createdAt, UUID id, Pageable pageable); + + long countByTitle(String title); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/DemoFlagRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/DemoFlagRepository.java new file mode 100644 index 00000000..921c0e5a --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/DemoFlagRepository.java @@ -0,0 +1,7 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.jpa.repository.JpaRepository; +import ru.mtuci.messenger.domain.DemoFlag; + +public interface DemoFlagRepository extends JpaRepository { +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/FailedJobRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/FailedJobRepository.java new file mode 100644 index 00000000..6e659379 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/FailedJobRepository.java @@ -0,0 +1,14 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.jpa.repository.JpaRepository; +import ru.mtuci.messenger.domain.FailedJobEntity; + +import java.util.List; +import java.util.UUID; + +public interface FailedJobRepository extends JpaRepository { + + long countByReplayedAtIsNull(); + + List findByAttachmentIdAndReplayedAtIsNull(UUID attachmentId); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/IdempotencyRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/IdempotencyRepository.java new file mode 100644 index 00000000..fe043f08 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/IdempotencyRepository.java @@ -0,0 +1,13 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.jpa.repository.JpaRepository; +import ru.mtuci.messenger.domain.IdempotencyRecord; + +import java.util.Optional; +import java.util.UUID; + +public interface IdempotencyRepository extends JpaRepository { + + Optional findByUserIdAndHttpMethodAndPathAndIdempotencyKey( + UUID userId, String httpMethod, String path, String idempotencyKey); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/MessageRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/MessageRepository.java new file mode 100644 index 00000000..5f2378fd --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/MessageRepository.java @@ -0,0 +1,31 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.domain.Pageable; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import ru.mtuci.messenger.domain.MessageEntity; + +import java.time.Instant; +import java.util.List; +import java.util.UUID; + +public interface MessageRepository extends JpaRepository { + + long countByChatId(UUID chatId); + + + @Query(""" + select m from MessageEntity m + where m.chatId = :chatId + order by m.createdAt asc, m.id asc + """) + List findFirst(UUID chatId, Pageable pageable); + + @Query(value = """ + select * from messages + where chat_id = :chatId + and (created_at, id) > (:createdAt, :id) + order by created_at asc, id asc + """, nativeQuery = true) + List findAfter(UUID chatId, Instant createdAt, UUID id, Pageable pageable); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/OutboxRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/OutboxRepository.java new file mode 100644 index 00000000..81bae54e --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/OutboxRepository.java @@ -0,0 +1,24 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; +import ru.mtuci.messenger.domain.OutboxMessage; + +import java.time.Instant; +import java.util.List; +import java.util.UUID; + +public interface OutboxRepository extends JpaRepository { + + long countByPublishedAtIsNull(); + + @Query(value = """ + select id from job_outbox + where published_at is null and available_at <= :now + order by created_at + limit :limit + for update skip locked + """, nativeQuery = true) + List lockDueIds(@Param("now") Instant now, @Param("limit") int limit); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/SessionRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/SessionRepository.java new file mode 100644 index 00000000..8481867b --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/SessionRepository.java @@ -0,0 +1,28 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Modifying; +import org.springframework.data.jpa.repository.Query; +import org.springframework.transaction.annotation.Transactional; +import ru.mtuci.messenger.domain.SessionEntity; + +import java.time.Instant; +import java.util.Optional; +import java.util.UUID; + +public interface SessionRepository extends JpaRepository { + + @Query(""" + select s from SessionEntity s + where s.tokenHash = :tokenHash and s.revokedAt is null and s.expiresAt > :now + """) + Optional findActive(String tokenHash, Instant now); + + @Transactional + @Modifying + @Query(""" + update SessionEntity s set s.revokedAt = :now + where s.tokenHash = :tokenHash and s.revokedAt is null and s.expiresAt > :now + """) + int revoke(String tokenHash, Instant now); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/repository/UserRepository.java b/backend/src/main/java/ru/mtuci/messenger/repository/UserRepository.java new file mode 100644 index 00000000..34e4dd05 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/repository/UserRepository.java @@ -0,0 +1,15 @@ +package ru.mtuci.messenger.repository; + +import org.springframework.data.jpa.repository.JpaRepository; +import ru.mtuci.messenger.domain.UserAccount; + +import java.util.Collection; +import java.util.Optional; +import java.util.UUID; + +public interface UserRepository extends JpaRepository { + + Optional findByUsername(String username); + + long countByIdIn(Collection ids); +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/AttachmentService.java b/backend/src/main/java/ru/mtuci/messenger/service/AttachmentService.java new file mode 100644 index 00000000..fee9c15d --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/AttachmentService.java @@ -0,0 +1,230 @@ +package ru.mtuci.messenger.service; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.http.InvalidMediaTypeException; +import org.springframework.http.MediaType; +import org.springframework.stereotype.Service; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.annotation.Transactional; +import org.springframework.transaction.support.TransactionTemplate; +import ru.mtuci.messenger.api.model.Attachment; +import ru.mtuci.messenger.api.model.Variant; +import ru.mtuci.messenger.domain.AttachmentEntity; +import ru.mtuci.messenger.domain.AttachmentVariantEntity; +import ru.mtuci.messenger.exception.ConflictException; +import ru.mtuci.messenger.exception.InvalidRequestException; +import ru.mtuci.messenger.exception.NotFoundException; +import ru.mtuci.messenger.exception.PayloadTooLargeException; +import ru.mtuci.messenger.exception.UnsupportedMediaException; +import ru.mtuci.messenger.job.AttachmentJobs; +import ru.mtuci.messenger.repository.AttachmentRepository; +import ru.mtuci.messenger.repository.AttachmentVariantRepository; +import ru.mtuci.messenger.repository.ChatMemberRepository; +import ru.mtuci.messenger.storage.ObjectStorage; + +import java.io.InputStream; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.time.Instant; +import java.util.ArrayList; +import java.util.Comparator; +import java.util.HexFormat; +import java.util.List; +import java.util.Set; +import java.util.UUID; + +@Service +public class AttachmentService { + + private static final List VARIANT_ORDER = List.of("thumbnail", "crop", "optimized"); + private static final Set VARIANTS = Set.copyOf(VARIANT_ORDER); + + private final AttachmentRepository attachments; + private final AttachmentVariantRepository variants; + private final ChatMemberRepository members; + private final ObjectStorage storage; + private final AttachmentJobs jobs; + private final TransactionTemplate transactions; + private final int maxBytes; + private final long maxPixels; + + public AttachmentService(AttachmentRepository attachments, AttachmentVariantRepository variants, + ChatMemberRepository members, ObjectStorage storage, AttachmentJobs jobs, + PlatformTransactionManager transactionManager, + @Value("${app.upload.max-bytes:10485760}") int maxBytes, + @Value("${app.upload.max-pixels:40000000}") long maxPixels) { + this.attachments = attachments; + this.variants = variants; + this.members = members; + this.storage = storage; + this.jobs = jobs; + this.transactions = new TransactionTemplate(transactionManager); + this.maxBytes = maxBytes; + this.maxPixels = maxPixels; + } + + public Attachment upload(UUID actorId, UUID chatId, String filename, String contentType, long contentLength, + InputStream body) { + requireMember(chatId, actorId); + String displayName = filename(filename); + String declared = declaredType(contentType); + if (contentLength > maxBytes) { + throw new PayloadTooLargeException(); + } + byte[] bytes; + try { + bytes = ImageFiles.readLimited(body, maxBytes); + } catch (PayloadTooLargeException tooLarge) { + throw tooLarge; + } catch (java.io.IOException exception) { + throw new ru.mtuci.messenger.exception.UnavailableException("Хранилище временно недоступно"); + } + ImageFiles.Image image = ImageFiles.inspect(bytes, declared, maxPixels); + UUID id = UUID.randomUUID(); + String stagingKey = "staging/" + id; + String objectKey = "originals/" + id; + storage.ensureBucket(); + storage.put(stagingKey, bytes, image.contentType()); + try { + AttachmentEntity saved = transactions.execute(status -> { + AttachmentEntity entity = attachments.saveAndFlush(new AttachmentEntity( + id, chatId, actorId, displayName, image.contentType(), bytes.length, sha256(bytes), + objectKey, stagingKey, UUID.randomUUID())); + jobs.enqueue(entity, 1, Instant.now()); + return entity; + }); + return toModel(saved); + } catch (RuntimeException exception) { + storage.deleteQuietly(stagingKey); + throw exception; + } + } + + @Transactional(readOnly = true) + public Attachment get(UUID actorId, UUID attachmentId) { + return toModel(visible(actorId, attachmentId)); + } + + public Attachment replay(UUID attachmentId) { + return toModel(jobs.replay(attachmentId)); + } + + @Transactional(readOnly = true) + public Download content(UUID actorId, UUID attachmentId, String variant) { + String name = variant == null || variant.isBlank() ? "original" : variant; + if (!"original".equals(name) && !VARIANTS.contains(name)) { + throw new InvalidRequestException("Некорректный запрос"); + } + AttachmentEntity attachment = visible(actorId, attachmentId); + if (!"ready".equals(attachment.getStatus())) { + throw new ConflictException("Вложение ещё не готово"); + } + if ("original".equals(name)) { + return new Download(storage.get(attachment.getObjectKey()), attachment.getContentType(), attachment.getSizeBytes()); + } + AttachmentVariantEntity row = variants.findByAttachmentIdAndPipelineVersionAndName( + attachment.getId(), attachment.getPipelineVersion(), name).orElseThrow(NotFoundException::new); + return new Download(storage.get(row.getObjectKey()), row.getContentType(), row.getSizeBytes()); + } + + private AttachmentEntity visible(UUID actorId, UUID attachmentId) { + AttachmentEntity attachment = attachments.findById(attachmentId).orElseThrow(NotFoundException::new); + if (!members.existsByChatIdAndUserId(attachment.getChatId(), actorId)) { + throw new NotFoundException(); + } + return attachment; + } + + private void requireMember(UUID chatId, UUID actorId) { + if (!members.existsByChatIdAndUserId(chatId, actorId)) { + throw new NotFoundException(); + } + } + + private static String filename(String value) { + if (value == null) { + throw new InvalidRequestException("Некорректный запрос"); + } + int codePoints = value.codePointCount(0, value.length()); + if (codePoints < 1 || codePoints > 255) { + throw new InvalidRequestException("Некорректный запрос"); + } + int offset = 0; + while (offset < value.length()) { + int codePoint = value.codePointAt(offset); + if (codePoint == '/' || codePoint == '\\' || Character.isISOControl(codePoint)) { + throw new InvalidRequestException("Некорректный запрос"); + } + offset += Character.charCount(codePoint); + } + return value; + } + + private static String declaredType(String header) { + if (header == null || header.isBlank()) { + throw new UnsupportedMediaException(); + } + MediaType media; + try { + media = MediaType.parseMediaType(header); + } catch (InvalidMediaTypeException exception) { + throw new UnsupportedMediaException(); + } + if (media.equalsTypeAndSubtype(MediaType.IMAGE_PNG)) { + return "image/png"; + } + if (media.equalsTypeAndSubtype(MediaType.parseMediaType("image/jpeg"))) { + return "image/jpeg"; + } + throw new UnsupportedMediaException(); + } + + private static String sha256(byte[] bytes) { + try { + return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes)); + } catch (NoSuchAlgorithmException exception) { + throw new IllegalStateException(exception); + } + } + + private Attachment toModel(AttachmentEntity entity) { + Attachment model = new Attachment(); + model.setId(entity.getId()); + model.setChatId(entity.getChatId()); + model.setOwnerId(entity.getOwnerId()); + model.setFilename(entity.getFilename()); + model.setContentType(Attachment.ContentTypeEnum.fromValue(entity.getContentType())); + model.setSizeBytes((int) entity.getSizeBytes()); + model.setSha256(entity.getSha256()); + model.setStatus(Attachment.StatusEnum.fromValue(entity.getStatus())); + model.setVariants("ready".equals(entity.getStatus()) ? variantModels(entity) : new ArrayList<>()); + model.setErrorCode(entity.getErrorCode()); + model.setCreatedAt(entity.getCreatedAt().atOffset(java.time.ZoneOffset.UTC)); + model.setUpdatedAt(entity.getUpdatedAt().atOffset(java.time.ZoneOffset.UTC)); + return model; + } + + private List variantModels(AttachmentEntity entity) { + List rows = new ArrayList<>(variants.findByAttachmentIdAndPipelineVersion( + entity.getId(), entity.getPipelineVersion())); + rows.sort(Comparator.comparingInt(row -> { + int index = VARIANT_ORDER.indexOf(row.getName()); + return index < 0 ? VARIANT_ORDER.size() : index; + })); + List models = new ArrayList<>(); + for (AttachmentVariantEntity row : rows) { + Variant variant = new Variant(); + variant.setName(Variant.NameEnum.fromValue(row.getName())); + variant.setContentType(Variant.ContentTypeEnum.fromValue(row.getContentType())); + variant.setSizeBytes((int) row.getSizeBytes()); + variant.setWidth(row.getWidth()); + variant.setHeight(row.getHeight()); + variant.setSha256(row.getSha256()); + models.add(variant); + } + return models; + } + + public record Download(InputStream body, String contentType, long size) { + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/ChatService.java b/backend/src/main/java/ru/mtuci/messenger/service/ChatService.java new file mode 100644 index 00000000..2d5f3ea0 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/ChatService.java @@ -0,0 +1,99 @@ +package ru.mtuci.messenger.service; + +import org.springframework.data.domain.PageRequest; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; +import ru.mtuci.messenger.api.model.Chat; +import ru.mtuci.messenger.api.model.ChatPage; +import ru.mtuci.messenger.api.model.CreateChat; +import ru.mtuci.messenger.domain.ChatEntity; +import ru.mtuci.messenger.domain.ChatMemberEntity; +import ru.mtuci.messenger.exception.InvalidRequestException; +import ru.mtuci.messenger.exception.NotFoundException; +import ru.mtuci.messenger.mapper.ChatMapper; +import ru.mtuci.messenger.repository.ChatMemberRepository; +import ru.mtuci.messenger.repository.ChatRepository; +import ru.mtuci.messenger.repository.UserRepository; + +import java.util.ArrayList; +import java.util.HashSet; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.UUID; + +@Service +public class ChatService { + + private final ChatRepository chats; + private final ChatMemberRepository members; + private final UserRepository users; + private final ChatMapper mapper; + + public ChatService(ChatRepository chats, ChatMemberRepository members, UserRepository users, ChatMapper mapper) { + this.chats = chats; + this.members = members; + this.users = users; + this.mapper = mapper; + } + + @Transactional + public Chat create(UUID actorId, CreateChat request) { + TextRules.title(request.getTitle()); + List memberIds = request.getMemberIds(); + if (memberIds == null || new HashSet<>(memberIds).size() != memberIds.size() || !memberIds.contains(actorId)) { + throw new InvalidRequestException("Некорректный запрос"); + } + ChatEntity chat = chats.save(new ChatEntity(request.getTitle())); + if (users.countByIdIn(memberIds) != memberIds.size()) { + throw new InvalidRequestException("Некорректный запрос"); + } + for (int index = 0; index < memberIds.size(); index++) { + members.save(new ChatMemberEntity(chat.getId(), memberIds.get(index), index)); + } + return mapper.toModel(chat, memberIds); + } + + @Transactional(readOnly = true) + public ChatPage list(UUID actorId, Integer limit, String cursor) { + int pageSize = limit == null ? 50 : limit; + CursorCodec.Position position = cursor == null ? null : CursorCodec.decode(cursor, null); + List rows = position == null + ? chats.findFirst(actorId, PageRequest.of(0, pageSize + 1)) + : chats.findAfter(actorId, position.createdAt(), position.id(), PageRequest.of(0, pageSize + 1)); + boolean more = rows.size() > pageSize; + List visible = more ? rows.subList(0, pageSize) : rows; + Map> byChat = memberIds(visible); + List items = visible.stream() + .map(chat -> mapper.toModel(chat, byChat.getOrDefault(chat.getId(), List.of()))) + .toList(); + String next = null; + if (more) { + ChatEntity last = visible.get(visible.size() - 1); + next = CursorCodec.encode(null, last.getCreatedAt(), last.getId()); + } + return new ChatPage(items, next); + } + + @Transactional(readOnly = true) + public Chat get(UUID actorId, UUID chatId) { + if (!members.existsByChatIdAndUserId(chatId, actorId)) { + throw new NotFoundException(); + } + ChatEntity chat = chats.findById(chatId).orElseThrow(NotFoundException::new); + List memberIds = members.findByChatIdOrderByOrdinalAsc(chatId).stream().map(ChatMemberEntity::getUserId).toList(); + return mapper.toModel(chat, memberIds); + } + + private Map> memberIds(List chats) { + Map> result = new LinkedHashMap<>(); + if (chats.isEmpty()) { + return result; + } + List ids = chats.stream().map(ChatEntity::getId).toList(); + for (ChatMemberEntity member : members.findByChatIdInOrderByOrdinalAsc(ids)) { + result.computeIfAbsent(member.getChatId(), ignored -> new ArrayList<>()).add(member.getUserId()); + } + return result; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/CursorCodec.java b/backend/src/main/java/ru/mtuci/messenger/service/CursorCodec.java new file mode 100644 index 00000000..0033fa7a --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/CursorCodec.java @@ -0,0 +1,48 @@ +package ru.mtuci.messenger.service; + +import ru.mtuci.messenger.exception.InvalidRequestException; + +import java.nio.charset.StandardCharsets; +import java.time.Instant; +import java.util.Base64; +import java.util.UUID; + +/** + * Непрозрачная позиция keyset-страницы: клиент получает её как next_cursor + * и возвращает в ?cursor=, чтобы следующая страница началась после (created_at, id). + * Для истории в курсор входит chatId: курсор другого чата отклоняется. + */ +public final class CursorCodec { + + private CursorCodec() { + } + + public static String encode(UUID scope, Instant createdAt, UUID id) { + String raw = scope == null ? createdAt + "|" + id : scope + "|" + createdAt + "|" + id; + return Base64.getUrlEncoder().withoutPadding().encodeToString(raw.getBytes(StandardCharsets.UTF_8)); + } + + public static Position decode(String cursor, UUID scope) { + String raw; + try { + raw = new String(Base64.getUrlDecoder().decode(cursor), StandardCharsets.UTF_8); + } catch (IllegalArgumentException exception) { + throw new InvalidRequestException("Некорректный курсор"); + } + String[] parts = raw.split("\\|", -1); + try { + if (scope == null && parts.length == 2) { + return new Position(Instant.parse(parts[0]), UUID.fromString(parts[1])); + } + if (scope != null && parts.length == 3 && scope.equals(UUID.fromString(parts[0]))) { + return new Position(Instant.parse(parts[1]), UUID.fromString(parts[2])); + } + } catch (RuntimeException exception) { + throw new InvalidRequestException("Некорректный курсор"); + } + throw new InvalidRequestException("Некорректный курсор"); + } + + public record Position(Instant createdAt, UUID id) { + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/DemoFlags.java b/backend/src/main/java/ru/mtuci/messenger/service/DemoFlags.java new file mode 100644 index 00000000..7862d212 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/DemoFlags.java @@ -0,0 +1,31 @@ +package ru.mtuci.messenger.service; + +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; +import ru.mtuci.messenger.domain.DemoFlag; +import ru.mtuci.messenger.repository.DemoFlagRepository; + +@Service +public class DemoFlags { + + public static final String OUTBOX_PAUSED = "outbox_paused"; + + private final DemoFlagRepository flags; + + public DemoFlags(DemoFlagRepository flags) { + this.flags = flags; + } + + @Transactional(readOnly = true) + public boolean outboxPaused() { + return flags.findById(OUTBOX_PAUSED).map(DemoFlag::isEnabled).orElse(false); + } + + @Transactional + public boolean setOutboxPaused(boolean paused) { + DemoFlag flag = flags.findById(OUTBOX_PAUSED).orElseGet(() -> new DemoFlag(OUTBOX_PAUSED, paused)); + flag.setEnabled(paused); + flags.save(flag); + return paused; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/ImageFiles.java b/backend/src/main/java/ru/mtuci/messenger/service/ImageFiles.java new file mode 100644 index 00000000..cff0a941 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/ImageFiles.java @@ -0,0 +1,171 @@ +package ru.mtuci.messenger.service; + +import ru.mtuci.messenger.exception.PayloadTooLargeException; +import ru.mtuci.messenger.exception.UnsupportedMediaException; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; + +public final class ImageFiles { + + private static final byte[] PNG_SIGNATURE = new byte[] {(byte) 137, 80, 78, 71, 13, 10, 26, 10}; + + private ImageFiles() { + } + + public record Image(String contentType, long width, long height) { + } + + public static byte[] readLimited(InputStream input, int maxBytes) throws IOException { + byte[] buffer = new byte[8192]; + ByteArrayOutputStream body = new ByteArrayOutputStream(Math.min(maxBytes, 64 * 1024)); + int total = 0; + int read; + while ((read = input.read(buffer)) >= 0) { + if (read == 0) { + continue; + } + if (total > maxBytes - read) { + throw new PayloadTooLargeException(); + } + body.write(buffer, 0, read); + total += read; + } + return body.toByteArray(); + } + + public static Image inspect(byte[] data, String declaredType, long maxPixels) { + if (data.length == 0) { + throw new UnsupportedMediaException(); + } + Image image = "image/png".equals(declaredType) ? png(data) : jpeg(data); + if (image == null || !declaredType.equals(image.contentType()) || image.width() <= 0 || image.height() <= 0) { + throw new UnsupportedMediaException(); + } + if (image.width() > maxPixels || image.height() > maxPixels || image.width() * image.height() > maxPixels) { + throw new PayloadTooLargeException(); + } + return image; + } + + private static Image png(byte[] data) { + if (data.length < 33 || !startsWith(data, PNG_SIGNATURE)) { + return null; + } + int offset = 8; + Long width = null; + Long height = null; + boolean end = false; + while (offset + 12 <= data.length) { + long length = u32(data, offset); + if (length > data.length || offset + 12L + length > data.length) { + return null; + } + String type = new String(data, offset + 4, 4, StandardCharsets.US_ASCII); + if (!type.chars().allMatch(Character::isLetter)) { + return null; + } + if ("IHDR".equals(type)) { + if (length != 13 || width != null || !validPngHeader(data, offset + 8)) { + return null; + } + width = u32(data, offset + 8); + height = u32(data, offset + 12); + } + offset += 12 + (int) length; + if ("IEND".equals(type)) { + end = length == 0; + break; + } + } + if (!end || width == null || offset != data.length) { + return null; + } + return new Image("image/png", width, height); + } + + private static boolean validPngHeader(byte[] data, int offset) { + int bitDepth = data[offset + 8] & 0xff; + int color = data[offset + 9] & 0xff; + int compression = data[offset + 10] & 0xff; + int filter = data[offset + 11] & 0xff; + int interlace = data[offset + 12] & 0xff; + if (compression != 0 || filter != 0 || (interlace != 0 && interlace != 1)) { + return false; + } + return switch (color) { + case 0 -> bitDepth == 1 || bitDepth == 2 || bitDepth == 4 || bitDepth == 8 || bitDepth == 16; + case 3 -> bitDepth == 1 || bitDepth == 2 || bitDepth == 4 || bitDepth == 8; + case 2, 4, 6 -> bitDepth == 8 || bitDepth == 16; + default -> false; + }; + } + + private static Image jpeg(byte[] data) { + if (data.length < 4 || (data[0] & 0xff) != 0xff || (data[1] & 0xff) != 0xd8) { + return null; + } + int offset = 2; + Long width = null; + Long height = null; + while (offset + 1 < data.length) { + if ((data[offset] & 0xff) != 0xff) { + return null; + } + while (offset < data.length && (data[offset] & 0xff) == 0xff) { + offset++; + } + if (offset >= data.length) { + return null; + } + int marker = data[offset++] & 0xff; + if (marker == 0xd9) { + return width == null ? null : new Image("image/jpeg", width, height); + } + if (marker == 0xd8 || marker == 0x01 || (marker >= 0xd0 && marker <= 0xd7)) { + continue; + } + if (offset + 1 >= data.length) { + return null; + } + int length = ((data[offset] & 0xff) << 8) | (data[offset + 1] & 0xff); + if (length < 2 || offset + length > data.length) { + return null; + } + if (isStartOfFrame(marker)) { + if (length < 7) { + return null; + } + height = (long) ((data[offset + 3] & 0xff) << 8) | (data[offset + 4] & 0xff); + width = (long) ((data[offset + 5] & 0xff) << 8) | (data[offset + 6] & 0xff); + } + offset += length; + } + return null; + } + + private static boolean isStartOfFrame(int marker) { + return marker >= 0xc0 && marker <= 0xcf && marker != 0xc4 && marker != 0xc8 && marker != 0xcc; + } + + private static boolean startsWith(byte[] data, byte[] prefix) { + if (data.length < prefix.length) { + return false; + } + for (int i = 0; i < prefix.length; i++) { + if (data[i] != prefix[i]) { + return false; + } + } + return true; + } + + private static long u32(byte[] data, int offset) { + return ((data[offset] & 0xffL) << 24) + | ((data[offset + 1] & 0xffL) << 16) + | ((data[offset + 2] & 0xffL) << 8) + | (data[offset + 3] & 0xffL); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/ImageVariants.java b/backend/src/main/java/ru/mtuci/messenger/service/ImageVariants.java new file mode 100644 index 00000000..47c395ab --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/ImageVariants.java @@ -0,0 +1,118 @@ +package ru.mtuci.messenger.service; + +import com.luciad.imageio.webp.CompressionType; +import com.luciad.imageio.webp.WebPWriteParam; +import net.coobird.thumbnailator.Thumbnails; + +import javax.imageio.IIOImage; +import javax.imageio.ImageIO; +import javax.imageio.ImageWriteParam; +import javax.imageio.ImageWriter; +import javax.imageio.stream.ImageOutputStream; +import java.awt.Color; +import java.awt.Graphics2D; +import java.awt.RenderingHints; +import java.awt.image.BufferedImage; +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.util.List; +import java.util.Locale; + +/** + * Derivatives for pipeline version 1. + * Thumbnail and crop are baseline JPEG at the given quality, metadata is not copied. + * EXIF orientation is applied before resize. Alpha is composited onto white. + * Optimized is lossy WebP inside 1280×1280 without upscaling; quality is a 0..1 factor + * (0.8 means libwebp quality 80). Small inputs are not promised to shrink. + */ +public final class ImageVariants { + + public record VariantBytes(String name, String contentType, int width, int height, byte[] bytes) { + } + + static { + ImageIO.scanForPlugins(); + } + + private ImageVariants() { + } + + public static int[] fitted(int width, int height, int box, boolean upscale) { + double scale = Math.min(box / (double) width, box / (double) height); + if (!upscale) { + scale = Math.min(1.0, scale); + } + return new int[] { + Math.max(1, (int) Math.floor(width * scale)), + Math.max(1, (int) Math.floor(height * scale)) + }; + } + + public static List render(byte[] original, float jpegQuality, float webpQuality) throws IOException { + BufferedImage oriented = Thumbnails.of(new ByteArrayInputStream(original)) + .scale(1.0) + .useExifOrientation(true) + .asBufferedImage(); + int width = oriented.getWidth(); + int height = oriented.getHeight(); + int[] thumb = fitted(width, height, 256, false); + int[] optimized = fitted(width, height, 1280, false); + VariantBytes thumbnail = new VariantBytes("thumbnail", "image/jpeg", thumb[0], thumb[1], + jpeg(scale(oriented, thumb[0], thumb[1]), jpegQuality)); + VariantBytes crop = new VariantBytes("crop", "image/jpeg", 128, 128, + jpeg(cropSquare(oriented, 128), jpegQuality)); + VariantBytes webp = new VariantBytes("optimized", "image/webp", optimized[0], optimized[1], + webp(scale(oriented, optimized[0], optimized[1]), webpQuality)); + return List.of(thumbnail, crop, webp); + } + + private static BufferedImage cropSquare(BufferedImage source, int size) { + int side = Math.min(source.getWidth(), source.getHeight()); + int x = (source.getWidth() - side) / 2; + int y = (source.getHeight() - side) / 2; + return scale(source.getSubimage(x, y, side, side), size, size); + } + + private static BufferedImage scale(BufferedImage source, int width, int height) { + BufferedImage target = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB); + Graphics2D graphics = target.createGraphics(); + try { + graphics.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR); + graphics.setColor(Color.WHITE); + graphics.fillRect(0, 0, width, height); + graphics.drawImage(source, 0, 0, width, height, null); + } finally { + graphics.dispose(); + } + return target; + } + + private static byte[] jpeg(BufferedImage image, float quality) throws IOException { + ImageWriter writer = ImageIO.getImageWritersByFormatName("jpeg").next(); + ImageWriteParam param = writer.getDefaultWriteParam(); + param.setCompressionMode(ImageWriteParam.MODE_EXPLICIT); + param.setCompressionQuality(quality); + return write(writer, param, image); + } + + private static byte[] webp(BufferedImage image, float quality) throws IOException { + ImageWriter writer = ImageIO.getImageWritersByMIMEType("image/webp").next(); + WebPWriteParam param = new WebPWriteParam(Locale.ROOT); + param.setCompressionMode(ImageWriteParam.MODE_EXPLICIT); + param.setCompressionType(CompressionType.Lossy); + param.setCompressionQuality(quality); + return write(writer, param, image); + } + + private static byte[] write(ImageWriter writer, ImageWriteParam param, BufferedImage image) throws IOException { + ByteArrayOutputStream bytes = new ByteArrayOutputStream(); + try (ImageOutputStream out = ImageIO.createImageOutputStream(bytes)) { + writer.setOutput(out); + writer.write(null, new IIOImage(image, null, null), param); + } finally { + writer.dispose(); + } + return bytes.toByteArray(); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/MessageService.java b/backend/src/main/java/ru/mtuci/messenger/service/MessageService.java new file mode 100644 index 00000000..8d39feb7 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/MessageService.java @@ -0,0 +1,202 @@ +package ru.mtuci.messenger.service; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; +import ru.mtuci.messenger.api.model.CreateMessage; +import ru.mtuci.messenger.api.model.Message; +import ru.mtuci.messenger.api.model.MessagePage; +import ru.mtuci.messenger.domain.AttachmentEntity; +import ru.mtuci.messenger.domain.IdempotencyRecord; +import ru.mtuci.messenger.domain.MessageEntity; +import ru.mtuci.messenger.exception.ConflictException; +import ru.mtuci.messenger.exception.InvalidRequestException; +import ru.mtuci.messenger.exception.NotFoundException; +import ru.mtuci.messenger.mapper.MessageMapper; +import ru.mtuci.messenger.repository.AttachmentRepository; +import ru.mtuci.messenger.repository.ChatMemberRepository; +import ru.mtuci.messenger.repository.IdempotencyRepository; +import ru.mtuci.messenger.repository.MessageRepository; +import org.springframework.data.domain.PageRequest; + +import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.sql.Connection; +import java.time.Duration; +import java.time.Instant; +import java.util.ArrayList; +import java.util.HexFormat; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Objects; +import java.util.Optional; +import java.util.UUID; +import java.util.regex.Pattern; + +@Service +public class MessageService { + + private static final Pattern IDEMPOTENCY_KEY = Pattern.compile("^[A-Za-z0-9._:-]{1,128}$"); + private static final String METHOD = "POST"; + + private final MessageRepository messages; + private final AttachmentRepository attachments; + private final IdempotencyRepository idempotency; + private final ChatMemberRepository members; + private final MessageMapper mapper; + private final JdbcTemplate jdbc; + private final Duration idempotencyTtl; + + public MessageService(MessageRepository messages, AttachmentRepository attachments, IdempotencyRepository idempotency, + ChatMemberRepository members, MessageMapper mapper, JdbcTemplate jdbc, + @Value("${app.idempotency-ttl:PT24H}") Duration idempotencyTtl) { + this.messages = messages; + this.attachments = attachments; + this.idempotency = idempotency; + this.members = members; + this.mapper = mapper; + this.jdbc = jdbc; + this.idempotencyTtl = idempotencyTtl; + } + + @Transactional + public Message create(UUID actorId, UUID chatId, String idempotencyKey, CreateMessage request) { + requireMember(chatId, actorId); + String key = requireKey(idempotencyKey); + List attachmentIds = attachmentIds(request); + requireText(request.getText(), !attachmentIds.isEmpty()); + requireAttachments(chatId, attachmentIds); + String path = "/api/v1/chats/" + chatId + "/messages"; + lock(actorId + "\n" + METHOD + "\n" + path + "\n" + key); + String hash = requestHash(request.getText(), attachmentIds); + Optional existing = idempotency.findByUserIdAndHttpMethodAndPathAndIdempotencyKey( + actorId, METHOD, path, key); + if (existing.isPresent()) { + IdempotencyRecord record = existing.get(); + if (record.getExpiresAt().isAfter(Instant.now())) { + if (!record.getRequestHash().equals(hash)) { + throw new ConflictException("Ключ идемпотентности уже использован с другим телом"); + } + return mapper.toModel(messages.findById(record.getMessageId()).orElseThrow(NotFoundException::new)); + } + idempotency.delete(record); + idempotency.flush(); + } + Instant now = Instant.now(); + MessageEntity saved = messages.saveAndFlush(new MessageEntity(chatId, actorId, request.getText(), attachmentIds)); + idempotency.saveAndFlush(new IdempotencyRecord( + actorId, METHOD, path, key, hash, saved.getId(), now, now.plus(idempotencyTtl))); + return mapper.toModel(messages.findById(saved.getId()).orElseThrow(NotFoundException::new)); + } + + @Transactional(readOnly = true) + public MessagePage list(UUID actorId, UUID chatId, Integer limit, String cursor) { + requireMember(chatId, actorId); + int pageSize = limit == null ? 50 : limit; + CursorCodec.Position position = cursor == null ? null : CursorCodec.decode(cursor, chatId); + List rows = position == null + ? messages.findFirst(chatId, PageRequest.of(0, pageSize + 1)) + : messages.findAfter(chatId, position.createdAt(), position.id(), PageRequest.of(0, pageSize + 1)); + boolean more = rows.size() > pageSize; + List visible = more ? rows.subList(0, pageSize) : rows; + String next = null; + if (more) { + MessageEntity last = visible.get(visible.size() - 1); + next = CursorCodec.encode(chatId, last.getCreatedAt(), last.getId()); + } + return new MessagePage(visible.stream().map(mapper::toModel).toList(), next); + } + + private void lock(String scope) { + byte[] digest = digest(scope); + long key = 0; + for (int i = 0; i < 8; i++) { + key = (key << 8) | (digest[i] & 0xffL); + } + long lockKey = key; + jdbc.execute((Connection connection) -> { + try (var statement = connection.prepareStatement("select pg_advisory_xact_lock(?)")) { + statement.setLong(1, lockKey); + statement.execute(); + } + return null; + }); + } + + private static String requireKey(String key) { + if (key == null || !IDEMPOTENCY_KEY.matcher(key).matches()) { + throw new InvalidRequestException("Некорректный запрос"); + } + return key; + } + + private void requireAttachments(UUID chatId, List attachmentIds) { + if (attachmentIds.isEmpty()) { + return; + } + List found = attachments.findAllById(attachmentIds); + if (found.size() != attachmentIds.size()) { + throw new NotFoundException(); + } + for (AttachmentEntity attachment : found) { + if (!chatId.equals(attachment.getChatId())) { + throw new NotFoundException(); + } + if ("failed".equals(attachment.getStatus())) { + throw new ConflictException("Вложение не удалось обработать"); + } + } + } + + private static List attachmentIds(CreateMessage request) { + List ids = request.getAttachmentIds(); + if (ids == null || ids.isEmpty()) { + return List.of(); + } + if (ids.size() > 10 || ids.stream().anyMatch(Objects::isNull) || new LinkedHashSet<>(ids).size() != ids.size()) { + throw new InvalidRequestException("Некорректный запрос"); + } + return new ArrayList<>(ids); + } + + private static void requireText(String text, boolean hasAttachments) { + if (text == null) { + throw new InvalidRequestException("Некорректный запрос"); + } + int codePoints = text.codePointCount(0, text.length()); + if (codePoints > 4000 || (text.isBlank() && !hasAttachments)) { + throw new InvalidRequestException("Некорректный запрос"); + } + } + + private static String requestHash(String text, List attachmentIds) { + StringBuilder canonical = new StringBuilder(text == null ? "" : text).append('\n'); + for (int i = 0; i < attachmentIds.size(); i++) { + if (i > 0) { + canonical.append(','); + } + canonical.append(attachmentIds.get(i)); + } + return sha256(canonical.toString()); + } + + private static String sha256(String value) { + return HexFormat.of().formatHex(digest(value)); + } + + private static byte[] digest(String value) { + try { + return MessageDigest.getInstance("SHA-256").digest(value.getBytes(StandardCharsets.UTF_8)); + } catch (NoSuchAlgorithmException exception) { + throw new IllegalStateException(exception); + } + } + + private void requireMember(UUID chatId, UUID actorId) { + if (!members.existsByChatIdAndUserId(chatId, actorId)) { + throw new NotFoundException(); + } + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/RetryDelay.java b/backend/src/main/java/ru/mtuci/messenger/service/RetryDelay.java new file mode 100644 index 00000000..06691c3f --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/RetryDelay.java @@ -0,0 +1,30 @@ +package ru.mtuci.messenger.service; + +import java.util.concurrent.ThreadLocalRandom; + +public final class RetryDelay { + + private RetryDelay() { + } + + public static long exponentialMillis(int attempt, long baseMs, long capMs) { + int steps = Math.max(attempt, 1); + long delay = Math.max(baseMs, 1); + for (int i = 1; i < steps && delay < capMs; i++) { + if (delay > capMs / 2) { + return capMs; + } + delay *= 2; + } + return Math.min(delay, capMs); + } + + public static long millis(int attempt, long baseMs, long capMs) { + long exp = exponentialMillis(attempt, baseMs, capMs); + long low = exp / 2; + if (exp <= low) { + return exp; + } + return low + ThreadLocalRandom.current().nextLong(exp - low + 1); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/TextRules.java b/backend/src/main/java/ru/mtuci/messenger/service/TextRules.java new file mode 100644 index 00000000..d32c4b80 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/TextRules.java @@ -0,0 +1,35 @@ +package ru.mtuci.messenger.service; + +import ru.mtuci.messenger.exception.InvalidRequestException; + +public final class TextRules { + + private TextRules() { + } + + public static void displayName(String value) { + text(value, 1, 100); + } + + public static void title(String value) { + text(value, 1, 100); + } + + public static void message(String value) { + text(value, 1, 4000); + } + + public static void password(String value) { + text(value, 12, 128); + } + + private static void text(String value, int minCodePoints, int maxCodePoints) { + if (value == null || value.isBlank()) { + throw new InvalidRequestException("Некорректный запрос"); + } + int codePoints = value.codePointCount(0, value.length()); + if (codePoints < minCodePoints || codePoints > maxCodePoints) { + throw new InvalidRequestException("Некорректный запрос"); + } + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/service/UserService.java b/backend/src/main/java/ru/mtuci/messenger/service/UserService.java new file mode 100644 index 00000000..d853eb83 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/service/UserService.java @@ -0,0 +1,147 @@ +package ru.mtuci.messenger.service; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.dao.DataIntegrityViolationException; +import org.springframework.security.crypto.password.PasswordEncoder; +import org.springframework.stereotype.Service; +import org.springframework.transaction.support.TransactionTemplate; +import ru.mtuci.messenger.api.model.CreateUser; +import ru.mtuci.messenger.api.model.Session; +import ru.mtuci.messenger.api.model.User; +import ru.mtuci.messenger.cache.SessionCache; +import ru.mtuci.messenger.domain.SessionEntity; +import ru.mtuci.messenger.domain.UserAccount; +import ru.mtuci.messenger.exception.ConflictException; +import ru.mtuci.messenger.exception.NotFoundException; +import ru.mtuci.messenger.exception.UnauthorizedException; +import ru.mtuci.messenger.mapper.UserMapper; +import ru.mtuci.messenger.repository.SessionRepository; +import ru.mtuci.messenger.repository.UserRepository; + +import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.security.SecureRandom; +import java.time.Duration; +import java.time.Instant; +import java.util.Base64; +import java.util.HexFormat; +import java.util.Optional; +import java.util.UUID; + +@Service +public class UserService { + + private final UserRepository users; + private final SessionRepository sessions; + private final SessionCache cache; + private final UserMapper mapper; + private final PasswordEncoder passwords; + private final TransactionTemplate transactions; + private final Duration sessionTtl; + private final SecureRandom random = new SecureRandom(); + private volatile String dummyHash; + + public UserService(UserRepository users, SessionRepository sessions, SessionCache cache, UserMapper mapper, + PasswordEncoder passwords, TransactionTemplate transactions, + @Value("${app.session-ttl}") Duration sessionTtl) { + this.users = users; + this.sessions = sessions; + this.cache = cache; + this.mapper = mapper; + this.passwords = passwords; + this.transactions = transactions; + this.sessionTtl = sessionTtl; + } + + public User register(CreateUser request) { + TextRules.displayName(request.getDisplayName()); + TextRules.password(request.getPassword()); + try { + return transactions.execute(status -> mapper.toModel(users.saveAndFlush(new UserAccount( + request.getUsername(), + request.getDisplayName(), + passwords.encode(request.getPassword()) + )))); + } catch (DataIntegrityViolationException exception) { + throw new ConflictException("Имя пользователя занято"); + } + } + + public User current(UUID userId) { + return users.findById(userId).map(mapper::toModel).orElseThrow(NotFoundException::new); + } + + public Session login(String authorization) { + String[] credentials = basic(authorization); + UserAccount user = users.findByUsername(credentials[0]).orElse(null); + String hash = user == null ? dummyHash() : user.getPasswordHash(); + if (user == null || !passwords.matches(credentials[1], hash)) { + throw UnauthorizedException.basic(); + } + String token = newToken(); + SessionEntity session = sessions.saveAndFlush(new SessionEntity(tokenHash(token), user.getId(), sessionTtl)); + cache.put(tokenHash(token), user.getId(), session.getExpiresAt()); + return mapper.toSession(token, session.getExpiresAt(), user); + } + + public Optional authenticate(String token) { + String hash = tokenHash(token); + Optional cached = cache.find(hash); + if (cached.isPresent()) { + return cached; + } + return sessions.findActive(hash, Instant.now()).map(session -> { + cache.put(hash, session.getUserId(), session.getExpiresAt()); + return session.getUserId(); + }); + } + + public void revoke(String token) { + String hash = tokenHash(token); + if (sessions.revoke(hash, Instant.now()) == 0) { + throw UnauthorizedException.bearer(); + } + cache.evict(hash); + } + + public static String tokenHash(String token) { + try { + return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(token.getBytes(StandardCharsets.UTF_8))); + } catch (NoSuchAlgorithmException exception) { + throw new IllegalStateException(exception); + } + } + + private String newToken() { + byte[] bytes = new byte[32]; + random.nextBytes(bytes); + return Base64.getUrlEncoder().withoutPadding().encodeToString(bytes); + } + + private String dummyHash() { + String current = dummyHash; + if (current == null) { + current = passwords.encode("not-a-real-password"); + dummyHash = current; + } + return current; + } + + private static String[] basic(String header) { + if (header == null || !header.regionMatches(true, 0, "Basic ", 0, 6)) { + throw UnauthorizedException.basic(); + } + String decoded; + try { + decoded = new String(Base64.getDecoder().decode(header.substring(6).trim()), StandardCharsets.UTF_8); + } catch (IllegalArgumentException exception) { + throw UnauthorizedException.basic(); + } + int colon = decoded.indexOf(':'); + if (colon <= 0) { + throw UnauthorizedException.basic(); + } + return new String[]{decoded.substring(0, colon), decoded.substring(colon + 1)}; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/storage/ObjectStorage.java b/backend/src/main/java/ru/mtuci/messenger/storage/ObjectStorage.java new file mode 100644 index 00000000..60470501 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/storage/ObjectStorage.java @@ -0,0 +1,149 @@ +package ru.mtuci.messenger.storage; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Component; +import ru.mtuci.messenger.exception.NotFoundException; +import ru.mtuci.messenger.exception.UnavailableException; +import software.amazon.awssdk.core.sync.RequestBody; +import software.amazon.awssdk.services.s3.S3Client; +import software.amazon.awssdk.services.s3.model.BucketAlreadyExistsException; +import software.amazon.awssdk.services.s3.model.BucketAlreadyOwnedByYouException; +import software.amazon.awssdk.services.s3.model.CreateBucketRequest; +import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; +import software.amazon.awssdk.services.s3.model.GetObjectRequest; +import software.amazon.awssdk.services.s3.model.HeadBucketRequest; +import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; +import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; +import software.amazon.awssdk.services.s3.model.NoSuchBucketException; +import software.amazon.awssdk.services.s3.model.NoSuchKeyException; +import software.amazon.awssdk.services.s3.model.PutObjectRequest; +import software.amazon.awssdk.services.s3.model.S3Exception; +import software.amazon.awssdk.services.s3.model.S3Object; + +import java.io.InputStream; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; + +@Component +public class ObjectStorage { + + private static final Logger log = LoggerFactory.getLogger(ObjectStorage.class); + + private final S3Client data; + private final S3Client bootstrap; + private final String bucket; + + public ObjectStorage(@Qualifier("s3Data") S3Client data, + @Qualifier("s3Bootstrap") S3Client bootstrap, + @Value("${app.s3.bucket}") String bucket) { + this.data = data; + this.bootstrap = bootstrap; + this.bucket = bucket; + } + + public record StoredObject(String key, Instant modified) { + } + + public void ensureBucket() { + try { + bootstrap.headBucket(HeadBucketRequest.builder().bucket(bucket).build()); + } catch (NoSuchBucketException missing) { + createBucket(); + } catch (S3Exception exception) { + if (exception.statusCode() == 404) { + createBucket(); + return; + } + throw storageFailure(exception); + } catch (RuntimeException exception) { + throw storageFailure(exception); + } + } + + public void put(String key, byte[] body, String contentType) { + try { + data.putObject(PutObjectRequest.builder() + .bucket(bucket) + .key(key) + .contentType(contentType) + .contentLength((long) body.length) + .build(), RequestBody.fromBytes(body)); + } catch (RuntimeException exception) { + throw storageFailure(exception); + } + } + + public InputStream get(String key) { + try { + return data.getObject(GetObjectRequest.builder().bucket(bucket).key(key).build()); + } catch (NoSuchKeyException missing) { + throw new NotFoundException(); + } catch (S3Exception exception) { + if (exception.statusCode() == 404) { + throw new NotFoundException(); + } + throw storageFailure(exception); + } catch (RuntimeException exception) { + throw storageFailure(exception); + } + } + + public List list(String prefix) { + List objects = new ArrayList<>(); + String token = null; + do { + ListObjectsV2Request.Builder request = ListObjectsV2Request.builder() + .bucket(bucket) + .prefix(prefix); + if (token != null) { + request.continuationToken(token); + } + ListObjectsV2Response page; + try { + page = data.listObjectsV2(request.build()); + } catch (RuntimeException exception) { + throw storageFailure(exception); + } + if (page.contents() != null) { + for (S3Object object : page.contents()) { + Instant modified = object.lastModified() == null ? Instant.EPOCH : object.lastModified(); + objects.add(new StoredObject(object.key(), modified)); + } + } + token = page.nextContinuationToken(); + } while (token != null); + return objects; + } + + public void deleteQuietly(String key) { + try { + data.deleteObject(DeleteObjectRequest.builder().bucket(bucket).key(key).build()); + } catch (RuntimeException exception) { + log.warn("s3 delete failed status={}", exception.getClass().getSimpleName()); + } + } + + private void createBucket() { + try { + bootstrap.createBucket(CreateBucketRequest.builder().bucket(bucket).build()); + } catch (BucketAlreadyExistsException | BucketAlreadyOwnedByYouException ignored) { + // another replica created the bucket + } catch (S3Exception exception) { + if (exception.statusCode() == 409) { + return; + } + throw storageFailure(exception); + } catch (RuntimeException exception) { + throw storageFailure(exception); + } + } + + private static UnavailableException storageFailure(RuntimeException exception) { + log.warn("s3 failure type={}", exception.getClass().getSimpleName()); + return new UnavailableException("Хранилище временно недоступно"); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/storage/OrphanSweeper.java b/backend/src/main/java/ru/mtuci/messenger/storage/OrphanSweeper.java new file mode 100644 index 00000000..e9aa38db --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/storage/OrphanSweeper.java @@ -0,0 +1,81 @@ +package ru.mtuci.messenger.storage; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.context.annotation.Profile; +import org.springframework.scheduling.annotation.Scheduled; +import org.springframework.stereotype.Component; +import ru.mtuci.messenger.repository.AttachmentRepository; +import ru.mtuci.messenger.repository.AttachmentVariantRepository; + +import java.time.Duration; +import java.time.Instant; +import java.util.List; +import java.util.function.Predicate; + +@Component +@Profile("!migrate") +@ConditionalOnProperty(name = "app.s3.sweep-enabled", havingValue = "true", matchIfMissing = true) +public class OrphanSweeper { + + private static final Logger log = LoggerFactory.getLogger(OrphanSweeper.class); + private static final List ACTIVE = List.of("queued", "processing"); + + private final ObjectStorage storage; + private final AttachmentRepository attachments; + private final AttachmentVariantRepository variants; + private final Duration grace; + private final Duration stagingRetention; + + public OrphanSweeper(ObjectStorage storage, AttachmentRepository attachments, AttachmentVariantRepository variants, + @Value("${app.s3.orphan-grace:2m}") Duration grace, + @Value("${app.staging.retention:86400s}") Duration stagingRetention) { + this.storage = storage; + this.attachments = attachments; + this.variants = variants; + this.grace = grace; + this.stagingRetention = stagingRetention; + } + + @Scheduled(fixedDelayString = "${app.s3.sweep-interval-ms:60000}", + initialDelayString = "${app.s3.sweep-initial-delay-ms:60000}") + public int sweep() { + Instant now = Instant.now(); + int removed = sweepPrefix("originals/", now.minus(grace), attachments::existsByObjectKey); + removed += sweepPrefix("variants/", now.minus(grace), variants::existsByObjectKey); + removed += sweepPrefix("staging/", now.minus(stagingRetention), this::stagingActive); + if (removed > 0) { + log.info("removed {} unreferenced objects", removed); + } + return removed; + } + + public int sweepOlderThan(Instant cutoff) { + int removed = sweepPrefix("originals/", cutoff, attachments::existsByObjectKey); + removed += sweepPrefix("variants/", cutoff, variants::existsByObjectKey); + removed += sweepPrefix("staging/", cutoff, this::stagingActive); + return removed; + } + + private boolean stagingActive(String key) { + return attachments.existsByStagingKeyAndStatusIn(key, ACTIVE); + } + + private int sweepPrefix(String prefix, Instant cutoff, Predicate referenced) { + int removed = 0; + for (ObjectStorage.StoredObject object : storage.list(prefix)) { + if (!expiredUnreferenced(object.modified(), cutoff, referenced.test(object.key()))) { + continue; + } + storage.deleteQuietly(object.key()); + removed++; + } + return removed; + } + + static boolean expiredUnreferenced(Instant modified, Instant cutoff, boolean referenced) { + return !referenced && !modified.isAfter(cutoff); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/storage/S3Config.java b/backend/src/main/java/ru/mtuci/messenger/storage/S3Config.java new file mode 100644 index 00000000..98b940e5 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/storage/S3Config.java @@ -0,0 +1,50 @@ +package ru.mtuci.messenger.storage; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; +import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; +import software.amazon.awssdk.http.apache.ApacheHttpClient; +import software.amazon.awssdk.regions.Region; +import software.amazon.awssdk.services.s3.S3Client; +import software.amazon.awssdk.services.s3.S3Configuration; + +import java.net.URI; +import java.time.Duration; + +@Configuration +public class S3Config { + + @Bean(name = "s3Data", destroyMethod = "close") + S3Client s3Data(@Value("${app.s3.endpoint}") String endpoint, + @Value("${app.s3.region}") String region, + @Value("${app.s3.access-key}") String accessKey, + @Value("${app.s3.secret-key}") String secretKey) { + return client(endpoint, region, accessKey, secretKey); + } + + @Bean(name = "s3Bootstrap", destroyMethod = "close") + S3Client s3Bootstrap(@Value("${app.s3.endpoint}") String endpoint, + @Value("${app.s3.region}") String region, + @Value("${app.s3.bootstrap-access-key}") String accessKey, + @Value("${app.s3.bootstrap-secret-key}") String secretKey) { + return client(endpoint, region, accessKey, secretKey); + } + + private static S3Client client(String endpoint, String region, String accessKey, String secretKey) { + return S3Client.builder() + .endpointOverride(URI.create(endpoint)) + .region(Region.of(region)) + .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(accessKey, secretKey))) + .serviceConfiguration(S3Configuration.builder().pathStyleAccessEnabled(true).build()) + .httpClientBuilder(ApacheHttpClient.builder() + .connectionTimeout(Duration.ofSeconds(2)) + .socketTimeout(Duration.ofSeconds(8))) + .overrideConfiguration(config -> config + .apiCallTimeout(Duration.ofSeconds(10)) + .apiCallAttemptTimeout(Duration.ofSeconds(8))) + .build(); + } + +} diff --git a/backend/src/main/java/ru/mtuci/messenger/storage/S3HealthIndicator.java b/backend/src/main/java/ru/mtuci/messenger/storage/S3HealthIndicator.java new file mode 100644 index 00000000..b222715f --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/storage/S3HealthIndicator.java @@ -0,0 +1,30 @@ +package ru.mtuci.messenger.storage; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.boot.health.contributor.Health; +import org.springframework.boot.health.contributor.HealthIndicator; +import org.springframework.stereotype.Component; + +@Component +public class S3HealthIndicator implements HealthIndicator { + + private static final Logger log = LoggerFactory.getLogger(S3HealthIndicator.class); + + private final ObjectStorage storage; + + public S3HealthIndicator(ObjectStorage storage) { + this.storage = storage; + } + + @Override + public Health health() { + try { + storage.ensureBucket(); + return Health.up().build(); + } catch (RuntimeException exception) { + log.info("s3 readiness failed type={}", exception.getClass().getSimpleName()); + return Health.down().build(); + } + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/storage/SweepConfig.java b/backend/src/main/java/ru/mtuci/messenger/storage/SweepConfig.java new file mode 100644 index 00000000..0681705f --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/storage/SweepConfig.java @@ -0,0 +1,11 @@ +package ru.mtuci.messenger.storage; + +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Profile; +import org.springframework.scheduling.annotation.EnableScheduling; + +@Configuration +@EnableScheduling +@Profile("!migrate") +public class SweepConfig { +} diff --git a/backend/src/main/java/ru/mtuci/messenger/web/ApiAdvice.java b/backend/src/main/java/ru/mtuci/messenger/web/ApiAdvice.java new file mode 100644 index 00000000..d0cda9a0 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/web/ApiAdvice.java @@ -0,0 +1,99 @@ +package ru.mtuci.messenger.web; + +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; +import jakarta.validation.ConstraintViolationException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.context.annotation.Profile; +import org.springframework.dao.DataAccessException; +import org.springframework.transaction.CannotCreateTransactionException; +import org.springframework.http.HttpHeaders; +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; +import org.springframework.http.converter.HttpMessageNotReadableException; +import org.springframework.web.HttpMediaTypeNotSupportedException; +import org.springframework.web.HttpRequestMethodNotSupportedException; +import org.springframework.web.bind.MethodArgumentNotValidException; +import org.springframework.web.bind.MissingServletRequestParameterException; +import org.springframework.web.bind.annotation.ExceptionHandler; +import org.springframework.web.bind.annotation.RestControllerAdvice; +import org.springframework.web.method.annotation.HandlerMethodValidationException; +import org.springframework.web.method.annotation.MethodArgumentTypeMismatchException; +import org.springframework.web.servlet.NoHandlerFoundException; +import org.springframework.web.servlet.resource.NoResourceFoundException; +import ru.mtuci.messenger.api.model.Error; +import ru.mtuci.messenger.api.model.ErrorError; +import ru.mtuci.messenger.exception.ApiException; +import ru.mtuci.messenger.exception.UnauthorizedException; + +@RestControllerAdvice +@Profile("!worker") +public class ApiAdvice { + + private static final Logger log = LoggerFactory.getLogger(ApiAdvice.class); + + private final InstanceIdentity instanceId; + + public ApiAdvice(InstanceIdentity instanceId) { + this.instanceId = instanceId; + } + + @ExceptionHandler(ApiException.class) + public ResponseEntity handleApi(HttpServletRequest request, HttpServletResponse response, ApiException exception) { + log.info("api error code={}", exception.code().getValue()); + String challenge = exception instanceof UnauthorizedException unauthorized ? unauthorized.challenge() : null; + return response(request, response, exception.status(), exception.code(), exception.getMessage(), challenge); + } + + @ExceptionHandler({ + HttpMessageNotReadableException.class, + MethodArgumentNotValidException.class, + ConstraintViolationException.class, + HandlerMethodValidationException.class, + MethodArgumentTypeMismatchException.class, + MissingServletRequestParameterException.class, + HttpRequestMethodNotSupportedException.class + }) + public ResponseEntity handleInvalid(HttpServletRequest request, HttpServletResponse response, Exception exception) { + log.info("invalid request type={}", exception.getClass().getSimpleName()); + return response(request, response, HttpStatus.BAD_REQUEST, ErrorError.CodeEnum.INVALID_REQUEST, "Некорректный запрос", null); + } + + @ExceptionHandler(HttpMediaTypeNotSupportedException.class) + public ResponseEntity handleMedia(HttpServletRequest request, HttpServletResponse response) { + return response(request, response, HttpStatus.UNSUPPORTED_MEDIA_TYPE, ErrorError.CodeEnum.UNSUPPORTED_MEDIA_TYPE, + "Неподдерживаемый тип содержимого", null); + } + + @ExceptionHandler({NoResourceFoundException.class, NoHandlerFoundException.class}) + public ResponseEntity handleMissing(HttpServletRequest request, HttpServletResponse response) { + return response(request, response, HttpStatus.NOT_FOUND, ErrorError.CodeEnum.NOT_FOUND, "Объект не найден", null); + } + + @ExceptionHandler({DataAccessException.class, CannotCreateTransactionException.class}) + public ResponseEntity handleDataAccess(HttpServletRequest request, HttpServletResponse response, Exception exception) { + log.warn("data access failure type={}", exception.getClass().getSimpleName()); + return response(request, response, HttpStatus.SERVICE_UNAVAILABLE, ErrorError.CodeEnum.UNAVAILABLE, + "Сервис временно недоступен", null); + } + + @ExceptionHandler(Exception.class) + public ResponseEntity handleUnexpected(HttpServletRequest request, HttpServletResponse response, Exception exception) { + log.error("unexpected failure type={}", exception.getClass().getName(), exception); + return response(request, response, HttpStatus.INTERNAL_SERVER_ERROR, ErrorError.CodeEnum.INTERNAL_ERROR, + "Внутренняя ошибка", null); + } + + private ResponseEntity response(HttpServletRequest request, HttpServletResponse response, HttpStatus status, + ErrorError.CodeEnum code, String message, String challenge) { + String requestId = RequestIdFilter.id(request); + response.setHeader(RequestIdFilter.HEADER, requestId); + response.setHeader(InstanceIdFilter.HEADER, instanceId.value()); + HttpHeaders headers = new HttpHeaders(); + if (challenge != null) { + headers.set(HttpHeaders.WWW_AUTHENTICATE, challenge); + } + return new ResponseEntity<>(new Error(new ErrorError(code, message, requestId)), headers, status); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/web/BearerAuthFilter.java b/backend/src/main/java/ru/mtuci/messenger/web/BearerAuthFilter.java new file mode 100644 index 00000000..dcbca8f3 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/web/BearerAuthFilter.java @@ -0,0 +1,75 @@ +package ru.mtuci.messenger.web; + +import jakarta.servlet.FilterChain; +import jakarta.servlet.ServletException; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.context.annotation.Profile; +import org.springframework.core.Ordered; +import org.springframework.core.annotation.Order; +import org.springframework.http.HttpMethod; +import org.springframework.stereotype.Component; +import org.springframework.web.filter.OncePerRequestFilter; +import org.springframework.web.servlet.HandlerExceptionResolver; +import ru.mtuci.messenger.exception.UnauthorizedException; +import ru.mtuci.messenger.service.UserService; + +import java.io.IOException; +import java.util.UUID; + +@Component +@Profile("!worker") +@Order(Ordered.HIGHEST_PRECEDENCE + 10) +public class BearerAuthFilter extends OncePerRequestFilter { + + public static final String TOKEN = "token"; + + private final UserService users; + private final HandlerExceptionResolver errors; + + public BearerAuthFilter(UserService users, @Qualifier("handlerExceptionResolver") HandlerExceptionResolver errors) { + this.users = users; + this.errors = errors; + } + + public static UUID userId(HttpServletRequest request) { + return (UUID) request.getAttribute("userId"); + } + + @Override + protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain) + throws ServletException, IOException { + if (isPublic(request)) { + chain.doFilter(request, response); + return; + } + try { + String header = request.getHeader("Authorization"); + if (header == null || !header.regionMatches(true, 0, "Bearer ", 0, 7)) { + throw UnauthorizedException.bearer(); + } + String token = header.substring(7).trim(); + UUID userId = users.authenticate(token).orElseThrow(UnauthorizedException::bearer); + request.setAttribute("userId", userId); + request.setAttribute(TOKEN, token); + } catch (RuntimeException exception) { + errors.resolveException(request, response, null, exception); + return; + } + chain.doFilter(request, response); + } + + private static boolean isPublic(HttpServletRequest request) { + String path = request.getRequestURI(); + if (path.startsWith("/internal/demo/")) { + return true; + } + if (HttpMethod.GET.matches(request.getMethod()) && (path.equals("/health/live") || path.equals("/health/ready") + || path.startsWith("/actuator/"))) { + return true; + } + return HttpMethod.POST.matches(request.getMethod()) + && (path.equals("/api/v1/users") || path.equals("/api/v1/auth/sessions")); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/web/InstanceIdFilter.java b/backend/src/main/java/ru/mtuci/messenger/web/InstanceIdFilter.java new file mode 100644 index 00000000..67ddc900 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/web/InstanceIdFilter.java @@ -0,0 +1,34 @@ +package ru.mtuci.messenger.web; + +import jakarta.servlet.FilterChain; +import jakarta.servlet.ServletException; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; +import org.springframework.context.annotation.Profile; +import org.springframework.core.Ordered; +import org.springframework.core.annotation.Order; +import org.springframework.stereotype.Component; +import org.springframework.web.filter.OncePerRequestFilter; + +import java.io.IOException; + +@Component +@Profile("!worker") +@Order(Ordered.HIGHEST_PRECEDENCE) +public class InstanceIdFilter extends OncePerRequestFilter { + + static final String HEADER = "X-Instance-Id"; + + private final InstanceIdentity instanceId; + + public InstanceIdFilter(InstanceIdentity instanceId) { + this.instanceId = instanceId; + } + + @Override + protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) + throws ServletException, IOException { + response.setHeader(HEADER, instanceId.value()); + filterChain.doFilter(request, response); + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/web/InstanceIdentity.java b/backend/src/main/java/ru/mtuci/messenger/web/InstanceIdentity.java new file mode 100644 index 00000000..6d60dffd --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/web/InstanceIdentity.java @@ -0,0 +1,19 @@ +package ru.mtuci.messenger.web; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Component; + +@Component +public class InstanceIdentity { + + private final String value; + + public InstanceIdentity(@Value("${app.instance-id:local}") String configured) { + String trimmed = configured == null ? "" : configured.trim(); + this.value = trimmed.isEmpty() ? "local" : trimmed; + } + + public String value() { + return value; + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/web/RequestIdFilter.java b/backend/src/main/java/ru/mtuci/messenger/web/RequestIdFilter.java new file mode 100644 index 00000000..c28eb92c --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/web/RequestIdFilter.java @@ -0,0 +1,47 @@ +package ru.mtuci.messenger.web; + +import jakarta.servlet.FilterChain; +import jakarta.servlet.ServletException; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; +import org.slf4j.MDC; +import org.springframework.context.annotation.Profile; +import org.springframework.core.Ordered; +import org.springframework.core.annotation.Order; +import org.springframework.stereotype.Component; +import org.springframework.web.filter.OncePerRequestFilter; + +import java.io.IOException; +import java.util.UUID; + +@Component +@Profile("!worker") +@Order(Ordered.HIGHEST_PRECEDENCE) +public class RequestIdFilter extends OncePerRequestFilter { + + static final String HEADER = "X-Request-Id"; + + static String id(HttpServletRequest request) { + Object existing = request.getAttribute(HEADER); + if (existing != null) { + return existing.toString(); + } + String header = request.getHeader(HEADER); + String id = header == null || header.isBlank() ? UUID.randomUUID().toString() : header; + request.setAttribute(HEADER, id); + return id; + } + + @Override + protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) + throws ServletException, IOException { + String requestId = id(request); + response.setHeader(HEADER, requestId); + MDC.put("requestId", requestId); + try { + filterChain.doFilter(request, response); + } finally { + MDC.remove("requestId"); + } + } +} diff --git a/backend/src/main/java/ru/mtuci/messenger/web/TimeFormatConfig.java b/backend/src/main/java/ru/mtuci/messenger/web/TimeFormatConfig.java new file mode 100644 index 00000000..266ab7b4 --- /dev/null +++ b/backend/src/main/java/ru/mtuci/messenger/web/TimeFormatConfig.java @@ -0,0 +1,25 @@ +package ru.mtuci.messenger.web; + +import com.fasterxml.jackson.annotation.JsonFormat; +import org.springframework.boot.jackson.autoconfigure.JsonMapperBuilderCustomizer; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.core.Ordered; +import org.springframework.core.annotation.Order; + +import java.time.OffsetDateTime; + +@Configuration +public class TimeFormatConfig { + + /** + * Python 3.9 {@code fromisoformat} принимает только 0, 3 или 6 знаков дробной секунды. + * Jackson по умолчанию отбрасывает хвостовые нули, и время вроде {@code .38078Z} ломает публичные тесты. + */ + @Bean + @Order(Ordered.LOWEST_PRECEDENCE) + JsonMapperBuilderCustomizer utcMicroseconds() { + JsonFormat.Value format = JsonFormat.Value.forPattern("yyyy-MM-dd'T'HH:mm:ss.SSSSSSXXX"); + return builder -> builder.withConfigOverride(OffsetDateTime.class, override -> override.setFormat(format)); + } +} diff --git a/backend/src/main/resources/application-migrate.yaml b/backend/src/main/resources/application-migrate.yaml new file mode 100644 index 00000000..5cd430e2 --- /dev/null +++ b/backend/src/main/resources/application-migrate.yaml @@ -0,0 +1,14 @@ +spring: + autoconfigure: + exclude: + - org.springframework.boot.kafka.autoconfigure.KafkaAutoConfiguration + +app: + broker: + enabled: false + outbox: + enabled: false + worker: + consume: false + s3: + sweep-enabled: false diff --git a/backend/src/main/resources/application-worker.yaml b/backend/src/main/resources/application-worker.yaml new file mode 100644 index 00000000..bdfa3f8f --- /dev/null +++ b/backend/src/main/resources/application-worker.yaml @@ -0,0 +1,9 @@ +spring: + main: + web-application-type: none + +app: + worker: + consume: true + s3: + sweep-enabled: false diff --git a/backend/src/main/resources/application.yaml b/backend/src/main/resources/application.yaml new file mode 100644 index 00000000..f5c6031e --- /dev/null +++ b/backend/src/main/resources/application.yaml @@ -0,0 +1,168 @@ +server: + address: ${APP_HOST:0.0.0.0} + port: ${APP_PORT:8080} + shutdown: graceful + tomcat: + connection-timeout: 3s + max-http-form-post-size: 12MB + max-swallow-size: 12MB + error: + include-message: never + include-stacktrace: never + include-binding-errors: never + whitelabel: + enabled: false + +spring: + application: + name: messenger + lifecycle: + timeout-per-shutdown-phase: 4s + threads: + virtual: + enabled: false + mvc: + problemdetails: + enabled: false + throw-exception-if-no-handler-found: true + web: + resources: + add-mappings: false + jackson: + use-jackson2-defaults: false + property-naming-strategy: SNAKE_CASE + time-zone: UTC + default-property-inclusion: always + deserialization: + fail-on-unknown-properties: true + datatype: + datetime: + write-dates-as-timestamps: false + write-utc-as-offset: false + datasource: + url: ${SPRING_DATASOURCE_URL:jdbc:postgresql://localhost:5432/messenger} + username: ${SPRING_DATASOURCE_USERNAME:messenger} + password: ${SPRING_DATASOURCE_PASSWORD:messenger} + hikari: + maximum-pool-size: 10 + minimum-idle: 1 + connection-timeout: 3000 + validation-timeout: 1000 + jpa: + open-in-view: false + hibernate: + ddl-auto: validate + properties: + hibernate.jdbc.time_zone: UTC + liquibase: + change-log: classpath:db/changelog/db.changelog-master.yaml + data: + redis: + host: ${SPRING_DATA_REDIS_HOST:localhost} + port: ${SPRING_DATA_REDIS_PORT:6379} + timeout: 1s + connect-timeout: 1s + repositories: + enabled: false + kafka: + bootstrap-servers: ${SPRING_KAFKA_BOOTSTRAP_SERVERS:localhost:9092} + producer: + acks: all + key-serializer: org.apache.kafka.common.serialization.StringSerializer + value-serializer: org.apache.kafka.common.serialization.StringSerializer + properties: + enable.idempotence: true + consumer: + group-id: attachment-worker + auto-offset-reset: earliest + enable-auto-commit: false + key-deserializer: org.apache.kafka.common.serialization.StringDeserializer + value-deserializer: org.apache.kafka.common.serialization.StringDeserializer + properties: + max-poll-records: ${WORKER_CONCURRENCY:2} + listener: + ack-mode: manual + missing-topics-fatal: false + +management: + endpoints: + access: + default: READ_ONLY + web: + exposure: + include: + - health + - info + - prometheus + endpoint: + health: + probes: + enabled: true + show-details: always + show-components: always + cache: + time-to-live: 0s + group: + readiness: + include: + - readinessState + - db + - s3 + health: + redis: + enabled: false + kafka: + enabled: false + prometheus: + metrics: + export: + enabled: true + +info: + app: + name: messenger + lab: 5 + +app: + session-ttl: ${SESSION_TTL_SECONDS:3600}s + idempotency-ttl: ${IDEMPOTENCY_TTL_SECONDS:86400}s + instance-id: ${HOSTNAME:local} + upload: + max-bytes: ${MAX_UPLOAD_BYTES:10485760} + max-pixels: ${MAX_IMAGE_PIXELS:40000000} + s3: + endpoint: ${S3_ENDPOINT:http://127.0.0.1:8333} + region: ${S3_REGION:us-east-1} + bucket: ${S3_BUCKET:messenger-attachments} + access-key: ${S3_ACCESS_KEY:messenger} + secret-key: ${S3_SECRET_KEY:messenger} + bootstrap-access-key: ${S3_BOOTSTRAP_ACCESS_KEY:messenger-bootstrap} + bootstrap-secret-key: ${S3_BOOTSTRAP_SECRET_KEY:messenger-bootstrap} + orphan-grace: ${S3_ORPHAN_GRACE:2m} + sweep-interval-ms: ${S3_SWEEP_INTERVAL_MS:60000} + sweep-initial-delay-ms: ${S3_SWEEP_INITIAL_DELAY_MS:60000} + broker: + enabled: true + outbox: + enabled: true + poll-ms: ${OUTBOX_POLL_MS:500} + worker: + consume: false + concurrency: ${WORKER_CONCURRENCY:2} + job: + max-attempts: ${JOB_MAX_ATTEMPTS:5} + timeout: ${JOB_TIMEOUT_SECONDS:60}s + backoff-base-ms: ${JOB_BACKOFF_BASE_MS:1000} + backoff-cap-ms: ${JOB_BACKOFF_CAP_MS:30000} + reaper-ms: ${JOB_REAPER_MS:5000} + demo: + token: ${DEMO_TOKEN:lab-demo} + image: + jpeg-quality: ${JPEG_QUALITY:0.85} + webp-quality: ${WEBP_QUALITY:0.8} + staging: + retention: ${STAGING_RETENTION_SECONDS:86400}s + +logging: + pattern: + console: "%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} %-5level [%X{requestId}] %msg%n" diff --git a/backend/src/main/resources/db/changelog/001-users.yaml b/backend/src/main/resources/db/changelog/001-users.yaml new file mode 100644 index 00000000..44636139 --- /dev/null +++ b/backend/src/main/resources/db/changelog/001-users.yaml @@ -0,0 +1,41 @@ +databaseChangeLog: + - changeSet: + id: 001-users + author: lab2 + changes: + - createTable: + tableName: users + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: users_pkey + - column: + name: username + type: varchar(32) + constraints: + nullable: false + unique: true + uniqueConstraintName: users_username_key + - column: + name: display_name + type: varchar(100) + constraints: + nullable: false + - column: + name: password_hash + type: varchar(255) + constraints: + nullable: false + - column: + name: created_at + type: timestamp with time zone + constraints: + nullable: false + - sql: + sql: alter table users add constraint users_username_pattern check (username ~ '^[a-z0-9_]{3,32}$') + - sql: + sql: alter table users add constraint users_display_name_text check (char_length(btrim(display_name)) > 0) diff --git a/backend/src/main/resources/db/changelog/002-sessions.yaml b/backend/src/main/resources/db/changelog/002-sessions.yaml new file mode 100644 index 00000000..44e8dc5c --- /dev/null +++ b/backend/src/main/resources/db/changelog/002-sessions.yaml @@ -0,0 +1,47 @@ +databaseChangeLog: + - changeSet: + id: 002-sessions + author: lab2 + changes: + - createTable: + tableName: sessions + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: sessions_pkey + - column: + name: token_hash + type: varchar(64) + constraints: + nullable: false + unique: true + uniqueConstraintName: sessions_token_hash_key + - column: + name: user_id + type: uuid + constraints: + nullable: false + - column: + name: expires_at + type: timestamp with time zone + constraints: + nullable: false + - column: + name: revoked_at + type: timestamp with time zone + - column: + name: created_at + type: timestamp with time zone + constraints: + nullable: false + - addForeignKeyConstraint: + baseTableName: sessions + baseColumnNames: user_id + referencedTableName: users + referencedColumnNames: id + constraintName: sessions_user_fk + onDelete: RESTRICT diff --git a/backend/src/main/resources/db/changelog/003-chats.yaml b/backend/src/main/resources/db/changelog/003-chats.yaml new file mode 100644 index 00000000..5683a523 --- /dev/null +++ b/backend/src/main/resources/db/changelog/003-chats.yaml @@ -0,0 +1,77 @@ +databaseChangeLog: + - changeSet: + id: 003-chats + author: lab2 + changes: + - createTable: + tableName: chats + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: chats_pkey + - column: + name: title + type: varchar(100) + constraints: + nullable: false + - column: + name: created_at + type: timestamp with time zone + constraints: + nullable: false + - sql: + sql: alter table chats add constraint chats_title_text check (char_length(btrim(title)) > 0) + - createTable: + tableName: chat_members + columns: + - column: + name: chat_id + type: uuid + constraints: + nullable: false + - column: + name: user_id + type: uuid + constraints: + nullable: false + - column: + name: ordinal + type: int + constraints: + nullable: false + - addPrimaryKey: + tableName: chat_members + columnNames: chat_id, user_id + constraintName: chat_members_pkey + - addForeignKeyConstraint: + baseTableName: chat_members + baseColumnNames: chat_id + referencedTableName: chats + referencedColumnNames: id + constraintName: chat_members_chat_fk + onDelete: RESTRICT + - addForeignKeyConstraint: + baseTableName: chat_members + baseColumnNames: user_id + referencedTableName: users + referencedColumnNames: id + constraintName: chat_members_user_fk + onDelete: RESTRICT + - createIndex: + indexName: chat_members_user_idx + tableName: chat_members + columns: + - column: + name: user_id + - createIndex: + indexName: chats_history_idx + tableName: chats + columns: + - column: + name: created_at + - column: + name: id diff --git a/backend/src/main/resources/db/changelog/004-messages.yaml b/backend/src/main/resources/db/changelog/004-messages.yaml new file mode 100644 index 00000000..29d26b70 --- /dev/null +++ b/backend/src/main/resources/db/changelog/004-messages.yaml @@ -0,0 +1,61 @@ +databaseChangeLog: + - changeSet: + id: 004-messages + author: lab2 + changes: + - createTable: + tableName: messages + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: messages_pkey + - column: + name: chat_id + type: uuid + constraints: + nullable: false + - column: + name: sender_id + type: uuid + constraints: + nullable: false + - column: + name: text + type: varchar(4000) + constraints: + nullable: false + - column: + name: created_at + type: timestamp with time zone + constraints: + nullable: false + - addForeignKeyConstraint: + baseTableName: messages + baseColumnNames: chat_id + referencedTableName: chats + referencedColumnNames: id + constraintName: messages_chat_fk + onDelete: RESTRICT + - addForeignKeyConstraint: + baseTableName: messages + baseColumnNames: sender_id + referencedTableName: users + referencedColumnNames: id + constraintName: messages_sender_fk + onDelete: RESTRICT + - sql: + sql: alter table messages add constraint messages_text_not_blank check (char_length(btrim(text)) > 0 and char_length(text) <= 4000) + - createIndex: + indexName: messages_history_idx + tableName: messages + columns: + - column: + name: chat_id + - column: + name: created_at + - column: + name: id diff --git a/backend/src/main/resources/db/changelog/005-idempotency.yaml b/backend/src/main/resources/db/changelog/005-idempotency.yaml new file mode 100644 index 00000000..c11d9691 --- /dev/null +++ b/backend/src/main/resources/db/changelog/005-idempotency.yaml @@ -0,0 +1,73 @@ +databaseChangeLog: + - changeSet: + id: 005-idempotency + author: lab3 + changes: + - createTable: + tableName: idempotency_records + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: idempotency_records_pkey + - column: + name: user_id + type: uuid + constraints: + nullable: false + - column: + name: http_method + type: varchar(8) + constraints: + nullable: false + - column: + name: path + type: varchar(256) + constraints: + nullable: false + - column: + name: idempotency_key + type: varchar(128) + constraints: + nullable: false + - column: + name: request_hash + type: varchar(64) + constraints: + nullable: false + - column: + name: message_id + type: uuid + constraints: + nullable: false + - column: + name: created_at + type: timestamp with time zone + constraints: + nullable: false + - column: + name: expires_at + type: timestamp with time zone + constraints: + nullable: false + - addUniqueConstraint: + tableName: idempotency_records + columnNames: user_id, http_method, path, idempotency_key + constraintName: idempotency_scope_uq + - addForeignKeyConstraint: + baseTableName: idempotency_records + baseColumnNames: user_id + referencedTableName: users + referencedColumnNames: id + constraintName: idempotency_user_fk + onDelete: RESTRICT + - addForeignKeyConstraint: + baseTableName: idempotency_records + baseColumnNames: message_id + referencedTableName: messages + referencedColumnNames: id + constraintName: idempotency_message_fk + onDelete: RESTRICT diff --git a/backend/src/main/resources/db/changelog/006-attachments.yaml b/backend/src/main/resources/db/changelog/006-attachments.yaml new file mode 100644 index 00000000..20b55d41 --- /dev/null +++ b/backend/src/main/resources/db/changelog/006-attachments.yaml @@ -0,0 +1,127 @@ +databaseChangeLog: + - changeSet: + id: 006-attachments + author: lab4 + changes: + - sql: + sql: alter table messages drop constraint messages_text_not_blank + - createTable: + tableName: attachments + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: attachments_pkey + - column: + name: chat_id + type: uuid + constraints: + nullable: false + - column: + name: owner_id + type: uuid + constraints: + nullable: false + - column: + name: filename + type: varchar(255) + constraints: + nullable: false + - column: + name: content_type + type: varchar(32) + constraints: + nullable: false + - column: + name: size_bytes + type: bigint + constraints: + nullable: false + - column: + name: sha256 + type: varchar(64) + constraints: + nullable: false + - column: + name: object_key + type: varchar(80) + constraints: + nullable: false + - column: + name: status + type: varchar(16) + constraints: + nullable: false + - column: + name: created_at + type: timestamp with time zone + constraints: + nullable: false + - column: + name: updated_at + type: timestamp with time zone + constraints: + nullable: false + - addUniqueConstraint: + tableName: attachments + columnNames: object_key + constraintName: attachments_object_key_uq + - addForeignKeyConstraint: + baseTableName: attachments + baseColumnNames: chat_id + referencedTableName: chats + referencedColumnNames: id + constraintName: attachments_chat_fk + onDelete: RESTRICT + - addForeignKeyConstraint: + baseTableName: attachments + baseColumnNames: owner_id + referencedTableName: users + referencedColumnNames: id + constraintName: attachments_owner_fk + onDelete: RESTRICT + - createIndex: + indexName: attachments_chat_idx + tableName: attachments + columns: + - column: + name: chat_id + - createTable: + tableName: message_attachments + columns: + - column: + name: message_id + type: uuid + constraints: + nullable: false + - column: + name: sort_order + type: int + constraints: + nullable: false + - column: + name: attachment_id + type: uuid + constraints: + nullable: false + - addPrimaryKey: + tableName: message_attachments + columnNames: message_id, sort_order + constraintName: message_attachments_pkey + - addForeignKeyConstraint: + baseTableName: message_attachments + baseColumnNames: message_id + referencedTableName: messages + referencedColumnNames: id + constraintName: message_attachments_message_fk + onDelete: RESTRICT + - addForeignKeyConstraint: + baseTableName: message_attachments + baseColumnNames: attachment_id + referencedTableName: attachments + referencedColumnNames: id + constraintName: message_attachments_attachment_fk + onDelete: RESTRICT diff --git a/backend/src/main/resources/db/changelog/007-jobs.yaml b/backend/src/main/resources/db/changelog/007-jobs.yaml new file mode 100644 index 00000000..e267dd68 --- /dev/null +++ b/backend/src/main/resources/db/changelog/007-jobs.yaml @@ -0,0 +1,221 @@ +databaseChangeLog: + - changeSet: + id: 007-jobs + author: lab5 + changes: + - modifyDataType: + tableName: attachments + columnName: object_key + newDataType: varchar(120) + - addColumn: + tableName: attachments + columns: + - column: + name: error_code + type: varchar(64) + - column: + name: pipeline_version + type: int + defaultValueNumeric: 1 + constraints: + nullable: false + - column: + name: attempt + type: int + defaultValueNumeric: 0 + constraints: + nullable: false + - column: + name: job_id + type: uuid + - column: + name: staging_key + type: varchar(120) + - column: + name: lease_until + type: timestamp with time zone + - column: + name: lease_token + type: uuid + - createIndex: + indexName: attachments_lease_idx + tableName: attachments + columns: + - column: + name: status + - column: + name: lease_until + - createTable: + tableName: attachment_variants + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: attachment_variants_pkey + - column: + name: attachment_id + type: uuid + constraints: + nullable: false + - column: + name: pipeline_version + type: int + constraints: + nullable: false + - column: + name: name + type: varchar(16) + constraints: + nullable: false + - column: + name: content_type + type: varchar(32) + constraints: + nullable: false + - column: + name: size_bytes + type: bigint + constraints: + nullable: false + - column: + name: width + type: int + constraints: + nullable: false + - column: + name: height + type: int + constraints: + nullable: false + - column: + name: sha256 + type: varchar(64) + constraints: + nullable: false + - column: + name: object_key + type: varchar(120) + constraints: + nullable: false + - addUniqueConstraint: + tableName: attachment_variants + columnNames: attachment_id, pipeline_version, name + constraintName: attachment_variants_identity_uq + - addUniqueConstraint: + tableName: attachment_variants + columnNames: object_key + constraintName: attachment_variants_object_key_uq + - addForeignKeyConstraint: + baseTableName: attachment_variants + baseColumnNames: attachment_id + referencedTableName: attachments + referencedColumnNames: id + constraintName: attachment_variants_attachment_fk + onDelete: RESTRICT + - createTable: + tableName: job_outbox + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: job_outbox_pkey + - column: + name: job_id + type: uuid + constraints: + nullable: false + - column: + name: attachment_id + type: uuid + constraints: + nullable: false + - column: + name: attempt + type: int + constraints: + nullable: false + - column: + name: payload + type: varchar(4000) + constraints: + nullable: false + - column: + name: available_at + type: timestamp with time zone + constraints: + nullable: false + - column: + name: published_at + type: timestamp with time zone + - column: + name: created_at + type: timestamp with time zone + constraints: + nullable: false + - createIndex: + indexName: job_outbox_due_idx + tableName: job_outbox + columns: + - column: + name: published_at + - column: + name: available_at + - createTable: + tableName: failed_jobs + columns: + - column: + name: id + type: uuid + constraints: + primaryKey: true + nullable: false + primaryKeyName: failed_jobs_pkey + - column: + name: job_id + type: uuid + constraints: + nullable: false + - column: + name: attachment_id + type: uuid + constraints: + nullable: false + - column: + name: error_code + type: varchar(64) + constraints: + nullable: false + - column: + name: attempt + type: int + constraints: + nullable: false + - column: + name: created_at + type: timestamp with time zone + constraints: + nullable: false + - column: + name: replayed_at + type: timestamp with time zone + - createTable: + tableName: demo_flags + columns: + - column: + name: name + type: varchar(64) + constraints: + primaryKey: true + nullable: false + primaryKeyName: demo_flags_pkey + - column: + name: enabled + type: boolean + constraints: + nullable: false diff --git a/backend/src/main/resources/db/changelog/db.changelog-master.yaml b/backend/src/main/resources/db/changelog/db.changelog-master.yaml new file mode 100644 index 00000000..5f95e3ff --- /dev/null +++ b/backend/src/main/resources/db/changelog/db.changelog-master.yaml @@ -0,0 +1,22 @@ +databaseChangeLog: + - include: + file: 001-users.yaml + relativeToChangelogFile: true + - include: + file: 002-sessions.yaml + relativeToChangelogFile: true + - include: + file: 003-chats.yaml + relativeToChangelogFile: true + - include: + file: 004-messages.yaml + relativeToChangelogFile: true + - include: + file: 005-idempotency.yaml + relativeToChangelogFile: true + - include: + file: 006-attachments.yaml + relativeToChangelogFile: true + - include: + file: 007-jobs.yaml + relativeToChangelogFile: true diff --git a/backend/src/test/java/ru/mtuci/messenger/ApiIntegrationTest.java b/backend/src/test/java/ru/mtuci/messenger/ApiIntegrationTest.java new file mode 100644 index 00000000..d2d57b24 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/ApiIntegrationTest.java @@ -0,0 +1,281 @@ +package ru.mtuci.messenger; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.data.redis.core.StringRedisTemplate; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import ru.mtuci.messenger.repository.ChatRepository; +import ru.mtuci.messenger.service.UserService; +import ru.mtuci.messenger.support.ApiClient; +import ru.mtuci.messenger.support.LabContainers; +import tools.jackson.databind.JsonNode; +import tools.jackson.databind.json.JsonMapper; + +import java.util.Base64; +import java.util.List; +import java.util.Map; +import java.util.UUID; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.Future; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; + +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) +class ApiIntegrationTest { + + @LocalServerPort + int port; + + @Autowired + JsonMapper jsonMapper; + + @Autowired + ChatRepository chatRepository; + + @Autowired + StringRedisTemplate redis; + + private ApiClient api; + + @DynamicPropertySource + static void containers(DynamicPropertyRegistry registry) { + LabContainers.register(registry); + } + + @BeforeEach + void client() { + api = new ApiClient(port, jsonMapper); + } + + @Test + void healthAndActuatorReportProcessAndDatabase() { + ApiClient.Result live = api.get("/health/live", null); + assertEquals(200, live.status()); + assertEquals("ok", live.body().get("status").asString()); + assertFalse(live.requestId().isBlank()); + ApiClient.Result ready = api.get("/health/ready", null); + assertEquals(200, ready.status()); + assertEquals("ok", ready.body().get("status").asString()); + ApiClient.Result actuator = api.get("/actuator/health/readiness", null); + assertEquals(200, actuator.status()); + assertEquals("UP", actuator.body().get("status").asString()); + } + + @Test + void registrationConflictValidationAndRace() throws Exception { + String username = name("reg"); + ApiClient.Result created = api.post("/api/v1/users", api.user(username, "Алиса", "Example-only-not-a-real-secret-123")); + assertEquals(201, created.status()); + assertEquals(username, created.body().get("username").asString()); + assertTrue(created.body().get("created_at").asString().endsWith("Z")); + assertFalse(created.raw().contains("password")); + assertEquals(created.requestId(), created.requestId()); + + ApiClient.Result conflict = api.post("/api/v1/users", api.user(username, "Алиса", "Example-only-not-a-real-secret-123")); + assertEquals(409, conflict.status()); + assertEquals("conflict", conflict.body().get("error").get("code").asString()); + assertEquals(conflict.requestId(), conflict.body().get("error").get("request_id").asString()); + + ApiClient.Result blank = api.post("/api/v1/users", api.user(name("blank"), " ", "Example-only-not-a-real-secret-123")); + assertEquals(400, blank.status()); + ApiClient.Result extra = api.post("/api/v1/users", Map.of("username", name("extra"), "display_name", "Имя", "password", "Example-only-not-a-real-secret-123", "role", "admin")); + assertEquals(400, extra.status()); + + String raced = name("race"); + CountDownLatch latch = new CountDownLatch(1); + try (ExecutorService pool = Executors.newFixedThreadPool(2)) { + List> results = List.of( + pool.submit(() -> once(latch, raced)), + pool.submit(() -> once(latch, raced)) + ); + latch.countDown(); + int createdCount = 0; + int conflictCount = 0; + for (Future result : results) { + int status = result.get(); + if (status == 201) { + createdCount++; + } else if (status == 409) { + conflictCount++; + } + } + assertEquals(1, createdCount); + assertEquals(1, conflictCount); + } + } + + @Test + void sessionLoginLogoutAndIgnoredUserHeader() { + String username = name("sess"); + String password = "Example-only-not-a-real-secret-123"; + JsonNode user = api.post("/api/v1/users", api.user(username, "Алиса", password)).body(); + JsonNode other = api.post("/api/v1/users", api.user(name("oth"), "Боб", password)).body(); + String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes()); + ApiClient.Result bad = api.postBasic("/api/v1/auth/sessions", Base64.getEncoder().encodeToString((username + ":wrong-password-xx").getBytes())); + assertEquals(401, bad.status()); + assertTrue(bad.authenticate().toLowerCase().startsWith("basic")); + + ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic); + assertEquals(201, session.status()); + assertEquals("Bearer", session.body().get("token_type").asString()); + assertTrue(session.body().get("token").asString().length() >= 32); + assertNotEquals(user.get("id").asString(), session.body().get("token").asString()); + assertEquals(user.get("id").asString(), session.body().get("user").get("id").asString()); + String token = session.body().get("token").asString(); + + ApiClient.Result basicOnApi = api.postBasic("/api/v1/users/me", basic); + assertEquals(401, basicOnApi.status()); + assertTrue(basicOnApi.authenticate().toLowerCase().startsWith("bearer")); + ApiClient.Result headerOnly = api.get("/api/v1/users/me", null, Map.of("X-User-Id", user.get("id").asString())); + assertEquals(401, headerOnly.status()); + ApiClient.Result me = api.get("/api/v1/users/me", token, Map.of("X-User-Id", other.get("id").asString())); + assertEquals(200, me.status()); + assertEquals(user.get("id").asString(), me.body().get("id").asString()); + + assertTrue(Boolean.TRUE.equals(redis.hasKey("session:" + UserService.tokenHash(token)))); + ApiClient.Result logout = api.delete("/api/v1/auth/sessions/current", token); + assertEquals(204, logout.status()); + assertTrue(logout.raw().isBlank()); + assertFalse(Boolean.TRUE.equals(redis.hasKey("session:" + UserService.tokenHash(token)))); + assertEquals(401, api.get("/api/v1/users/me", token).status()); + assertEquals(401, api.delete("/api/v1/auth/sessions/current", token).status()); + } + + @Test + void chatsAreAtomicAndHiddenFromOutsiders() { + String password = "Example-only-not-a-real-secret-123"; + JsonNode alice = register(name("alice"), "Алиса", password); + JsonNode bob = register(name("bob"), "Боб", password); + String aliceToken = login(alice.get("username").asString(), password); + String bobToken = login(bob.get("username").asString(), password); + String eveToken = login(register(name("eve"), "Ева", password).get("username").asString(), password); + + String title = "rollback-" + UUID.randomUUID(); + ApiClient.Result unknown = api.post("/api/v1/chats", aliceToken, Map.of( + "title", title, + "member_ids", List.of(alice.get("id").asString(), UUID.randomUUID().toString()) + )); + assertEquals(400, unknown.status()); + assertEquals(0, chatRepository.countByTitle(title)); + + ApiClient.Result duplicates = api.post("/api/v1/chats", aliceToken, Map.of( + "title", "dup-" + UUID.randomUUID(), + "member_ids", List.of(alice.get("id").asString(), alice.get("id").asString()) + )); + assertEquals(400, duplicates.status()); + ApiClient.Result withoutSelf = api.post("/api/v1/chats", aliceToken, Map.of( + "title", "self-" + UUID.randomUUID(), + "member_ids", List.of(bob.get("id").asString()) + )); + assertEquals(400, withoutSelf.status()); + + ApiClient.Result created = api.post("/api/v1/chats", aliceToken, Map.of( + "title", "Контрактный тест", + "member_ids", List.of(alice.get("id").asString(), bob.get("id").asString()) + )); + assertEquals(201, created.status()); + String chatId = created.body().get("id").asString(); + assertEquals(404, api.get("/api/v1/chats/" + chatId, eveToken).status()); + JsonNode bobList = api.get("/api/v1/chats?limit=100", bobToken).body(); + assertTrue(containsId(bobList.get("items"), chatId)); + JsonNode eveList = api.get("/api/v1/chats?limit=100", eveToken).body(); + assertFalse(containsId(eveList.get("items"), chatId)); + + ApiClient.Result second = api.post("/api/v1/chats", aliceToken, Map.of( + "title", "Второй", + "member_ids", List.of(alice.get("id").asString()) + )); + assertEquals(201, second.status()); + JsonNode page = api.get("/api/v1/chats?limit=1", aliceToken).body(); + assertEquals(1, page.get("items").size()); + assertFalse(page.get("next_cursor").isNull()); + JsonNode next = api.get("/api/v1/chats?limit=1&cursor=" + page.get("next_cursor").asString(), aliceToken).body(); + assertEquals(1, next.get("items").size()); + assertNotEquals(page.get("items").get(0).get("id").asString(), next.get("items").get(0).get("id").asString()); + } + + @Test + void messageHistoryUsesKeysetCursor() { + String password = "Example-only-not-a-real-secret-123"; + JsonNode alice = register(name("ma"), "Алиса", password); + JsonNode bob = register(name("mb"), "Боб", password); + String aliceToken = login(alice.get("username").asString(), password); + String bobToken = login(bob.get("username").asString(), password); + String eveToken = login(register(name("me"), "Ева", password).get("username").asString(), password); + String chatId = api.post("/api/v1/chats", aliceToken, Map.of( + "title", "История", + "member_ids", List.of(alice.get("id").asString(), bob.get("id").asString()) + )).body().get("id").asString(); + String otherChat = api.post("/api/v1/chats", aliceToken, Map.of( + "title", "Другой", + "member_ids", List.of(alice.get("id").asString()) + )).body().get("id").asString(); + + ApiClient.Result first = sendMessage(aliceToken, chatId, "Привет, мир"); + ApiClient.Result second = sendMessage(bobToken, chatId, "Сообщение Боба"); + assertEquals(201, first.status()); + assertTrue(first.body().get("created_at").asString().matches("\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{6}Z")); + assertFalse(first.instanceId().isBlank()); + assertEquals(first.instanceId(), api.get("/api/v1/users/me", aliceToken).instanceId()); + assertEquals(alice.get("id").asString(), first.body().get("sender_id").asString()); + assertEquals(201, second.status()); + assertEquals(400, sendMessage(aliceToken, chatId, "").status()); + assertEquals(400, sendMessage(aliceToken, chatId, " ").status()); + assertEquals(404, sendMessage(eveToken, chatId, "чужое").status()); + assertEquals(400, api.get("/api/v1/chats/" + chatId + "/messages?limit=0", aliceToken).status()); + + JsonNode page = api.get("/api/v1/chats/" + chatId + "/messages?limit=1", bobToken).body(); + assertEquals(1, page.get("items").size()); + assertEquals(first.body().get("id").asString(), page.get("items").get(0).get("id").asString()); + String cursor = page.get("next_cursor").asString(); + JsonNode rest = api.get("/api/v1/chats/" + chatId + "/messages?limit=1&cursor=" + cursor, aliceToken).body(); + assertEquals(second.body().get("id").asString(), rest.get("items").get(0).get("id").asString()); + assertTrue(rest.get("next_cursor").isNull()); + assertEquals(400, api.get("/api/v1/chats/" + otherChat + "/messages?cursor=" + cursor, aliceToken).status()); + } + + private int once(CountDownLatch latch, String username) throws Exception { + latch.await(); + return api.post("/api/v1/users", api.user(username, "Гонка", "Example-only-not-a-real-secret-123")).status(); + } + + private JsonNode register(String username, String displayName, String password) { + ApiClient.Result result = api.post("/api/v1/users", api.user(username, displayName, password)); + assertEquals(201, result.status()); + return result.body(); + } + + private String login(String username, String password) { + String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes()); + ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic); + assertEquals(201, session.status()); + return session.body().get("token").asString(); + } + + private static boolean containsId(JsonNode items, String id) { + for (JsonNode item : items) { + if (id.equals(item.get("id").asString())) { + return true; + } + } + return false; + } + + private ApiClient.Result sendMessage(String token, String chatId, String text) { + return api.post("/api/v1/chats/" + chatId + "/messages", token, Map.of("text", text), + Map.of("Idempotency-Key", UUID.randomUUID().toString())); + } + + private static String name(String prefix) { + return (prefix + UUID.randomUUID().toString().replace("-", "")).substring(0, 20); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/AttachmentIntegrationTest.java b/backend/src/test/java/ru/mtuci/messenger/AttachmentIntegrationTest.java new file mode 100644 index 00000000..5e6d1115 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/AttachmentIntegrationTest.java @@ -0,0 +1,240 @@ +package ru.mtuci.messenger; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import ru.mtuci.messenger.storage.ObjectStorage; +import ru.mtuci.messenger.storage.OrphanSweeper; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import ru.mtuci.messenger.support.ApiClient; +import ru.mtuci.messenger.support.LabContainers; +import tools.jackson.databind.JsonNode; +import tools.jackson.databind.json.JsonMapper; + +import java.net.URI; +import java.net.URLEncoder; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.security.MessageDigest; +import java.util.Base64; +import java.util.HexFormat; +import java.util.List; +import java.util.Map; +import java.time.Instant; +import java.util.UUID; + +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) +class AttachmentIntegrationTest { + + @LocalServerPort + int port; + + @Autowired + JsonMapper jsonMapper; + + @Autowired + ObjectStorage storage; + + @Autowired + OrphanSweeper sweeper; + + private ApiClient api; + private final HttpClient http = HttpClient.newHttpClient(); + + @DynamicPropertySource + static void containers(DynamicPropertyRegistry registry) { + LabContainers.register(registry); + } + + @BeforeEach + void client() { + api = new ApiClient(port, jsonMapper); + } + + @Test + void uploadIsPrivateAndReadableByAnotherMember() throws Exception { + String password = "Example-only-not-a-real-secret-123"; + JsonNode alice = register("at", password); + JsonNode bob = register("bt", password); + JsonNode eve = register("et", password); + String aliceToken = login(alice.get("username").asString(), password); + String bobToken = login(bob.get("username").asString(), password); + String eveToken = login(eve.get("username").asString(), password); + String chatId = chat(aliceToken, List.of(alice.get("id").asString(), bob.get("id").asString())); + String otherChat = chat(aliceToken, List.of(alice.get("id").asString())); + byte[] png = Files.readAllBytes(Path.of("../tests/fixtures/sample.png")); + + HttpResponse created = upload(aliceToken, chatId, "sample.png", "image/png", png); + assertEquals(202, created.statusCode()); + JsonNode attachment = jsonMapper.readTree(created.body()); + assertEquals("queued", attachment.get("status").asString()); + assertEquals(sha256(png), attachment.get("sha256").asString()); + assertEquals(png.length, attachment.get("size_bytes").asInt()); + assertEquals(false, attachment.has("object_key")); + String attachmentId = attachment.get("id").asString(); + + ApiClient.Result message = api.post("/api/v1/chats/" + chatId + "/messages", aliceToken, + Map.of("text", "", "attachment_ids", List.of(attachmentId)), + Map.of("Idempotency-Key", UUID.randomUUID().toString())); + assertEquals(201, message.status()); + assertEquals(attachmentId, message.body().get("attachment_ids").get(0).asString()); + + JsonNode ready = waitReady(bobToken, attachmentId); + JsonNode thumbnail = variant(ready, "thumbnail"); + assertEquals(256, thumbnail.get("width").asInt()); + assertEquals(160, thumbnail.get("height").asInt()); + assertEquals("image/jpeg", thumbnail.get("content_type").asString()); + JsonNode crop = variant(ready, "crop"); + assertEquals(128, crop.get("width").asInt()); + assertEquals(128, crop.get("height").asInt()); + JsonNode optimized = variant(ready, "optimized"); + assertEquals("image/webp", optimized.get("content_type").asString()); + assertEquals(320, optimized.get("width").asInt()); + assertEquals(200, optimized.get("height").asInt()); + + HttpResponse download = download(bobToken, attachmentId); + assertEquals(200, download.statusCode()); + assertEquals("image/png", download.headers().firstValue("Content-Type").orElse("")); + assertArrayEquals(png, download.body()); + assertEquals(404, api.get("/api/v1/attachments/" + attachmentId, eveToken).status()); + assertEquals(404, download(eveToken, attachmentId).statusCode()); + + HttpResponse badName = upload(aliceToken, chatId, "../secret.png", "image/png", png); + assertEquals(400, badName.statusCode()); + HttpResponse spoofed = upload(aliceToken, chatId, "photo.png", "image/png", new byte[] {1, 2, 3, 4}); + assertEquals(415, spoofed.statusCode()); + ApiClient.Result foreign = api.post("/api/v1/chats/" + otherChat + "/messages", aliceToken, + Map.of("text", "", "attachment_ids", List.of(attachmentId)), + Map.of("Idempotency-Key", UUID.randomUUID().toString())); + assertEquals(404, foreign.status()); + assertEquals("not_found", foreign.body().get("error").get("code").asString()); + + ApiClient.Result missing = api.post("/api/v1/chats/" + chatId + "/messages", aliceToken, + Map.of("text", "", "attachment_ids", List.of(UUID.randomUUID().toString())), + Map.of("Idempotency-Key", UUID.randomUUID().toString())); + assertEquals(404, missing.status()); + + HttpResponse corrupt = upload(aliceToken, chatId, "broken.png", "image/png", + new byte[] {(byte) 137, 80, 78, 71, 13, 10, 26, 10, 0, 1, 2}); + assertEquals(415, corrupt.statusCode()); + assertEquals("unsupported_media_type", jsonMapper.readTree(corrupt.body()).get("error").get("code").asString()); + + byte[] jpeg = new byte[] { + (byte) 0xff, (byte) 0xd8, + (byte) 0xff, (byte) 0xc0, 0x00, 0x0b, 0x08, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x11, 0x00, + (byte) 0xff, (byte) 0xd9 + }; + HttpResponse mime = upload(aliceToken, chatId, "photo.png", "image/png", jpeg); + assertEquals(415, mime.statusCode()); + + HttpResponse tooBig = upload(aliceToken, chatId, "big.png", "image/png", new byte[10_485_761]); + assertEquals(413, tooBig.statusCode()); + assertEquals("payload_too_large", jsonMapper.readTree(tooBig.body()).get("error").get("code").asString()); + assertTrue(tooBig.statusCode() < 500); + } + + @Test + void sweepDropsOrphanAndKeepsReferencedObject() throws Exception { + String password = "Example-only-not-a-real-secret-123"; + JsonNode alice = register("sw", password); + String token = login(alice.get("username").asString(), password); + String chatId = chat(token, List.of(alice.get("id").asString())); + byte[] png = Files.readAllBytes(Path.of("../tests/fixtures/sample.png")); + HttpResponse created = upload(token, chatId, "sample.png", "image/png", png); + assertEquals(202, created.statusCode()); + String id = jsonMapper.readTree(created.body()).get("id").asString(); + waitReady(token, id); + + storage.put("originals/orphan-" + UUID.randomUUID(), new byte[] {9, 9, 9}, "application/octet-stream"); + int removed = sweeper.sweepOlderThan(Instant.now().plusSeconds(120)); + assertTrue(removed >= 1); + assertEquals(200, download(token, id).statusCode()); + assertTrue(storage.list("originals/").stream().noneMatch(object -> object.key().contains("orphan-"))); + } + + private JsonNode waitReady(String token, String id) throws Exception { + long deadline = System.nanoTime() + java.time.Duration.ofSeconds(40).toNanos(); + JsonNode last = null; + while (System.nanoTime() < deadline) { + ApiClient.Result result = api.get("/api/v1/attachments/" + id, token); + assertEquals(200, result.status()); + last = result.body(); + String status = last.get("status").asString(); + if ("ready".equals(status)) { + return last; + } + if ("failed".equals(status)) { + fail("attachment failed: " + last.get("error_code")); + } + Thread.sleep(200); + } + fail("attachment did not become ready: " + last); + return last; + } + + private static JsonNode variant(JsonNode attachment, String name) { + for (JsonNode variant : attachment.get("variants")) { + if (name.equals(variant.get("name").asString())) { + return variant; + } + } + fail("missing variant " + name); + return null; + } + + private HttpResponse upload(String token, String chatId, String filename, String type, byte[] body) throws Exception { + String name = URLEncoder.encode(filename, StandardCharsets.UTF_8); + HttpRequest request = HttpRequest.newBuilder(URI.create(base() + "/api/v1/chats/" + chatId + "/attachments?filename=" + name)) + .header("Authorization", "Bearer " + token) + .header("Content-Type", type) + .POST(HttpRequest.BodyPublishers.ofByteArray(body)) + .build(); + return http.send(request, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); + } + + private HttpResponse download(String token, String attachmentId) throws Exception { + HttpRequest request = HttpRequest.newBuilder(URI.create(base() + "/api/v1/attachments/" + attachmentId + "/content")) + .header("Authorization", "Bearer " + token) + .GET() + .build(); + return http.send(request, HttpResponse.BodyHandlers.ofByteArray()); + } + + private String base() { + return "http://127.0.0.1:" + port; + } + + private String chat(String token, List members) { + return api.post("/api/v1/chats", token, Map.of("title", "Вложения", "member_ids", members)).body().get("id").asString(); + } + + private JsonNode register(String prefix, String password) { + String username = (prefix + UUID.randomUUID().toString().replace("-", "")).substring(0, 20); + ApiClient.Result result = api.post("/api/v1/users", api.user(username, "Алиса", password)); + assertEquals(201, result.status()); + return result.body(); + } + + private String login(String username, String password) { + String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes(StandardCharsets.UTF_8)); + ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic); + assertEquals(201, session.status()); + return session.body().get("token").asString(); + } + + private static String sha256(byte[] bytes) throws Exception { + return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(bytes)); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/DatabaseOutageIntegrationTest.java b/backend/src/test/java/ru/mtuci/messenger/DatabaseOutageIntegrationTest.java new file mode 100644 index 00000000..32336b63 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/DatabaseOutageIntegrationTest.java @@ -0,0 +1,75 @@ +package ru.mtuci.messenger; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.Timeout; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.testcontainers.postgresql.PostgreSQLContainer; +import ru.mtuci.messenger.support.ApiClient; +import ru.mtuci.messenger.support.LabContainers; +import tools.jackson.databind.json.JsonMapper; + +import java.util.UUID; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = { + "spring.data.redis.host=127.0.0.1", + "spring.data.redis.port=1", + "spring.data.redis.timeout=200ms", + "spring.data.redis.connect-timeout=200ms", + "spring.datasource.hikari.connection-timeout=1000", + "spring.datasource.hikari.validation-timeout=500", + "spring.datasource.hikari.maximum-pool-size=2" +}) +@DirtiesContext +class DatabaseOutageIntegrationTest { + + private static final PostgreSQLContainer POSTGRES = new PostgreSQLContainer("postgres:16.10"); + + static { + POSTGRES.start(); + } + + @LocalServerPort + int port; + + @Autowired + JsonMapper jsonMapper; + + private ApiClient api; + + @DynamicPropertySource + static void database(DynamicPropertyRegistry registry) { + registry.add("spring.datasource.url", POSTGRES::getJdbcUrl); + registry.add("spring.datasource.username", POSTGRES::getUsername); + registry.add("spring.datasource.password", POSTGRES::getPassword); + LabContainers.registerS3(registry); + } + + @BeforeEach + void client() { + api = new ApiClient(port, jsonMapper); + } + + @Test + @Timeout(40) + void readinessAndWritesFailWhenDatabaseStops() { + assertEquals(200, api.get("/health/ready", null).status()); + POSTGRES.stop(); + ApiClient.Result ready = api.get("/health/ready", null); + assertEquals(503, ready.status()); + assertEquals("unavailable", ready.body().get("error").get("code").asString()); + ApiClient.Result register = api.post("/api/v1/users", api.user( + ("db" + UUID.randomUUID().toString().replace("-", "")).substring(0, 16), + "База", + "Example-only-not-a-real-secret-123" + )); + assertEquals(503, register.status()); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/HistoryIndexIntegrationTest.java b/backend/src/test/java/ru/mtuci/messenger/HistoryIndexIntegrationTest.java new file mode 100644 index 00000000..d4f3227c --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/HistoryIndexIntegrationTest.java @@ -0,0 +1,65 @@ +package ru.mtuci.messenger; + +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import ru.mtuci.messenger.support.LabContainers; + +import java.nio.file.Files; +import java.nio.file.Path; +import java.sql.Timestamp; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import java.util.UUID; + +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +@SpringBootTest +class HistoryIndexIntegrationTest { + + @Autowired + JdbcTemplate jdbc; + + @DynamicPropertySource + static void containers(DynamicPropertyRegistry registry) { + LabContainers.register(registry); + } + + @Test + void historyPageUsesCompositeIndex() throws Exception { + UUID userId = UUID.randomUUID(); + UUID chatId = UUID.randomUUID(); + Instant created = Instant.parse("2026-01-01T00:00:00Z"); + jdbc.update("insert into users (id, username, display_name, password_hash, created_at) values (?, ?, ?, ?, ?)", + userId, ("idx" + userId.toString().replace("-", "")).substring(0, 16), "Индекс", "hash", Timestamp.from(created)); + jdbc.update("insert into chats (id, title, created_at) values (?, ?, ?)", chatId, "Индекс", Timestamp.from(created)); + jdbc.update("insert into chat_members (chat_id, user_id, ordinal) values (?, ?, 0)", chatId, userId); + + List batch = new ArrayList<>(); + for (int index = 0; index < 10_000; index++) { + batch.add(new Object[]{UUID.randomUUID(), chatId, userId, "m" + index, Timestamp.from(created.plusMillis(index))}); + } + jdbc.batchUpdate("insert into messages (id, chat_id, sender_id, text, created_at) values (?, ?, ?, ?, ?)", batch); + jdbc.execute("analyze messages"); + String plan = String.join("\n", jdbc.query(""" + explain (analyze, buffers) + select id from messages + where chat_id = ? + and (created_at, id) > (?, ?) + order by created_at asc, id asc + limit 50 + """, (rs, row) -> rs.getString(1), chatId, Timestamp.from(created.plusMillis(5000)), UUID.randomUUID())); + Path evidence = Files.isDirectory(Path.of("evidence")) ? Path.of("evidence") : Path.of("..", "evidence"); + Files.createDirectories(evidence); + Files.writeString(evidence.resolve("explain-messages.txt"), plan + "\n"); + assertTrue(plan.contains("messages_history_idx"), plan); + assertTrue(plan.contains("Index Cond"), plan); + assertFalse(plan.contains("Seq Scan on messages"), plan); + assertFalse(plan.contains("Rows Removed by Filter"), plan); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/IdempotencyIntegrationTest.java b/backend/src/test/java/ru/mtuci/messenger/IdempotencyIntegrationTest.java new file mode 100644 index 00000000..acec8248 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/IdempotencyIntegrationTest.java @@ -0,0 +1,164 @@ +package ru.mtuci.messenger; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import ru.mtuci.messenger.repository.MessageRepository; +import ru.mtuci.messenger.support.ApiClient; +import ru.mtuci.messenger.support.LabContainers; +import tools.jackson.databind.JsonNode; +import tools.jackson.databind.json.JsonMapper; + +import java.util.ArrayList; +import java.util.Base64; +import java.util.List; +import java.util.Map; +import java.util.UUID; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.Future; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotEquals; + +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) +class IdempotencyIntegrationTest { + + @LocalServerPort + int port; + + @Autowired + JsonMapper jsonMapper; + + @Autowired + MessageRepository messages; + + private ApiClient api; + + @DynamicPropertySource + static void containers(DynamicPropertyRegistry registry) { + LabContainers.register(registry); + } + + @BeforeEach + void client() { + api = new ApiClient(port, jsonMapper); + } + + @Test + void replayConflictAndRejectedKey() { + String password = "Example-only-not-a-real-secret-123"; + JsonNode alice = register(name("ia"), password); + String token = login(alice.get("username").asString(), password); + String chatId = chat(token, List.of(alice.get("id").asString())); + String key = "same-" + UUID.randomUUID(); + + ApiClient.Result rejected = send(token, chatId, "ещё не записано", key.replace("same", "bad key")); + assertEquals(400, rejected.status()); + assertFalse(rejected.instanceId().isBlank()); + assertEquals(400, api.post("/api/v1/chats/" + chatId + "/messages", token, Map.of("text", "без ключа")).status()); + + String recover = "recover-" + UUID.randomUUID(); + assertEquals(400, send(token, chatId, "", recover).status()); + ApiClient.Result first = send(token, chatId, "Привет, мир", recover); + assertEquals(201, first.status()); + ApiClient.Result replay = send(token, chatId, "Привет, мир", recover); + assertEquals(201, replay.status()); + assertEquals(first.body(), replay.body()); + assertEquals(409, send(token, chatId, "другой body", recover).status()); + assertEquals(1, messages.countByChatId(UUID.fromString(chatId))); + + ApiClient.Result fresh = send(token, chatId, "Привет, мир", key); + assertEquals(201, fresh.status()); + assertNotEquals(first.body().get("id").asString(), fresh.body().get("id").asString()); + } + + @Test + void sameKeyIsIndependentAcrossUsersAndChats() { + String password = "Example-only-not-a-real-secret-123"; + JsonNode alice = register(name("ib"), password); + JsonNode bob = register(name("ic"), password); + String aliceToken = login(alice.get("username").asString(), password); + String bobToken = login(bob.get("username").asString(), password); + String shared = chat(aliceToken, List.of(alice.get("id").asString(), bob.get("id").asString())); + String other = chat(aliceToken, List.of(alice.get("id").asString())); + String key = "shared-" + UUID.randomUUID(); + + ApiClient.Result aliceMessage = send(aliceToken, shared, "общий ключ", key); + ApiClient.Result bobMessage = send(bobToken, shared, "общий ключ", key); + ApiClient.Result otherChat = send(aliceToken, other, "общий ключ", key); + assertEquals(201, aliceMessage.status()); + assertEquals(201, bobMessage.status()); + assertEquals(201, otherChat.status()); + assertNotEquals(aliceMessage.body().get("id").asString(), bobMessage.body().get("id").asString()); + assertNotEquals(aliceMessage.body().get("id").asString(), otherChat.body().get("id").asString()); + } + + @Test + void parallelSameKeyCreatesOneMessage() throws Exception { + String password = "Example-only-not-a-real-secret-123"; + JsonNode alice = register(name("ip"), password); + String token = login(alice.get("username").asString(), password); + String chatId = chat(token, List.of(alice.get("id").asString())); + String key = "parallel-" + UUID.randomUUID(); + int threads = 8; + ExecutorService pool = Executors.newFixedThreadPool(threads); + CountDownLatch start = new CountDownLatch(1); + try { + List> futures = new ArrayList<>(); + for (int i = 0; i < threads; i++) { + futures.add(pool.submit(() -> { + start.await(); + return send(token, chatId, "одно сообщение", key); + })); + } + start.countDown(); + JsonNode body = null; + for (Future future : futures) { + ApiClient.Result result = future.get(); + assertEquals(201, result.status()); + if (body == null) { + body = result.body(); + } else { + assertEquals(body, result.body()); + } + } + } finally { + pool.shutdownNow(); + } + assertEquals(1, messages.countByChatId(UUID.fromString(chatId))); + } + + private ApiClient.Result send(String token, String chatId, String text, String key) { + return api.post("/api/v1/chats/" + chatId + "/messages", token, Map.of("text", text), + Map.of("Idempotency-Key", key)); + } + + private String chat(String token, List members) { + return api.post("/api/v1/chats", token, Map.of("title", "Идемпотентность", "member_ids", members)) + .body().get("id").asString(); + } + + private JsonNode register(String username, String password) { + ApiClient.Result result = api.post("/api/v1/users", api.user(username, "Алиса", password)); + assertEquals(201, result.status()); + return result.body(); + } + + private String login(String username, String password) { + String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes()); + ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic); + assertEquals(201, session.status()); + return session.body().get("token").asString(); + } + + private static String name(String prefix) { + return (prefix + UUID.randomUUID().toString().replace("-", "")).substring(0, 20); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/RedisOutageIntegrationTest.java b/backend/src/test/java/ru/mtuci/messenger/RedisOutageIntegrationTest.java new file mode 100644 index 00000000..39bd1efb --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/RedisOutageIntegrationTest.java @@ -0,0 +1,62 @@ +package ru.mtuci.messenger; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import ru.mtuci.messenger.support.ApiClient; +import ru.mtuci.messenger.support.LabContainers; +import tools.jackson.databind.json.JsonMapper; + +import java.util.Base64; +import java.util.UUID; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = { + "spring.data.redis.host=127.0.0.1", + "spring.data.redis.port=1", + "spring.data.redis.timeout=200ms", + "spring.data.redis.connect-timeout=200ms" +}) +class RedisOutageIntegrationTest { + + @LocalServerPort + int port; + + @Autowired + JsonMapper jsonMapper; + + private ApiClient api; + + @DynamicPropertySource + static void database(DynamicPropertyRegistry registry) { + registry.add("spring.datasource.url", LabContainers.POSTGRES::getJdbcUrl); + registry.add("spring.datasource.username", LabContainers.POSTGRES::getUsername); + registry.add("spring.datasource.password", LabContainers.POSTGRES::getPassword); + LabContainers.registerS3(registry); + } + + @BeforeEach + void client() { + api = new ApiClient(port, jsonMapper); + } + + @Test + void databaseSessionWorksWhenRedisIsDown() { + assertEquals(200, api.get("/health/ready", null).status()); + String username = ("red" + UUID.randomUUID().toString().replace("-", "")).substring(0, 16); + String password = "Example-only-not-a-real-secret-123"; + assertEquals(201, api.post("/api/v1/users", api.user(username, "Кэш", password)).status()); + String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes()); + ApiClient.Result session = api.postBasic("/api/v1/auth/sessions", basic); + assertEquals(201, session.status()); + ApiClient.Result me = api.get("/api/v1/users/me", session.body().get("token").asString()); + assertEquals(200, me.status()); + assertEquals(username, me.body().get("username").asString()); + assertEquals(200, api.get("/actuator/health", null).status()); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/SessionExpiryIntegrationTest.java b/backend/src/test/java/ru/mtuci/messenger/SessionExpiryIntegrationTest.java new file mode 100644 index 00000000..4cff296b --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/SessionExpiryIntegrationTest.java @@ -0,0 +1,53 @@ +package ru.mtuci.messenger; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import ru.mtuci.messenger.support.ApiClient; +import ru.mtuci.messenger.support.LabContainers; +import tools.jackson.databind.json.JsonMapper; + +import java.util.Base64; +import java.util.UUID; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = "app.session-ttl=2s") +class SessionExpiryIntegrationTest { + + @LocalServerPort + int port; + + @Autowired + JsonMapper jsonMapper; + + private ApiClient api; + + @DynamicPropertySource + static void containers(DynamicPropertyRegistry registry) { + LabContainers.register(registry); + } + + @BeforeEach + void client() { + api = new ApiClient(port, jsonMapper); + } + + @Test + void expiredTokenIsRejected() throws Exception { + String username = ("exp" + UUID.randomUUID().toString().replace("-", "")).substring(0, 16); + String password = "Example-only-not-a-real-secret-123"; + assertEquals(201, api.post("/api/v1/users", api.user(username, "Срок", password)).status()); + String basic = Base64.getEncoder().encodeToString((username + ":" + password).getBytes()); + String token = api.postBasic("/api/v1/auth/sessions", basic).body().get("token").asString(); + assertEquals(200, api.get("/api/v1/users/me", token).status()); + Thread.sleep(2500); + ApiClient.Result expired = api.get("/api/v1/users/me", token); + assertEquals(401, expired.status()); + assertEquals("unauthorized", expired.body().get("error").get("code").asString()); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/service/ImageFilesTest.java b/backend/src/test/java/ru/mtuci/messenger/service/ImageFilesTest.java new file mode 100644 index 00000000..2acbee9b --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/service/ImageFilesTest.java @@ -0,0 +1,68 @@ +package ru.mtuci.messenger.service; + +import org.junit.jupiter.api.Test; +import ru.mtuci.messenger.exception.PayloadTooLargeException; +import ru.mtuci.messenger.exception.UnsupportedMediaException; + +import java.nio.file.Files; +import java.nio.file.Path; + +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class ImageFilesTest { + + @Test + void readsFixtureWithoutDecodingPixels() throws Exception { + byte[] png = Files.readAllBytes(Path.of("../tests/fixtures/sample.png")); + ImageFiles.Image image = ImageFiles.inspect(png, "image/png", 40_000_000); + assertEquals("image/png", image.contentType()); + assertEquals(320, image.width()); + assertEquals(200, image.height()); + assertArrayEquals(png, ImageFiles.readLimited(new java.io.ByteArrayInputStream(png), 10_485_760)); + } + + @Test + void rejectsSpoofedMimeCorruptAndPixelBomb() { + byte[] jpeg = new byte[] { + (byte) 0xff, (byte) 0xd8, + (byte) 0xff, (byte) 0xc0, 0x00, 0x0b, 0x08, 0x00, 0x01, 0x00, 0x01, 0x01, 0x01, 0x11, 0x00, + (byte) 0xff, (byte) 0xd9 + }; + assertEquals(1, ImageFiles.inspect(jpeg, "image/jpeg", 40_000_000).width()); + assertThrows(UnsupportedMediaException.class, () -> ImageFiles.inspect(jpeg, "image/png", 40_000_000)); + assertThrows(UnsupportedMediaException.class, () -> ImageFiles.inspect(new byte[] {1, 2, 3}, "image/png", 40_000_000)); + byte[] bomb = png(8_000, 8_000); + assertThrows(PayloadTooLargeException.class, () -> ImageFiles.inspect(bomb, "image/png", 40_000_000)); + byte[] huge = new byte[8]; + assertThrows(PayloadTooLargeException.class, () -> ImageFiles.readLimited(new java.io.ByteArrayInputStream(huge), 7)); + } + + private static byte[] png(int width, int height) { + byte[] data = new byte[8 + 12 + 13 + 12]; + byte[] signature = new byte[] {(byte) 137, 80, 78, 71, 13, 10, 26, 10}; + System.arraycopy(signature, 0, data, 0, 8); + data[11] = 13; + data[12] = 'I'; + data[13] = 'H'; + data[14] = 'D'; + data[15] = 'R'; + putInt(data, 16, width); + putInt(data, 20, height); + data[24] = 8; + data[25] = 2; + data[37] = 'I'; + data[38] = 'E'; + data[39] = 'N'; + data[40] = 'D'; + return data; + } + + private static void putInt(byte[] data, int offset, int value) { + data[offset] = (byte) (value >>> 24); + data[offset + 1] = (byte) (value >>> 16); + data[offset + 2] = (byte) (value >>> 8); + data[offset + 3] = (byte) value; + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/service/ImageVariantsTest.java b/backend/src/test/java/ru/mtuci/messenger/service/ImageVariantsTest.java new file mode 100644 index 00000000..92397b97 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/service/ImageVariantsTest.java @@ -0,0 +1,75 @@ +package ru.mtuci.messenger.service; + +import org.junit.jupiter.api.Test; + +import javax.imageio.ImageIO; +import java.awt.Color; +import java.awt.image.BufferedImage; +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.util.Map; +import java.util.function.Function; +import java.util.stream.Collectors; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; + +class ImageVariantsTest { + + @Test + void fixtureSizeBecomesContractDerivatives() throws Exception { + byte[] png = png(320, 200); + Map variants = ImageVariants.render(png, 0.85f, 0.8f).stream() + .collect(Collectors.toMap(ImageVariants.VariantBytes::name, Function.identity())); + ImageVariants.VariantBytes thumbnail = variants.get("thumbnail"); + assertEquals("image/jpeg", thumbnail.contentType()); + assertEquals(256, thumbnail.width()); + assertEquals(160, thumbnail.height()); + assertTrue(thumbnail.bytes()[0] == (byte) 0xff && thumbnail.bytes()[1] == (byte) 0xd8); + BufferedImage decoded = ImageIO.read(new ByteArrayInputStream(thumbnail.bytes())); + assertEquals(256, decoded.getWidth()); + assertEquals(160, decoded.getHeight()); + + assertEquals(128, variants.get("crop").width()); + assertEquals(128, variants.get("crop").height()); + + ImageVariants.VariantBytes optimized = variants.get("optimized"); + assertEquals("image/webp", optimized.contentType()); + assertEquals(320, optimized.width()); + assertEquals(200, optimized.height()); + BufferedImage webp = ImageIO.read(new ByteArrayInputStream(optimized.bytes())); + assertEquals(320, webp.getWidth()); + assertEquals(200, webp.getHeight()); + } + + @Test + void smallImageIsNotUpscaledExceptCenteredCrop() throws Exception { + Map variants = ImageVariants.render(png(20, 10), 0.85f, 0.8f).stream() + .collect(Collectors.toMap(ImageVariants.VariantBytes::name, Function.identity())); + assertEquals(20, variants.get("thumbnail").width()); + assertEquals(10, variants.get("thumbnail").height()); + assertEquals(20, variants.get("optimized").width()); + assertEquals(128, variants.get("crop").width()); + assertEquals(128, variants.get("crop").height()); + } + + @Test + void fitUsesFloorAndDoesNotEnlarge() { + assertEquals(256, ImageVariants.fitted(320, 200, 256, false)[0]); + assertEquals(160, ImageVariants.fitted(320, 200, 256, false)[1]); + assertEquals(320, ImageVariants.fitted(320, 200, 1280, false)[0]); + assertEquals(10, ImageVariants.fitted(10, 10, 256, false)[0]); + } + + private static byte[] png(int width, int height) throws Exception { + BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB); + for (int y = 0; y < height; y++) { + for (int x = 0; x < width; x++) { + image.setRGB(x, y, new Color(x * 255 / width, y * 255 / height, 80).getRGB()); + } + } + ByteArrayOutputStream out = new ByteArrayOutputStream(); + ImageIO.write(image, "png", out); + return out.toByteArray(); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/service/RetryDelayTest.java b/backend/src/test/java/ru/mtuci/messenger/service/RetryDelayTest.java new file mode 100644 index 00000000..881238b2 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/service/RetryDelayTest.java @@ -0,0 +1,24 @@ +package ru.mtuci.messenger.service; + +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; + +class RetryDelayTest { + + @Test + void backoffGrowsThenStaysInsideHalfToFullWindow() { + assertEquals(1000, RetryDelay.exponentialMillis(1, 1000, 30_000)); + assertEquals(2000, RetryDelay.exponentialMillis(2, 1000, 30_000)); + assertEquals(16_000, RetryDelay.exponentialMillis(5, 1000, 30_000)); + assertEquals(30_000, RetryDelay.exponentialMillis(20, 1000, 30_000)); + for (int attempt = 1; attempt <= 8; attempt++) { + long exp = RetryDelay.exponentialMillis(attempt, 1000, 30_000); + for (int i = 0; i < 20; i++) { + long delay = RetryDelay.millis(attempt, 1000, 30_000); + assertTrue(delay >= exp / 2 && delay <= exp, "attempt " + attempt + " delay " + delay); + } + } + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/service/TextRulesTest.java b/backend/src/test/java/ru/mtuci/messenger/service/TextRulesTest.java new file mode 100644 index 00000000..04f703d5 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/service/TextRulesTest.java @@ -0,0 +1,16 @@ +package ru.mtuci.messenger.service; + +import org.junit.jupiter.api.Test; +import ru.mtuci.messenger.exception.InvalidRequestException; + +import static org.junit.jupiter.api.Assertions.assertThrows; + +class TextRulesTest { + + @Test + void rejectsBlankAndTooShortPassword() { + assertThrows(InvalidRequestException.class, () -> TextRules.displayName(" ")); + assertThrows(InvalidRequestException.class, () -> TextRules.message(" \n ")); + assertThrows(InvalidRequestException.class, () -> TextRules.password("short")); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/storage/OrphanSweeperTest.java b/backend/src/test/java/ru/mtuci/messenger/storage/OrphanSweeperTest.java new file mode 100644 index 00000000..09b704b1 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/storage/OrphanSweeperTest.java @@ -0,0 +1,19 @@ +package ru.mtuci.messenger.storage; + +import org.junit.jupiter.api.Test; + +import java.time.Instant; + +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +class OrphanSweeperTest { + + @Test + void graceKeepsFreshObjectAndReferencedRow() { + Instant cutoff = Instant.parse("2026-11-06T07:00:00Z"); + assertFalse(OrphanSweeper.expiredUnreferenced(cutoff.plusSeconds(1), cutoff, false)); + assertFalse(OrphanSweeper.expiredUnreferenced(cutoff.minusSeconds(30), cutoff, true)); + assertTrue(OrphanSweeper.expiredUnreferenced(cutoff.minusSeconds(30), cutoff, false)); + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/support/ApiClient.java b/backend/src/test/java/ru/mtuci/messenger/support/ApiClient.java new file mode 100644 index 00000000..4c9a7342 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/support/ApiClient.java @@ -0,0 +1,93 @@ +package ru.mtuci.messenger.support; + +import tools.jackson.databind.JsonNode; +import tools.jackson.databind.json.JsonMapper; + +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.LinkedHashMap; +import java.util.Map; + +public final class ApiClient { + + private final HttpClient http = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(5)).build(); + private final String base; + private final JsonMapper json; + + public ApiClient(int port, JsonMapper json) { + this.base = "http://127.0.0.1:" + port; + this.json = json; + } + + public Result get(String path, String token) { + return send("GET", path, token, null, null, null); + } + + public Result get(String path, String token, Map headers) { + return send("GET", path, token, null, null, headers); + } + + public Result post(String path, Object body) { + return send("POST", path, null, null, body, null); + } + + public Result post(String path, String token, Object body) { + return send("POST", path, token, null, body, null); + } + + public Result post(String path, String token, Object body, Map headers) { + return send("POST", path, token, null, body, headers); + } + + public Result postBasic(String path, String basic) { + return send("POST", path, null, basic, null, null); + } + + public Result delete(String path, String token) { + return send("DELETE", path, token, null, null, null); + } + + private Result send(String method, String path, String token, String basic, Object body, Map extra) { + HttpRequest.Builder builder = HttpRequest.newBuilder(URI.create(base + path)).timeout(Duration.ofSeconds(20)); + if (body == null) { + builder.method(method, HttpRequest.BodyPublishers.noBody()); + } else { + builder.method(method, HttpRequest.BodyPublishers.ofString(json.writeValueAsString(body), StandardCharsets.UTF_8)); + builder.header("Content-Type", "application/json"); + } + if (token != null) { + builder.header("Authorization", "Bearer " + token); + } + if (basic != null) { + builder.header("Authorization", "Basic " + basic); + } + if (extra != null) { + extra.forEach(builder::header); + } + try { + HttpResponse response = http.send(builder.build(), HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); + String raw = response.body() == null ? "" : response.body(); + JsonNode tree = raw.isBlank() ? null : json.readTree(raw); + return new Result(response.statusCode(), response.headers().firstValue("X-Request-Id").orElse(""), + response.headers().firstValue("X-Instance-Id").orElse(""), + response.headers().firstValue("WWW-Authenticate").orElse(""), tree, raw); + } catch (Exception exception) { + throw new IllegalStateException(method + " " + path, exception); + } + } + + public Map user(String username, String displayName, String password) { + Map body = new LinkedHashMap<>(); + body.put("username", username); + body.put("display_name", displayName); + body.put("password", password); + return body; + } + + public record Result(int status, String requestId, String instanceId, String authenticate, JsonNode body, String raw) { + } +} diff --git a/backend/src/test/java/ru/mtuci/messenger/support/LabContainers.java b/backend/src/test/java/ru/mtuci/messenger/support/LabContainers.java new file mode 100644 index 00000000..52aac901 --- /dev/null +++ b/backend/src/test/java/ru/mtuci/messenger/support/LabContainers.java @@ -0,0 +1,64 @@ +package ru.mtuci.messenger.support; + +import org.springframework.test.context.DynamicPropertyRegistry; +import org.testcontainers.containers.GenericContainer; +import org.testcontainers.containers.wait.strategy.Wait; +import org.testcontainers.kafka.KafkaContainer; +import org.testcontainers.postgresql.PostgreSQLContainer; +import org.testcontainers.utility.DockerImageName; +import org.testcontainers.utility.MountableFile; + +import java.time.Duration; + +public final class LabContainers { + + public static final String S3_ACCESS_KEY = "messenger-test"; + public static final String S3_SECRET_KEY = "messenger-test-secret"; + public static final String S3_BOOTSTRAP_ACCESS_KEY = "messenger-bootstrap"; + public static final String S3_BOOTSTRAP_SECRET_KEY = "messenger-bootstrap-secret"; + public static final String S3_BUCKET = "messenger-attachments"; + + public static final PostgreSQLContainer POSTGRES = new PostgreSQLContainer("postgres:16.10"); + public static final GenericContainer REDIS = new GenericContainer<>("redis:7.4.6").withExposedPorts(6379); + public static final KafkaContainer KAFKA = new KafkaContainer(DockerImageName.parse("apache/kafka:3.9.1")); + public static final GenericContainer S3 = new GenericContainer<>("chrislusf/seaweedfs:4.52") + .withCopyToContainer(MountableFile.forClasspathResource("s3.json"), "/etc/seaweedfs/s3.json") + .withCommand("server", "-ip.bind=0.0.0.0", "-s3", "-s3.config=/etc/seaweedfs/s3.json", + "-filer.exposeDirectoryData=false") + .withExposedPorts(8333) + .waitingFor(Wait.forListeningPort().withStartupTimeout(Duration.ofSeconds(120))); + + static { + POSTGRES.start(); + REDIS.start(); + KAFKA.start(); + S3.start(); + } + + private LabContainers() { + } + + public static void register(DynamicPropertyRegistry registry) { + registry.add("spring.datasource.url", POSTGRES::getJdbcUrl); + registry.add("spring.datasource.username", POSTGRES::getUsername); + registry.add("spring.datasource.password", POSTGRES::getPassword); + registry.add("spring.data.redis.host", REDIS::getHost); + registry.add("spring.data.redis.port", () -> REDIS.getMappedPort(6379)); + registerS3(registry); + } + + public static void registerS3(DynamicPropertyRegistry registry) { + registry.add("app.s3.endpoint", () -> "http://" + S3.getHost() + ":" + S3.getMappedPort(8333)); + registry.add("app.s3.region", () -> "us-east-1"); + registry.add("app.s3.bucket", () -> S3_BUCKET); + registry.add("app.s3.access-key", () -> S3_ACCESS_KEY); + registry.add("app.s3.secret-key", () -> S3_SECRET_KEY); + registry.add("app.s3.bootstrap-access-key", () -> S3_BOOTSTRAP_ACCESS_KEY); + registry.add("app.s3.bootstrap-secret-key", () -> S3_BOOTSTRAP_SECRET_KEY); + registry.add("app.s3.sweep-initial-delay-ms", () -> "3600000"); + registry.add("spring.kafka.bootstrap-servers", KAFKA::getBootstrapServers); + registry.add("app.worker.consume", () -> "true"); + registry.add("app.outbox.poll-ms", () -> "200"); + registry.add("app.job.reaper-ms", () -> "1000"); + } +} diff --git a/backend/src/test/resources/s3.json b/backend/src/test/resources/s3.json new file mode 100644 index 00000000..27328455 --- /dev/null +++ b/backend/src/test/resources/s3.json @@ -0,0 +1,24 @@ +{ + "identities": [ + { + "name": "messenger-api", + "credentials": [ + { + "accessKey": "messenger-test", + "secretKey": "messenger-test-secret" + } + ], + "actions": ["Read", "Write", "List"] + }, + { + "name": "messenger-bootstrap", + "credentials": [ + { + "accessKey": "messenger-bootstrap", + "secretKey": "messenger-bootstrap-secret" + } + ], + "actions": ["Admin"] + } + ] +} diff --git a/compose.yaml b/compose.yaml new file mode 100644 index 00000000..7309a316 --- /dev/null +++ b/compose.yaml @@ -0,0 +1,245 @@ +name: messenger-lab5 + +x-app-image: &app-image + image: messenger-lab5:1.0.0 + pull_policy: never + +x-datasource: &datasource + SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/${POSTGRES_DB:-messenger} + SPRING_DATASOURCE_USERNAME: ${POSTGRES_USER:-messenger} + SPRING_DATASOURCE_PASSWORD: ${POSTGRES_PASSWORD:-messenger} + SPRING_DATA_REDIS_HOST: redis + SPRING_DATA_REDIS_PORT: "6379" + S3_ENDPOINT: http://s3:8333 + S3_REGION: ${S3_REGION:-us-east-1} + S3_BUCKET: ${S3_BUCKET:-messenger-attachments} + S3_ACCESS_KEY: ${S3_ACCESS_KEY:?set S3_ACCESS_KEY in .env} + S3_SECRET_KEY: ${S3_SECRET_KEY:?set S3_SECRET_KEY in .env} + S3_BOOTSTRAP_ACCESS_KEY: ${S3_BOOTSTRAP_ACCESS_KEY:?set S3_BOOTSTRAP_ACCESS_KEY in .env} + S3_BOOTSTRAP_SECRET_KEY: ${S3_BOOTSTRAP_SECRET_KEY:?set S3_BOOTSTRAP_SECRET_KEY in .env} + S3_ORPHAN_GRACE: ${S3_ORPHAN_GRACE:-2m} + MAX_UPLOAD_BYTES: ${MAX_UPLOAD_BYTES:-10485760} + MAX_IMAGE_PIXELS: ${MAX_IMAGE_PIXELS:-40000000} + SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092 + WORKER_CONCURRENCY: ${WORKER_CONCURRENCY:-2} + JOB_MAX_ATTEMPTS: ${JOB_MAX_ATTEMPTS:-5} + JOB_TIMEOUT_SECONDS: ${JOB_TIMEOUT_SECONDS:-60} + JOB_BACKOFF_BASE_MS: ${JOB_BACKOFF_BASE_MS:-1000} + JOB_BACKOFF_CAP_MS: ${JOB_BACKOFF_CAP_MS:-30000} + STAGING_RETENTION_SECONDS: ${STAGING_RETENTION_SECONDS:-86400} + DEMO_TOKEN: ${DEMO_TOKEN:-lab-demo} + JPEG_QUALITY: ${JPEG_QUALITY:-0.85} + WEBP_QUALITY: ${WEBP_QUALITY:-0.8} + LIQUIBASE_ANALYTICS_ENABLED: "false" + JAVA_TOOL_OPTIONS: -XX:MaxRAMPercentage=75.0 -Djava.awt.headless=true + +services: + postgres: + image: postgres:16.10 + environment: + POSTGRES_USER: ${POSTGRES_USER:-messenger} + POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-messenger} + POSTGRES_DB: ${POSTGRES_DB:-messenger} + volumes: + - postgres-data:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-messenger} -d ${POSTGRES_DB:-messenger}"] + interval: 5s + timeout: 3s + retries: 20 + shm_size: 128mb + restart: unless-stopped + deploy: + resources: + limits: + cpus: "1.0" + memory: 512M + + redis: + image: redis:7.4.6 + command: ["redis-server", "--appendonly", "yes"] + volumes: + - redis-data:/data + healthcheck: + test: ["CMD", "redis-cli", "ping"] + interval: 5s + timeout: 3s + retries: 20 + restart: unless-stopped + deploy: + resources: + limits: + cpus: "0.25" + memory: 128M + + kafka: + image: apache/kafka:3.9.1 + hostname: kafka + environment: + CLUSTER_ID: 5L6g3nShT-eMCtK--X86sw + KAFKA_NODE_ID: 1 + KAFKA_PROCESS_ROLES: broker,controller + KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://localhost:9093 + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092 + KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT + KAFKA_CONTROLLER_QUORUM_VOTERS: 1@localhost:9093 + KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 + KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 + KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 + KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 + KAFKA_NUM_PARTITIONS: 2 + KAFKA_LOG_DIRS: /var/lib/kafka/data + KAFKA_HEAP_OPTS: "-Xmx256M -Xms256M" + volumes: + - kafka-data:/var/lib/kafka/data + healthcheck: + test: ["CMD-SHELL", "/opt/kafka/bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092 >/dev/null"] + interval: 5s + timeout: 5s + retries: 20 + start_period: 30s + restart: unless-stopped + deploy: + resources: + limits: + cpus: "0.50" + memory: 768M + + s3: + image: chrislusf/seaweedfs:4.52 + command: + - server + - -ip.bind=0.0.0.0 + - -s3 + - -s3.config=/etc/seaweedfs/s3.json + - -filer.exposeDirectoryData=false + environment: + S3_ACCESS_KEY: ${S3_ACCESS_KEY:?set S3_ACCESS_KEY in .env} + S3_SECRET_KEY: ${S3_SECRET_KEY:?set S3_SECRET_KEY in .env} + S3_BOOTSTRAP_ACCESS_KEY: ${S3_BOOTSTRAP_ACCESS_KEY:?set S3_BOOTSTRAP_ACCESS_KEY in .env} + S3_BOOTSTRAP_SECRET_KEY: ${S3_BOOTSTRAP_SECRET_KEY:?set S3_BOOTSTRAP_SECRET_KEY in .env} + volumes: + - s3-data:/data + - ./infra/s3.json:/etc/seaweedfs/s3.json:ro + healthcheck: + test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1:9333/cluster/status"] + interval: 5s + timeout: 3s + retries: 20 + start_period: 20s + restart: unless-stopped + deploy: + resources: + limits: + cpus: "0.50" + memory: 512M + + migrate: + <<: *app-image + build: + context: . + dockerfile: Dockerfile + command: ["--spring.profiles.active=migrate"] + environment: + <<: *datasource + SPRING_LIQUIBASE_ENABLED: "true" + SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600} + IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400} + depends_on: + postgres: + condition: service_healthy + redis: + condition: service_healthy + healthcheck: + disable: true + restart: "no" + deploy: + resources: + limits: + cpus: "0.50" + memory: 1024M + + api: + <<: *app-image + environment: + <<: *datasource + APP_HOST: 0.0.0.0 + APP_PORT: "8080" + APP_ENV: ${APP_ENV:-lab} + SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600} + IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400} + SPRING_LIQUIBASE_ENABLED: "false" + depends_on: + migrate: + condition: service_completed_successfully + postgres: + condition: service_healthy + redis: + condition: service_healthy + kafka: + condition: service_healthy + s3: + condition: service_healthy + stop_grace_period: 12s + init: true + restart: unless-stopped + deploy: + replicas: ${API_REPLICAS:-2} + resources: + limits: + cpus: "1.0" + memory: 1024M + + worker: + <<: *app-image + command: ["--spring.profiles.active=worker"] + environment: + <<: *datasource + JAVA_TOOL_OPTIONS: -XX:MaxRAMPercentage=60.0 -XX:+ExitOnOutOfMemoryError -Djava.awt.headless=true + SPRING_LIQUIBASE_ENABLED: "false" + depends_on: + migrate: + condition: service_completed_successfully + postgres: + condition: service_healthy + kafka: + condition: service_healthy + s3: + condition: service_healthy + healthcheck: + test: ["CMD-SHELL", "kill -0 1"] + interval: 5s + timeout: 3s + retries: 10 + start_period: 40s + stop_grace_period: 20s + init: true + restart: unless-stopped + deploy: + resources: + limits: + cpus: "1.0" + memory: 1024M + + lb: + image: haproxy:3.2.6 + volumes: + - ./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro + ports: + - "127.0.0.1:8080:8080" + depends_on: + api: + condition: service_healthy + restart: unless-stopped + deploy: + resources: + limits: + cpus: "0.25" + memory: 128M + +volumes: + postgres-data: + redis-data: + kafka-data: + s3-data: diff --git a/evidence/explain-messages.txt b/evidence/explain-messages.txt new file mode 100644 index 00000000..b8a7cc3a --- /dev/null +++ b/evidence/explain-messages.txt @@ -0,0 +1,10 @@ +Limit (cost=0.29..5.09 rows=50 width=24) (actual time=0.008..0.015 rows=50 loops=1) + Buffers: shared hit=5 + -> Index Only Scan using messages_history_idx on messages (cost=0.29..481.62 rows=5004 width=24) (actual time=0.007..0.013 rows=50 loops=1) + Index Cond: ((chat_id = 'eb32a209-b5b6-42b6-b902-84c9636886d9'::uuid) AND (ROW(created_at, id) > ROW('2026-01-01 03:00:05+03'::timestamp with time zone, '4f2daf20-57e7-41f3-bac5-566e109942d3'::uuid))) + Heap Fetches: 50 + Buffers: shared hit=5 +Planning: + Buffers: shared hit=27 +Planning Time: 0.082 ms +Execution Time: 0.027 ms diff --git a/evidence/grade4.txt b/evidence/grade4.txt new file mode 100644 index 00000000..ef3d7b12 --- /dev/null +++ b/evidence/grade4.txt @@ -0,0 +1,48 @@ +== worker down, API restart, then worker == +queued upload id=9c4ab448-af4d-4acf-883d-4a8929a8b5e1 status=queued error=None variants=[] content=conflict +after API restart id=9c4ab448-af4d-4acf-883d-4a8929a8b5e1 status=queued error=None variants=[] +after worker start id=9c4ab448-af4d-4acf-883d-4a8929a8b5e1 status=ready error=None variants=['thumbnail:image/jpeg:256x160', 'crop:image/jpeg:128x128', 'optimized:image/webp:320x200'] +bytes original=111817 thumbnail=(256, 160) image/jpeg crop=(128, 128) image/jpeg optimized=(320, 200) image/webp +== broker restart with durable topic == +topic end before kafka restart: 2 -> 3 +topic end after kafka restart: 3 +after broker restart id=45a5da06-a3c3-4d12-ad25-3d5a9154d531 status=ready error=None variants=['thumbnail:image/jpeg:256x160', 'crop:image/jpeg:128x128', 'optimized:image/webp:320x200'] +== outbox paused across API kill == +paused=True unpublished=1 content=conflict id=26bd80e5-7074-44b8-b2d5-1a28b14343fe status=queued error=None variants=[] +after API kill paused=True unpublished=1 id=26bd80e5-7074-44b8-b2d5-1a28b14343fe status=queued error=None variants=[] +after outbox resume id=26bd80e5-7074-44b8-b2d5-1a28b14343fe status=ready error=None variants=['thumbnail:image/jpeg:256x160', 'crop:image/jpeg:128x128', 'optimized:image/webp:320x200'] +== worker exit after object write, before ack == +restart_count 0->1 id=9e1ed645-d5b7-4006-a84c-75ac547eb165 status=ready error=None variants=['thumbnail:image/jpeg:256x160', 'crop:image/jpeg:128x128', 'optimized:image/webp:320x200'] db=crop:1 +optimized:1 +thumbnail:1 +replay of ready status=ready variants=3 +== permanent failure and explicit replay == +failed registry 1->2 message=conflict id=05e76bf9-a6d0-4dc9-9ea0-ee1950a1218b status=failed error=processing_failed variants=[] +topics: +attachments.process end=6 +attachments.failed end=1 +replay of failed status=queued +after replay id=05e76bf9-a6d0-4dc9-9ea0-ee1950a1218b status=ready error=None variants=['thumbnail:image/jpeg:256x160', 'crop:image/jpeg:128x128', 'optimized:image/webp:320x200'] db=crop:1 +optimized:1 +thumbnail:1 second_replay=ready +== rejected upload == +corrupt png unsupported_media_type +== runtime limits == +messenger-lab5-worker-1 Up 54 seconds (healthy) 8080/tcp +messenger-lab5-api-2 Up About a minute (healthy) 8080/tcp +messenger-lab5-api-1 Up About a minute (healthy) 8080/tcp +messenger-lab5-kafka-1 Up 2 minutes (healthy) 9092/tcp +messenger-lab5-lb-1 Up 19 minutes 127.0.0.1:8080->8080/tcp +messenger-lab5-postgres-1 Up 19 minutes (healthy) 5432/tcp +messenger-lab5-redis-1 Up 19 minutes (healthy) 6379/tcp +messenger-lab5-s3-1 Up 19 minutes (healthy) 7333/tcp, 8080/tcp, 8333/tcp, 8888/tcp, 9333/tcp, 18080/tcp, 18888/tcp, 19333/tcp +api-1 1073741824 1000000000 {} +api-2 1073741824 1000000000 {} +worker-1 1073741824 1000000000 {} +lb-1 134217728 250000000 {"8080/tcp":[{"HostIp":"127.0.0.1","HostPort":"8080"}]} +postgres-1 536870912 1000000000 {} +redis-1 134217728 250000000 {} +kafka-1 805306368 500000000 {} +s3-1 536870912 500000000 {} +anonymous s3 status=403 +PASS grade4 evidence diff --git a/evidence/results.txt b/evidence/results.txt new file mode 100644 index 00000000..58f94434 --- /dev/null +++ b/evidence/results.txt @@ -0,0 +1,33 @@ +Прогон 12 ноября 2026, Mac arm64, 12 CPU, 24 GiB, Docker 28.5.1. +Секреты и Bearer в этот файл не записывались. + +make test +PASS: lab5 public API scenario; infrastructure and grading evidence remain separate. + +make test-persistence ACTION_SCRIPT=scripts/restart.sh +После перехода на Kafka хук перезапускает и kafka. PASS. +Хук перезапускает lb, api, worker, postgres, redis, s3, rabbitmq и не удаляет volumes. +Первый прогон с паузой 5 с после /health/ready прошёл. +Повтор сразу после серии отказов: TimeoutError на GET content (клиент ждёт 2 с), чтение SeaweedFS не успело. +Пауза в хуке увеличена до 12 с. Следующий прогон: +PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence. + +mvn -f backend/pom.xml -q test +Код выхода 0. 22 теста, включая AttachmentIntegrationTest, ImageVariantsTest и RetryDelayTest. + +make test-failover ACTION_SCRIPT=scripts/stop-one.sh +Первый запуск сразу после persistence: TimeoutError на POST вложения ещё до hook (клиент ждёт 2 с). +Повтор на прогретом стенде: +{"observed_instances_before": 2, "successful_writes": 117, "failed_attempts": 1, "first_recovery_seconds": 1.795} +PASS: failover preserved confirmed data and existing session. +Остановленная реплика поднята вручную: docker compose --env-file .env start api. + +python3 scripts/grade4-evidence.py +После замены брокера на Kafka 3.9.1, 12 ноября 2026: +PASS grade4 evidence +Топик attachments.process: конец лога 2 → 3, после restart kafka остался 3, вложение стало ready. +Топик attachments.failed end=1 после постоянного сбоя. +Подробности без токенов: evidence/grade4.txt + +Анонимный GET объекта SeaweedFS из контейнера api без ключа: HTTP 403. +Наружу опубликован только lb, 127.0.0.1:8080. Память контейнеров в байтах — в grade4.txt. diff --git a/infra/haproxy.cfg b/infra/haproxy.cfg new file mode 100644 index 00000000..ccd5dbf8 --- /dev/null +++ b/infra/haproxy.cfg @@ -0,0 +1,39 @@ +global + maxconn 200 + log stdout format raw local0 + +defaults + mode http + log global + option httplog + timeout connect 1s + timeout client 60s + timeout server 30s + timeout http-request 60s + timeout check 1s + retries 0 + +resolvers docker + nameserver dns 127.0.0.11:53 + resolve_retries 3 + timeout resolve 1s + timeout retry 1s + hold valid 1s + hold other 1s + hold refused 1s + hold nx 1s + hold timeout 1s + hold obsolete 1s + accepted_payload_size 8192 + +frontend http + bind *:8080 + default_backend api + +backend api + balance roundrobin + option httpchk + http-check send meth GET uri /health/ready ver HTTP/1.1 hdr host api + http-check expect status 200 + default-server inter 1s fall 2 rise 1 on-marked-down shutdown-sessions + server-template api 8 api:8080 check resolvers docker init-addr none resolve-prefer ipv4 diff --git a/infra/s3.json b/infra/s3.json new file mode 100644 index 00000000..81bf62df --- /dev/null +++ b/infra/s3.json @@ -0,0 +1,24 @@ +{ + "identities": [ + { + "name": "messenger-api", + "credentials": [ + { + "accessKey": "${S3_ACCESS_KEY}", + "secretKey": "${S3_SECRET_KEY}" + } + ], + "actions": ["Read", "Write", "List"] + }, + { + "name": "messenger-bootstrap", + "credentials": [ + { + "accessKey": "${S3_BOOTSTRAP_ACCESS_KEY}", + "secretKey": "${S3_BOOTSTRAP_SECRET_KEY}" + } + ], + "actions": ["Admin"] + } + ] +} diff --git a/scripts/grade4-evidence.py b/scripts/grade4-evidence.py new file mode 100644 index 00000000..b96f81ec --- /dev/null +++ b/scripts/grade4-evidence.py @@ -0,0 +1,278 @@ +#!/usr/bin/env python3 +"""Grade-4 fault demos. Prints statuses and counts only; never prints tokens or passwords.""" +import json +import subprocess +import sys +import time +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "tests")) +from client import Client, Failure # noqa: E402 +from image_probe import jpeg_size # noqa: E402 + +lines = [] + + +def log(message): + print(message, flush=True) + lines.append(message) + + +def run(args, check=True): + result = subprocess.run(args, cwd=ROOT, text=True, capture_output=True) + if check and result.returncode != 0: + raise SystemExit(args[0] + " failed: " + result.stderr[-400:]) + return result + + +def compose(*args): + run(["docker", "compose", "--env-file", ".env", *args]) + + +def demo_token(): + for line in (ROOT / ".env").read_text().splitlines(): + if line.startswith("DEMO_TOKEN="): + return line.split("=", 1)[1].strip() + return "lab-demo" + + +def psql(sql): + result = run([ + "docker", "exec", "messenger-lab5-postgres-1", "sh", "-c", + 'psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" -At -c "$1"', "psql", sql, + ]) + return result.stdout.strip() + + +def topic_end(topic): + result = run([ + "docker", "exec", "messenger-lab5-kafka-1", + "/opt/kafka/bin/kafka-get-offsets.sh", + "--bootstrap-server", "localhost:9092", + "--topic", topic, + ]) + total = 0 + for line in result.stdout.splitlines(): + parts = line.strip().split(":") + if len(parts) >= 3 and parts[-1].lstrip("-").isdigit(): + total += int(parts[-1]) + return total + + +def offsets(): + return "attachments.process end=" + str(topic_end("attachments.process")) + "\nattachments.failed end=" + str(topic_end("attachments.failed")) + + +def restart_count(): + result = run(["docker", "inspect", "-f", "{{.RestartCount}}", "messenger-lab5-worker-1"]) + return int(result.stdout.strip()) + + +def wait_health(client): + deadline = time.monotonic() + 90 + while True: + try: + client.call("GET", "/health/ready", schema="Health") + return + except Failure: + if time.monotonic() > deadline: + raise + time.sleep(1) + + +def poll(client, actor, attachment, seconds=70): + deadline = time.monotonic() + seconds + last = attachment + while time.monotonic() < deadline: + last = client.call("GET", "/api/v1/attachments/" + attachment["id"], actor=actor, schema="Attachment")["json"] + if last["status"] in ("ready", "failed"): + return last + time.sleep(0.4) + raise SystemExit("attachment stayed " + last["status"]) + + +def summary(attachment): + variants = [ + f"{item['name']}:{item['content_type']}:{item['width']}x{item['height']}" + for item in attachment.get("variants") or [] + ] + return f"id={attachment['id']} status={attachment['status']} error={attachment.get('error_code')} variants={variants}" + + +def webp_size(data): + if data[:4] != b"RIFF" or data[8:12] != b"WEBP": + raise SystemExit("optimized payload is not WebP") + index = 12 + while index + 8 <= len(data): + kind = data[index:index + 4] + size = int.from_bytes(data[index + 4:index + 8], "little") + payload = data[index + 8:index + 8 + size] + if kind == b"VP8 ": + width = int.from_bytes(payload[6:8], "little") & 0x3FFF + height = int.from_bytes(payload[8:10], "little") & 0x3FFF + return width, height + if kind == b"VP8X": + width = 1 + int.from_bytes(payload[4:7], "little") + height = 1 + int.from_bytes(payload[7:10], "little") + return width, height + index += 8 + size + (size & 1) + raise SystemExit("WebP dimensions not found") + + +def content(client, actor, attachment, variant=None): + path = "/api/v1/attachments/" + attachment["id"] + "/content" + if variant: + path += "?variant=" + variant + return client.call("GET", path, actor=actor) + + +def main(): + token = demo_token() + client = Client("http://127.0.0.1:8080", timeout=15) + wait_health(client) + headers = {"X-Demo-Token": token} + + log("== worker down, API restart, then worker ==") + compose("stop", "worker") + alice, bob, _eve, chat = client.bootstrap() + attachment, original = client.attachment(alice, chat) + client.message(alice, chat, "вложение ещё queued", attachments=[attachment["id"]]) + blocked = client.call("GET", "/api/v1/attachments/" + attachment["id"] + "/content", 409, actor=bob) + log("queued upload " + summary(attachment) + " content=" + blocked["json"]["error"]["code"]) + compose("restart", "api") + wait_health(client) + queued = client.call("GET", "/api/v1/attachments/" + attachment["id"], actor=bob, schema="Attachment")["json"] + log("after API restart " + summary(queued)) + if queued["status"] != "queued": + raise SystemExit("job was lost while worker and API were restarted") + compose("start", "worker") + ready = poll(client, bob, queued) + log("after worker start " + summary(ready)) + downloaded = content(client, bob, ready) + if downloaded["bytes"] != original: + raise SystemExit("original bytes changed") + thumb = content(client, bob, ready, "thumbnail") + crop = content(client, bob, ready, "crop") + optimized = content(client, bob, ready, "optimized") + log(f"bytes original={len(original)} thumbnail={jpeg_size(thumb['bytes'])} {thumb['headers'].get_content_type()} " + f"crop={jpeg_size(crop['bytes'])} {crop['headers'].get_content_type()} " + f"optimized={webp_size(optimized['bytes'])} {optimized['headers'].get_content_type()}") + + log("== broker restart with durable topic ==") + compose("stop", "worker") + published_before = topic_end("attachments.process") + held, _ = client.attachment(alice, chat) + deadline = time.monotonic() + 20 + published = published_before + while time.monotonic() < deadline: + published = topic_end("attachments.process") + if published > published_before: + break + time.sleep(0.5) + log("topic end before kafka restart: " + str(published_before) + " -> " + str(published)) + if published <= published_before: + raise SystemExit("accepted job was not waiting in the broker") + compose("restart", "kafka") + deadline = time.monotonic() + 60 + while time.monotonic() < deadline: + state = run(["docker", "inspect", "-f", "{{.State.Health.Status}}", "messenger-lab5-kafka-1"], check=False) + if state.stdout.strip() == "healthy": + break + time.sleep(1) + retained = topic_end("attachments.process") + log("topic end after kafka restart: " + str(retained)) + if retained < published: + raise SystemExit("broker restart dropped the waiting job") + compose("start", "worker") + held_ready = poll(client, alice, held) + log("after broker restart " + summary(held_ready)) + + log("== outbox paused across API kill ==") + paused = json.loads(client.call("POST", "/internal/demo/outbox-pause", headers=headers)["bytes"]) + staged, _ = client.attachment(alice, chat) + outbox = json.loads(client.call("GET", "/internal/demo/outbox", headers=headers)["bytes"]) + blocked = client.call("GET", "/api/v1/attachments/" + staged["id"] + "/content", 409, actor=alice) + log(f"paused={paused['paused']} unpublished={outbox['unpublished']} content={blocked['json']['error']['code']} {summary(staged)}") + compose("kill", "api") + compose("start", "api") + wait_health(client) + outbox = json.loads(client.call("GET", "/internal/demo/outbox", headers=headers)["bytes"]) + staged = client.call("GET", "/api/v1/attachments/" + staged["id"], actor=alice, schema="Attachment")["json"] + log(f"after API kill paused={outbox['paused']} unpublished={outbox['unpublished']} {summary(staged)}") + if not outbox["paused"] or outbox["unpublished"] < 1 or staged["status"] != "queued": + raise SystemExit("outbox intention did not survive API kill") + client.call("POST", "/internal/demo/outbox-resume", headers=headers) + staged = poll(client, alice, staged) + log("after outbox resume " + summary(staged)) + + log("== worker exit after object write, before ack ==") + before = restart_count() + run(["docker", "exec", "messenger-lab5-worker-1", "touch", "/tmp/messenger-crash-after-write"]) + crashed, _ = client.attachment(alice, chat) + deadline = time.monotonic() + 40 + while time.monotonic() < deadline and restart_count() <= before: + time.sleep(0.5) + after = restart_count() + crashed = poll(client, alice, crashed) + variants = psql("select name || ':' || count(*) from attachment_variants where attachment_id='" + + crashed["id"] + "' group by name order by name") + log(f"restart_count {before}->{after} {summary(crashed)} db={variants}") + replay = json.loads(client.call("POST", "/internal/demo/attachments/" + crashed["id"] + "/replay", headers=headers)["bytes"]) + log("replay of ready status=" + replay["status"] + " variants=" + str(len(replay.get("variants") or []))) + + log("== permanent failure and explicit replay ==") + failed_before = json.loads(client.call("GET", "/internal/demo/failed", headers=headers)["bytes"])["count"] + run(["docker", "exec", "messenger-lab5-worker-1", "touch", "/tmp/messenger-fail-permanent"]) + broken, _ = client.attachment(alice, chat) + broken = poll(client, alice, broken) + denied = client.call("POST", "/api/v1/chats/" + chat["id"] + "/messages", 409, actor=alice, + body={"text": "нельзя прикрепить failed", "attachment_ids": [broken["id"]]}, + headers={"Idempotency-Key": "fail-" + broken["id"]}) + failed_after = json.loads(client.call("GET", "/internal/demo/failed", headers=headers)["bytes"])["count"] + log(f"failed registry {failed_before}->{failed_after} message={denied['json']['error']['code']} {summary(broken)}") + log("topics:\n" + offsets()) + replay = json.loads(client.call("POST", "/internal/demo/attachments/" + broken["id"] + "/replay", headers=headers)["bytes"]) + log("replay of failed status=" + replay["status"]) + recovered = poll(client, alice, replay) + variants = psql("select name || ':' || count(*) from attachment_variants where attachment_id='" + + recovered["id"] + "' group by name order by name") + again = json.loads(client.call("POST", "/internal/demo/attachments/" + recovered["id"] + "/replay", headers=headers)["bytes"]) + log(f"after replay {summary(recovered)} db={variants} second_replay={again['status']}") + + log("== rejected upload ==") + rejected = client.call("POST", "/api/v1/chats/" + chat["id"] + "/attachments?filename=bad.png", 415, + actor=alice, raw=b"not-an-image", headers={"Content-Type": "image/png"}) + log("corrupt png " + rejected["json"]["error"]["code"]) + + log("== runtime limits ==") + ps = run(["docker", "ps", "--filter", "name=messenger-lab5", "--format", "{{.Names}} {{.Status}} {{.Ports}}"]) + log(ps.stdout.rstrip()) + for name in ("api-1", "api-2", "worker-1", "lb-1", "postgres-1", "redis-1", "kafka-1", "s3-1"): + inspected = run(["docker", "inspect", "-f", + "{{.HostConfig.Memory}} {{.HostConfig.NanoCpus}} {{json .HostConfig.PortBindings}}", + "messenger-lab5-" + name]) + log(name + " " + inspected.stdout.strip()) + anonymous = run(["docker", "exec", "messenger-lab5-api-1", "curl", "-s", "-o", "/dev/null", "-w", "%{http_code}", + "http://s3:8333/messenger-attachments/originals/missing"]) + log("anonymous s3 status=" + anonymous.stdout.strip()) + log("PASS grade4 evidence") + + +if __name__ == "__main__": + try: + main() + except (Failure, TimeoutError, OSError) as exc: + log("FAIL " + str(exc)) + raise SystemExit(1) + finally: + subprocess.run(["docker", "exec", "messenger-lab5-worker-1", "rm", "-f", + "/tmp/messenger-crash-after-write", "/tmp/messenger-fail-permanent"], cwd=ROOT) + token = demo_token() + subprocess.run(["curl", "-fsS", "-X", "POST", "-H", "X-Demo-Token: " + token, + "http://127.0.0.1:8080/internal/demo/outbox-resume"], cwd=ROOT, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + subprocess.run(["docker", "compose", "--env-file", ".env", "start", "worker", "api"], cwd=ROOT, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + (ROOT / "evidence").mkdir(exist_ok=True) + (ROOT / "evidence" / "grade4.txt").write_text("\n".join(lines) + "\n") diff --git a/scripts/restart.sh b/scripts/restart.sh new file mode 100755 index 00000000..e920a0ac --- /dev/null +++ b/scripts/restart.sh @@ -0,0 +1,22 @@ +#!/bin/sh +set -eu +cd "$(dirname "$0")/.." +if [ -f .env ]; then + docker compose --env-file .env restart lb api worker postgres redis s3 kafka +else + docker compose restart lb api worker postgres redis s3 kafka +fi + +i=0 +while [ "$i" -lt 60 ]; do + if curl -fsS -m 2 http://127.0.0.1:8080/health/ready >/dev/null 2>&1; then + # SeaweedFS answers master status before object reads fit the 2s client timeout. + sleep 12 + curl -fsS -m 2 http://127.0.0.1:8080/health/ready >/dev/null 2>&1 + exit 0 + fi + i=$((i + 1)) + sleep 1 +done +echo "stack did not become ready" >&2 +exit 1 diff --git a/scripts/stop-one.sh b/scripts/stop-one.sh new file mode 100755 index 00000000..4d07173d --- /dev/null +++ b/scripts/stop-one.sh @@ -0,0 +1,24 @@ +#!/bin/sh +set -eu +cd "$(dirname "$0")/.." +: "${TARGET_INSTANCE_ID:?test provides target API instance alias}" + +cid="" +for id in $(docker ps -q --filter "label=com.docker.compose.project=messenger-lab5" --filter "label=com.docker.compose.service=api"); do + full=$(docker inspect -f '{{.Id}}' "$id") + host=$(docker inspect -f '{{.Config.Hostname}}' "$id") + name=$(docker inspect -f '{{.Name}}' "$id") + name=${name#/} + short=$(printf '%s' "$full" | cut -c1-12) + if [ "$host" = "$TARGET_INSTANCE_ID" ] || [ "$short" = "$TARGET_INSTANCE_ID" ] || [ "$name" = "$TARGET_INSTANCE_ID" ] || [ "$full" = "$TARGET_INSTANCE_ID" ]; then + cid=$id + break + fi +done + +if [ -z "$cid" ]; then + echo "no running api replica matches TARGET_INSTANCE_ID" >&2 + exit 1 +fi + +docker stop -t 6 "$cid" >/dev/null -- 2.54.0