diff --git a/.gigacode/plans/1793452448001-happy-orchid.plan.md b/.gigacode/plans/1793452448001-happy-orchid.plan.md new file mode 100644 index 0000000..14c1bf4 --- /dev/null +++ b/.gigacode/plans/1793452448001-happy-orchid.plan.md @@ -0,0 +1,443 @@ +# Технический аудит проекта Lab4 — Messenger + +## A. Общая оценка + +**Проект:** `messenger-lab2` (Java 21 / Spring Boot / PostgreSQL / Redis / MinIO) +**Лаба:** 4 — «S3-хранилище и вложения» (contract version 1.4.0) +**Архитектура:** 2+ API-реплики → nginx LB → PostgreSQL + Redis + MinIO + +### Соответствие заданию +Проект реализует полноценный REST-мессенджер с аутентификацией, чатами, сообщениями, idempotency, вложениями, S3-интеграцией и фоновым reconciler'ом. Архитектура соответствует требованиям lab4 (2 API-реплики, LB, S3, PostgreSQL, Redis). + +### Целостность архитектуры +Архитектура целостная на уровне Java-кода: слои Controller → Service → Repository, MapStruct-мапперы, валидация, обработка исключений, ShedLock для координации. Однако **существенные проблемы в конфигурационных файлах (build.gradle, Dockerfile) блокируют сборку и запуск.** + +### Вердикт +**Вариант 1: Проект можно сохранить и точечно исправить.** Java-код написан качественно, архитектура соответствует заданию. Критические проблемы находятся в конфигурациях сборки и Docker, которые исправляются точечно. + +--- + +## B. Карта архитектуры + +``` + ┌─────────────┐ + │ Client │ + └──────┬──────┘ + │ :8080 + ┌──────▼──────┐ + │ nginx │ (LB, round-robin + keepalive) + │ :8080 → :80│ + └──────┬──────┘ + │ + ┌──────────────────┼──────────────────┐ + │ │ + ┌────────▼────────┐ ┌───────▼───────┐ + │ API replica 1 │ │ API replica 2 │ + │ Spring Boot 21 │ │ Spring Boot │ + │ ShedLock-aware │ │ ShedLock- │ + └──┬────┬────┬─────┘ └──┬────┬──────┘ + │ │ │ │ │ + ┌────▼──┐ │ ┌─▼─────┐ ┌─▼────┐ + │Postgre│ │ │ Redis │ │ MinIO │ + │SQL:16 │ │ │ 7.4 │ │:9000 │ + └───────┘ │ └───────┘ └──────┘ + │ + ┌───────▼───────┐ + │ Flyway migrate│ (one-shot) + └───────────────┘ +``` + +### Жизненный цикл запуска: +1. `docker compose up db redis minio` → сервисы становятся healthy +2. `minio-init` создаёт bucket `messenger-attachments`, пользователя, политику +3. `migrate` запускает Spring Boot в `NONE` режиме, выполняет Flyway V1__init.sql +4. `api` реплики запускаются с `SPRING_FLYWAY_ENABLED=false`, подключаются ко всем сервисам +5. nginx начинает проксировать трафик на здоровые API-реплики + +--- + +## C. Критические проблемы (блокирующие запуск) + +### C1. build.gradle — несуществующая версия Spring Boot 4.1.1 +**Файл:** `build.gradle`, строка 3 +**Проблема:** `org.springframework.boot` version `4.1.1` — такая версия не существует. Актуальная мажорная версия — 3.x. +**Последствие:** Gradle не разрешит ни один spring-boot-starter dependency. Сборка упадёт. + +### C2. build.gradle — несуществующие artifact names +**Файл:** `build.gradle` +**Проблемные зависимости:** +| Зависимость | Правильное имя | +|---|---| +| `spring-boot-starter-webmvc` | `spring-boot-starter-web` | +| `spring-boot-starter-jackson` | (часть `spring-boot-starter-web`) | +| `spring-boot-starter-restclient-test` | (часть `spring-boot-starter-test`) | +| `spring-boot-resttestclient` | не существует | +| `spring-boot-starter-actuator-test` | не существует | +| `spring-boot-starter-validation-test` | не существует | +| `spring-boot-starter-webmvc-test` | `spring-boot-starter-web-test` | + +**Последствие:** Gradle не разрешит эти зависимости. Сборка упадёт. + +### C3. build.gradle — невалидный Gradle-синтаксис (двоеточие) +**Файл:** `build.gradle`, строка `runtimeOnly 'io.netty:netty-transport-native-epoll::linux-x86_64'` +**Проблема:** Двойное двоеточие `::` — невалидный синтаксис Gradle. +**Последствие:** Ошибка парсинга build-файла. + +### C4. Dockerfile — несуществующий Gradle 9.7.1 +**Файл:** `Dockerfile`, строки `COPY gradle-9.7.1-bin.zip`, `ENV PATH`, `RUN unzip` +**Проблема:** Gradle 9.7.1 не существует. Файл `gradle-9.7.1-bin.zip` отсутствует в репозитории. +**Последствие:** Docker build упадёт на шаге `COPY gradle-9.7.1-bin.zip`. + +### C5. Dockerfile — конфликт с gradle-wrapper +**Файл:** `Dockerfile` +**Проблема:** Dockerfile использует ручную установку Gradle через unzip, но в репозитории есть `gradle/wrapper/gradle-wrapper.jar` и `gradlew`. Правильный подход — использовать gradlew wrapper. +**Последствие:** Docker build не найдёт `gradle-9.7.1-bin.zip` в build context. + +### C6. Dockerfile — HEALTHCHECK использует readiness вместо liveness +**Файл:** `Dockerfile`, HEALTHCHECK +**Проблема:** `HEALTHCHECK` проверяет `/actuator/health/readiness`, но readiness зависит от БД. Если БД недоступна, контейнер помечается как unhealthy, хотя процесс жив. +**Рекомендация:** Использовать `/actuator/health/liveness` для Docker HEALTHCHECK. + +### C7. docker-compose.yml — secrets-файлы отсутствуют +**Файл:** `docker-compose.yml`, секция `secrets` +**Проблема:** Все secrets ссылаются на файлы `./secrets/*.txt`, которых нет в репозитории. +**Последствие:** `docker compose up` упадёт с ошибкой «secrets failed: stat ... no such file». + +### C8. docker-compose.yml — deploy.replicas не работает в Docker Compose v2 +**Файл:** `docker-compose.yml`, service `api` +**Проблема:** `deploy.replicas: 2` работает только в Docker Swarm mode, не в обычном `docker compose up`. +**Последствие:** Будет запущена только 1 API-реплика. + +### C9. application.yaml — SPRING_PROFILES_ACTIVE: lab +**Файл:** `application.yaml`, docker-compose.yml (service api) +**Проблема:** `SPRING_PROFILES_ACTIVE: ${APP_ENV:-lab}` — профиль `lab` не определён ни в одном файле. +**Последствие:** Ничего критичного (неопределённый профиль не вызывает ошибку), но может запутать. + +--- + +## D. Некритические проблемы + +### D1. settings.gradle — неверное имя проекта +**Файл:** `settings.gradle` +**Текущее:** `rootProject.name = 'messenger-lab1'` +**Ожидаемое:** `messenger` или `messenger-lab4` + +### D2. requirements-dev.txt — отсутствует jsonschema +**Файл:** `requirements-dev.txt` +**Проблема:** `scripts/validate_openapi.py` импортирует `jsonschema`, но он не указан в `requirements-dev.txt`. +**Последствие:** `make validate` упадёт с `ModuleNotFoundError: No module named 'jsonschema'`. + +### D3. MinIO image — loose tag +**Файл:** `docker-compose.yml` +**Проблема:** `pgsty/silo:latest` и `pgsty/mc:latest` используют `:latest`. +**Рекомендация:** Зафиксировать конкретные версии. + +### D4. nginx.conf — короткий proxy_connect_timeout +**Файл:** `nginx.conf` +**Проблема:** `proxy_connect_timeout 1s` — очень короткий таймаут. При высокой нагрузке может вызывать преждевременный failover. +**Рекомендация:** Увеличить до 5s. + +### D5. OpenAPI Attachment status enum +**Файл:** `contracts/openapi.json` +**Проблема:** `Attachment.status` enum содержит только `["ready"]`, но реализация использует `UPLOADING`, `READY`, `BROKEN`. +**Последствие:** Валидация examples.json может не пройти. + +### D6. OpenAPI variant enum +**Файл:** `contracts/openapi.json` +**Проблема:** `variant` enum содержит только `["original"]`, но smoke.py для LAB>=5 запрашивает `?variant=thumbnail`. + +### D7. ShedLock — @SchedulerLock без @Scheduled на методе reconcile() +**Файл:** `AttachmentReconciler.java` +**Проблема:** `@SchedulerLock` находится на публичном методе `reconcile()`, который аннотирован `@Scheduled`. Это корректно, НО метод `reconcile()` может быть вызван напрямую из тестов, что обойдёт ShedLock. + +### D8. S3Properties — file-based secrets через env vars +**Файл:** `S3Properties.java` +**Проблема:** `resolvedAccessKey()` читает из файла по пути из env var `S3_ACCESS_KEY_ID_FILE`, но в docker-compose.yml эти env vars не передаются. +**Последствие:** Файловый fallback не сработает, ключи берутся из `s3.access-key` / `s3.secret-key`, которые пустые по умолчанию. + +### D9. application.yaml — session.store не используется +**Файл:** `docker-compose.yml`, env `SESSION_STORE: ${SESSION_STORE:-postgres}` +**Проблема:** Нигде в коде не используется свойство `session.store`. Redis используется для кэширования сессий, но не как «store of sessions». + +--- + +## E. Таблица конфигураций + +| Параметр | Где задаётся | Где используется | Фактическое значение | Проблема | +|---|---|---|---|---| +| SPRING_DATASOURCE_URL | application.yaml, docker-compose.yml | DataSource, HikariCP | `jdbc:postgresql://db:5432/messenger` | OK | +| SPRING_DATASOURCE_USERNAME | docker-compose.yml (secrets) | DataSource | читается из `/run/secrets/postgres_user` | OK | +| SPRING_DATASOURCE_PASSWORD | docker-compose.yml (secrets) | DataSource | читается из `/run/secrets/postgres_password` | OK | +| SPRING_FLYWAY_ENABLED | docker-compose.yml (migrate: true, api: false) | FlywayAutoConfiguration | migrate=true, api=false | OK | +| SPRING_PROFILES_ACTIVE | docker-compose.yml (api) | Spring Environment | `lab` (не определён) | D9 | +| SPRING_DATA_REDIS_HOST | application.yaml, docker-compose.yml | RedisConfig, LettuceConnectionFactory | `redis` | OK | +| S3_ENDPOINT | docker-compose.yml (api) | S3Properties, MinioClient | `http://minio:9000` | OK | +| S3_ACCESS_KEY_ID | docker-compose.yml (secrets) | S3Properties.resolvedAccessKey() | читается из секрета | D8 | +| S3_SECRET_ACCESS_KEY | docker-compose.yml (secrets) | S3Properties.resolvedSecretKey() | читается из секрета | D8 | +| SESSION_TTL_SECONDS | docker-compose.yml (api) | SessionProperties.ttl() | `400s` | OK | +| INSTANCE_ID | docker-compose.yml (api entrypoint) | InstanceProperties | `${HOSTNAME}` | OK | +| RECONCILER_ENABLED | docker-compose.yml (migrate: false) | ReconcilerProperties.enabled() | api: не задано (=true), migrate: false | OK | +| SPRING_TASK_SCHEDULING_ENABLED | docker-compose.yml (migrate) | Spring Boot scheduling | `false` | OK (для migrate) | +| SPRING_MAIN_WEB-APPLICATION-TYPE | docker-compose.yml (migrate entrypoint) | Spring Boot auto-config | `none` | OK (для migrate) | +| ShedLock table | V1__init.sql | ShedLockConfig | `shedlock` | OK | +| shedlock.lock_at_most_for | ShedLockConfig | JdbcTemplateLockProvider | `30m` | OK | +| reconciler.interval-ms | application.yaml | @Scheduled(fixedDelayString) | `3600000` (1 час) | OK | +| reconciler.lock-at-most-for | @SchedulerLock | ShedLock | `30m` | OK | +| reconciler.lock-at-least-for | @SchedulerLock | ShedLock | `1m` | OK | +| MAX_UPLOAD_BYTES | application.yaml | MultipartConfigElement | `10485760` (10 MiB) | OK | +| rate.rate-per-minute | application.yaml | Bucket4j | `600` | OK | + +--- + +## F. Scheduled / ShedLock / Reconciler — детальный анализ + +### Объявление @EnableScheduling +**Файл:** `MessengerLab1Application.java` +```java +@SpringBootApplication(...) +@EnableScheduling +public class MessengerLab1Application { ... } +``` +**Статус:** OK — аннотация на main-классе. + +### Объявление @EnableSchedulerLock +**Файл:** `ShedLockConfig.java` +```java +@Configuration +@EnableSchedulerLock(defaultLockAtMostFor = "30m") +public class ShedLockConfig { + @Bean + public LockProvider lockProvider(DataSource dataSource) { + return new JdbcTemplateLockProvider( + JdbcTemplateLockProvider.Configuration.builder() + .withJdbcTemplate(new JdbcTemplate(dataSource)) + .withTableName("shedlock") + .usingDbTime() + .build() + ); + } +} +``` +**Статус:** OK — используется `JdbcTemplateLockProvider` с таблицей `shedlock` и `usingDbTime()`. + +### Таблица shedlock +**Файл:** `V1__init.sql` +```sql +CREATE TABLE shedlock ( + name VARCHAR(64) NOT NULL PRIMARY KEY, + lock_until TIMESTAMP NOT NULL, + locked_at TIMESTAMP NOT NULL, + locked_by VARCHAR(255) NOT NULL +); +``` +**Статус:** OK — соответствует требованиям ShedLock 5.x. + +### Метод @Scheduled +**Файл:** `AttachmentReconciler.java` +```java +@Scheduled(fixedDelayString = "${reconciler.interval-ms:3600000}") +@SchedulerLock(name = "reconciler", + lockAtMostFor = "${reconciler.lock-at-most-for:30m}", + lockAtLeastFor = "${reconciler.lock-at-least-for:1m}") +public void reconcile() { + if (!reconcilerProps.enabled()) { + log.debug("Reconciler is disabled, skipping"); + return; + } + executeReconciliation(); +} +``` +**Статус:** OK — корректная комбинация `@Scheduled` + `@SchedulerLock`. + +### Условия управления +- **migrate контейнер:** `SPRING_TASK_SCHEDULING_ENABLED=false` + `RECONCILER_ENABLED=false` — планировщик отключён. OK. +- **api контейнер:** `RECONCILER_ENABLED` не задан → по умолчанию `true` (из `ReconcilerProperties`). OK. +- **dry-run:** `reconcilerProps.dryRun()` → если `true`, `executeReconciliation()` логирует и выходит без изменений. OK. + +### Координация нескольких реплик +ShedLock через таблицу `shedlock` гарантирует, что метод `reconcile()` выполнится только на одной реплике в любой момент времени. `lockAtMostFor=30m` предотвращает вечную блокировку при падении реплики. `lockAtLeastFor=1m` предотвращает повторное выполнение в течение 1 минуты. + +### Восстановление после падения +Если реплика падает во время выполнения reconciler'а, ShedLock автоматически снимёт блокировку по истечении `lockAtMostFor` (30 минут). Другая реплика начнёт выполнение. + +### Потенциальные проблемы +1. **Метод `reconcile()` публичный** — может быть вызван напрямую из тестов, обойдя ShedLock. Это не проблема для продакшена, но для тестов создаётся новый экземпляр `AttachmentReconciler` вручную (как видно из `AttachmentIntegrationTest.java`). +2. **`lockAtMostFor` в `@SchedulerLock` читается из свойства**, а не использует дефолт из `@EnableSchedulerLock`. Это корректно, но если свойство не задано, используется `30m`. OK. +3. **Нет отдельного bean для scheduling** — `@EnableScheduling` создаёт `ScheduledAnnotationBeanPostProcessor` автоматически. OK. + +--- + +## G. Docker и запуск + +### Правильный порядок запуска +1. Создать файлы secrets в `./secrets/` +2. `docker compose build` +3. `docker compose up -d db redis minio` +4. Дождаться healthcheck всех трёх сервисов +5. `docker compose up -d minio-init` (запускается автоматически через depends_on) +6. `docker compose up -d migrate` (запускается автоматически через depends_on) +7. Дождаться завершения migrate +8. `docker compose up -d api` (запускается автоматически через depends_on) +9. `docker compose up -d lb` (запускается автоматически через depends_on) + +### Выявленные проблемы +1. **Dockerfile не соберётся** из-за Gradle 9.7.1 (C4, C5) +2. **build.gradle не соберётся** из-за Spring Boot 4.1.1 и несуществующих артефактов (C1, C2) +3. **Секреты отсутствуют** — нужно создать файлы в `./secrets/` (C7) +4. **2 реплики API не запустятся** в обычном Compose — нужен `--scale api=2` или Swarm (C8) +5. **HEALTHCHECK использует readiness** вместо liveness (C6) + +--- + +## H. Матрица тестирования + +| Проверка | Статус | Причина | Что требуется | +|---|---|---|---| +| Health endpoint | ✅ Реализован | `HealthController` + `HealthService` | Сборка и запуск | +| Регистрация пользователей | ✅ Реализован | `UserController.registerUser()` | Сборка и запуск | +| Login / сессии | ✅ Реализован | `AuthController` + `SessionService` + Redis cache | Сборка и запуск | +| Чат CRUD | ✅ Реализован | `ChatController` + `ChatService` | Сборка и запуск | +| Отправка сообщений | ✅ Реализован | `MessageController` + `MessageService` | Сборка и запуск | +| Idempotency | ✅ Реализован | `IdempotencyService` + `IdempotencyRepository` (native query) | Сборка и запуск | +| Пагинация (cursor) | ✅ Реализован | `CursorCodec` + repository queries | Сборка и запуск | +| Ошибки доступа | ✅ Реализован | `GlobalExceptionHandler` + security config | Сборка и запуск | +| Загрузка файлов | ✅ Реализован | `AttachmentController` + `AttachmentService` + MinIO | Сборка и запуск | +| SHA-256 checksum | ✅ Реализован | `MessageDigest.getInstance("SHA-256")` в `AttachmentService.uploadToS3()` | Сборка и запуск | +| Logout / revoke | ✅ Реализован | `AuthController` + `SessionService.revoke()` | Сборка и запуск | +| Persistence (restart) | ⏳ Не проверено | Зависит от PostgreSQL volume | Запуск + тест | +| Failover (LB) | ⏳ Не проверено | Зависит от 2 API-реплик + nginx | Запуск + тест | +| Reconciler stale detection | ✅ Реализован | `AttachmentReconciler.reconcileStaleAttachments()` | Запуск + тест | +| Reconciler orphan cleanup | ✅ Реализован | `AttachmentReconciler.reconcileOrphanedS3Objects()` | Запуск + тест | +| Reconciler dry-run | ✅ Реализован | `ReconcilerProperties.dryRun()` | Запуск + тест | +| Reconciler verify READY | ✅ Реализован | `AttachmentReconciler.verifyReadyAttachments()` | Запуск + тест | +| Concurrent uploads | ✅ Реализован | `AttachmentIntegrationTest.shouldHandleConcurrentUploads()` | Запуск + тест | + +--- + +## I. План исправлений + +### Исправление 1: build.gradle — версии и имена зависимостей +**Файл:** `build.gradle` +**Текущая проблема:** Spring Boot 4.1.1 не существует, артефакты `spring-boot-starter-webmvc`, `spring-boot-starter-jackson` и др. не существуют. +**Предлагаемое изменение:** +```groovy +plugins { + id 'org.springframework.boot' version '3.3.5' // было 4.1.1 +} + +dependencies { + implementation 'org.springframework.boot:spring-boot-starter-web' // было webmvc + implementation 'org.springframework.boot:spring-boot-starter-jackson' // УДАЛИТЬ (часть web) + // ... + testImplementation 'org.springframework.boot:spring-boot-starter-test:3.3.5' // было 4.1.1 + // УДАЛИТЬ: spring-boot-starter-restclient-test, spring-boot-resttestclient + // УДАЛИТЬ: spring-boot-starter-actuator-test, spring-boot-starter-validation-test, spring-boot-starter-webmvc-test +} +``` +**Почему необходимо:** Без этого сборка невозможна. +**Функции, которые может затронуть:** Нет — это только исправление сборок. +**Как проверить:** `gradle bootJar --no-daemon` + +### Исправление 2: build.gradle — исправить netty dependency +**Файл:** `build.gradle` +**Текущая проблема:** `runtimeOnly 'io.netty:netty-transport-native-epoll::linux-x86_64'` (двойное двоеточие) +**Предлагаемое изменение:** +```groovy +runtimeOnly 'io.netty:netty-transport-native-epoll:4.1.115.Fallback:linux-x86_64' +``` +или удалить полностью (Netty уже включён как транзитивная зависимость Spring Boot). + +### Исправление 3: Dockerfile — использовать gradlew wrapper +**Файл:** `Dockerfile` +**Текущая проблема:** Ручная установка несуществующего Gradle 9.7.1. +**Предлагаемое изменение:** +```dockerfile +FROM eclipse-temurin:21-jdk-jammy AS builder +WORKDIR /build +COPY gradle gradle +COPY gradlew settings.gradle build.gradle ./ +RUN chmod +x gradlew +RUN ./gradlew dependencies --no-daemon +COPY contracts contracts +COPY src src +RUN ./gradlew bootJar -x test --no-daemon +``` +**Почему необходимо:** Dockerfile должен собираться с существующими инструментами. +**Функции, которые может затронуть:** Нет — это только Docker build. +**Как проверить:** `docker build -t messenger:test .` + +### Исправление 4: Dockerfile — HEALTHCHECK на liveness +**Файл:** `Dockerfile` +**Текущая проблема:** HEALTHCHECK проверяет readiness. +**Предлагаемое изменение:** +```dockerfile +HEALTHCHECK --interval=10s --timeout=3s --start-period=20s --retries=3 \ + CMD wget -qO- http://127.0.0.1:8080/actuator/health/liveness | grep -q UP || exit 1 +``` + +### Исправление 5: docker-compose.yml — scale для API +**Файл:** `docker-compose.yml` +**Текущая проблема:** `deploy.replicas: 2` не работает в Docker Compose v2. +**Предлагаемое изменение:** Удалить `deploy.replicas: 2` и запускать с `docker compose up --scale api=2 -d`. + +### Исправление 6: docker-compose.yml — HEALTHCHECK API +**Файл:** `docker-compose.yml`, service `api` +**Текущая проблема:** healthcheck проверяет readiness, но readiness зависит от БД. +**Предлагаемое изменение:** +```yaml +healthcheck: + test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:8080/health/live | grep -q 'ok'"] +``` + +### Исправление 7: Создать файлы secrets +**Файл:** `./secrets/*.txt` (7 файлов) +**Предлагаемое изменение:** Создать файлы: +``` +./secrets/postgres_db.txt → messenger +./secrets/postgres_user.txt → messenger +./secrets/postgres_password.txt → messenger_pass +./secrets/minio_root_user.txt → +./secrets/minio_root_password.txt → +./secrets/s3_access_key_id.txt → messenger_s3_user +./secrets/s3_secret_access_key.txt → messenger_s3_secret +``` + +### Исправление 8: requirements-dev.txt — добавить jsonschema +**Файл:** `requirements-dev.txt` +**Предлагаемое изменение:** +``` +openapi-spec-validator==0.7.2 +jsonschema==4.23.0 +``` + +### Исправление 9: settings.gradle — имя проекта +**Файл:** `settings.gradle` +**Предлагаемое изменение:** +```groovy +rootProject.name = 'messenger' +``` + +### Исправление 10: contracts/openapi.json — статус attachment +**Файл:** `contracts/openapi.json` +**Предлагаемое изменение:** Добавить все возможные статусы в enum: +```json +"status": { "type": "string", "enum": ["uploading", "ready", "broken"] } +``` + +--- + +## J. Итоговый вердикт + +### Вариант 1: Проект можно сохранить и точечно исправить ✅ + +**Обоснование:** +1. **Java-код написан качественно** — 92 Java-файла, чистая архитектура, корректные паттерны проектирования. +2. **Архитектура соответствует заданию** — 2 API-реплики, LB, PostgreSQL, Redis, MinIO, ShedLock, Flyway. +3. **Scheduled/Reconciler/ShedLock работают корректно** — правильная таблица, правильные провайдеры, правильные условия отключения для migrate. +4. **Аутентификация/авторизация реализована правильно** — Basic auth для login, Bearer token для API, Redis-кэш сессий. +5. **Критические проблемы — только в конфигурациях сборки** — build.gradle (версии Spring Boot), Dockerfile (Gradle version). Это исправляется точечно без переписывания кода. +6. **Docker-инфраструктура спроектирована правильно** — healthcheck-ы, depends_on, secrets, volumes, one-shot миграции. + +**Общее количество проблем:** 10 (4 критических, 6 некритических) +**Общее количество исправлений:** 10 (все точечные, без переписывания архитектуры) +**Оценка рисков:** Низкая — исправления касаются только конфигурационных файлов. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 1bec547..ba7e26e 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,18 +1,10 @@ ## Лабораторная 4 -Автор / группа: TODO -Целевая оценка (3/4/5): TODO -Трек 5, если заявлен: TODO +Автор / группа: Иванов Данил БВТ2301 +Целевая оценка: 5, трек 2 — согласованность двух хранилищ. -Что реализовано и что перенесено: TODO -Команды запуска из чистого clone: TODO -Результаты проверок и ссылка на REPORT.md: TODO -Ограничения / известные ошибки: TODO -Источники и использование ИИ: TODO +Исправлены контракт вложений/сообщений, ограничение входного потока, проверка изображения/MIME/filename, скачивание через API и отзыв сессий между репликами. Reconciler выполняет реальный dry-run и повторяемое исправление orphan/missing; PostgreSQL shared/exclusive advisory lock защищает окно S3 PUT → COMMIT на нескольких API. -- [ ] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами. -- [ ] Контракт и публичные тесты не ослаблены. -- [ ] make check и make test выполнены; приложен реальный вывод. -- [ ] Сценарии отказов/безопасности своего уровня воспроизведены. -- [ ] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens. -- [ ] README задания сохранён, REPORT.md содержит инструкцию именно моего решения. +Запуск, транзакционные границы, команды, результаты и ограничения: [REPORT.md](../REPORT.md). Методики: [tests/README.md](../tests/README.md). Доказательства: [evidence](../evidence/). + +Публичные tests и контракт не менялись. Проверки не заменяют HA БД/S3 и большого нагрузочного исследования. PR не опубликован; результаты относятся к рабочему дереву. Использована помощь Codex при аудите, исправлениях и проверках. diff --git a/REPORT.md b/REPORT.md index 8f61ac2..278284c 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,139 +1,91 @@ # Отчёт — лабораторная 4 -- Автор, группа: [ВСТАВИТЬ] -- Целевая оценка: 5 (Track 2 — Согласованность двух хранилищ) -- Commit/PR: [ВСТАВИТЬ] -- Использованные источники: Spring Boot 4.1, MinIO 8.5, PostgreSQL 16, Flyway, ShedLock 5.14, MinIO Java Client 9.0.3 +- Автор, группа: Иванов Данил, БВТ2301 (из шаблона PR проекта). +- Целевая оценка: **5, трек 2 — согласованность двух хранилищ**. +- Базовый commit: `057143dd915f0fe75b72e253db103800940eeb38`. Проверено изменённое рабочее дерево; PR не опубликован. +- Дата: 01.11.2026 (Europe/Moscow). Помощь Codex: аудит, исправления, тесты, отчёт. Библиотеки: [build.gradle](build.gradle). Источники: [README](README.md), [контракт](contracts/README.md), [OpenAPI](contracts/openapi.json). Публичные проверки и контракт не изменялись. ## Запуск из чистого clone -```sh -# 1. Переименовать compose.example.yaml в compose.yaml и заполнить .env -cp compose.example.yaml docker-compose.yml -cp .env.example .env # и заполнить переменные +Docker Desktop с Linux containers/Compose, Python 3.9+, PowerShell 7. Проверены Temurin Java 21.0.13, PostgreSQL 16.15, Spring Boot 4.1.1, nginx 1.27-alpine, Redis 7.4-alpine. S3: сборка Silo на базе MinIO RELEASE.2026-10-26T00-00-00Z, закреплённая digest (pgsty/silo). Для локальных Java-тестов JDK 21; Docker собирает приложение самостоятельно. -# 2. Запустить инфраструктуру -docker compose --env-file .env up -d --build - -# 3. Дождаться готовности -sleep 30 -curl -f http://localhost:8080/health/ready - -# 4. Запустить smoke-тесты -make test - -# 5. Остановка -docker compose --env-file .env down -v # -v удаляет все данные +```powershell +pwsh -File scripts/bootstrap.ps1 +docker compose --env-file .env config --quiet +docker compose up -d --build +# Дождаться healthy обеих api и HTTP 200 /health/ready. +$env:BASE_URL='http://127.0.0.1:8080' +py tests/smoke.py +py tests/extended.py +py tests/faults.py +py tests/resilience.py --mode persistence --action-script scripts/restart.cmd +py tests/resilience.py --mode failover --action-script scripts/stop-one.cmd +docker compose up -d api ``` -**Версии образов:** -- `eclipse-temurin:21-jdk` (Java 21) -- `postgres:16-alpine` (PostgreSQL 16) -- `minio/minio:latest` (MinIO) -- `redis:7-alpine` (Redis) -- `nginx:alpine` (Nginx LB) +Bootstrap создаёт локальные secrets/.env, не печатает значения и не перезаписывает существующие файлы. Они исключены из Git. Flyway выполняется при старте API, собственная блокировка миграций координирует реплики. Профиль migration не нужен. MinIO bootstrap идемпотентно создаёт приватный bucket, отдельного app user и bucket policy; ошибки завершают bootstrap. S3 и mc закреплены digest. Консоль отключена, порты БД/S3/API наружу не опубликованы, LB — loopback:8080. -**Prerequisites:** Docker 24+, Docker Compose v2, Java 21, Python 3.9+, Gradle 9.3 - -**Секреты:** создаются отдельно через `.env` или `/run/secrets/`. В репозитории не хранятся. - -**Безопасная остановка:** `docker compose --env-file .env stop` — сохраняет данные. -**Удаление данных:** `docker compose --env-file .env down -v` — удаляет volumes. +Остановка с сохранением данных: `docker compose down`. Намеренное удаление учебных данных: `docker compose down -v`, не часть persistence. Linux hooks `.sh` требуют chmod +x. Методики всех проверок — [tests/README.md](tests/README.md). ## Архитектура и решения -**Схема:** -``` -Client → Nginx (LB) → API × 2 → PostgreSQL (метаданные) - → Redis (сессии) - → MinIO (байты) -``` +Клиент → nginx → API ×2 → PostgreSQL (ACL, пользователи, сессии, сообщения, идемпотентность, метаданные) + приватный MinIO (оригиналы). Redis не источник истины. Из lab3 сохранены Argon2, Bearer-сессии, keyset-пагинация и атомарная идемпотентность PostgreSQL. Сессии читаются из БД: исключена гонка заполнения кэша после logout. -**Источник истины:** PostgreSQL для метад attachments, MinIO для бинарных данных. +История сортируется по (created_at, id), cursor привязан к чату; snapshot между HTTP-страницами нет. Время назначает JVM каждой реплики (UTC, миллисекунды), UUID разрешает совпадение времени. Подтверждённые записи читаются из основной БД. При конкурентной вставке с временем раньше уже пройденного cursor запись может не попасть в текущий обход; необходима синхронизация часов хостов. Для неизменной истории повторов нет. -**Транзакционные границы:** -- `upload()` — S3-вызов вне транзакции БД (не держит соединение) -- `saveAttachmentReady()` — `REQUIRES_NEW`, атомарный INSERT со статусом READY -- При падении БД: `compensateS3Upload()` удаляет файл из S3 (best-effort) +Upload читает максимум 10 MiB + 1 byte. MIME сверяется с содержимым, лимит 40 млн пикселей проверяется до декодирования. Filename с управляющими символами/разделителями отклоняется; S3 key — UUID. Одна загрузка на реплику, ожидание слота до 30s; заданы heap/container limits и S3 timeouts. Оригиналы не сохраняются локально в API. Контроллеры делегируют работу сервисам. -**Что перенесено из lab3:** регистрация, сессии, чаты, сообщения, идемпотентность, rate limiting. +Upload-транзакция держит shared PostgreSQL advisory lock 741204. Reconciler берёт exclusive try-lock: либо пропускает цикл, либо видит завершённые загрузки. Это закрывает окно PUT → COMMIT между репликами, включая загрузки дольше grace. Ключи не переиспользуются. При падении после PUT/до COMMIT остаётся orphan, сверка удалит его после grace. Ошибка сохранения может компенсироваться сразу, ошибка COMMIT — сверкой. 201 возвращается после завершения транзакции. -**Изменения в lab4:** добавлен MinIO S3, Attachment-сущность, upload/download endpoints, Reconciler. +Reconciler выполняет S3 list и индексированные exists(object_key), затем постранично проверяет метаданные по id. READY → BROKEN не сдвигает страницы. Старый orphan удаляется, достоверный NoSuchKey переводит старую READY-запись в BROKEN. Сетевая ошибка не считается отсутствием. Метаданные сохраняются для диагностики; BROKEN возвращает 409 без невалидного ready DTO. Существующие используемые объекты не удаляются. Dry-run выполняет тот же обход, пишет решения и ничего не меняет. Публичная транзакционная граница Spring охватывает цикл. + +Настройки: [Compose](docker-compose.yml), [application.yaml](src/main/resources/application.yaml). Используются SESSION_TTL_SECONDS, RATE_PER_MINUTE, MAX_MESSAGES_PER_CHAT, S3_*, MAX_UPLOAD_BYTES/MAX_IMAGE_PIXELS, RECONCILER_*. Credentials поступают из secrets. SESSION_STORE не переключает реализацию: сессии всегда PostgreSQL. Readiness требует БД и S3, Redis не критичен. LB не повторяет произвольные POST после неопределённого результата. ## Доказательства критериев -Все доказательства хранятся в `evidence/`. Формат: команда и её вывод, без дополнительного текста. - -### Как запустить доказательства - -```sh -# Компиляция -./gradlew compileJava compileTestJava --no-daemon - -# Отдельный тест -cd "D:/Лабы/mtusi2026lab4"; ./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldRejectOversizedFile" --no-daemon - -# Docker Compose -docker compose config --quiet -``` +Правила: [evidence/README.md](evidence/README.md). Токены и credentials не сохраняются. Итоговое состояние: [две healthy API и readiness](evidence/final-state.txt); точные исходники: [SHA-256 manifest](evidence/source-sha256.json); сводка JUnit: [unit-summary](evidence/unit-summary.json). Команды/методики: [tests/README.md](tests/README.md). | Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства | | --- | --- | --- | --- | -| Компиляция | `./gradlew compileJava compileTestJava --no-daemon` | BUILD SUCCESSFUL | [evidence/01-compile-java.txt](evidence/01-compile-java.txt) | -| Docker Compose | `docker compose config --quiet` | Конфигурация валидна | [evidence/02-docker-compose-config.txt](evidence/02-docker-compose-config.txt) | -| Grade 4: oversized | `shouldRejectOversizedFile` | Файл > 10 MiB отклоняется | [evidence/03-grade4-oversized.txt](evidence/03-grade4-oversized.txt) | -| Grade 4: corrupted | `shouldRejectCorruptedImage` | Повреждённое изображение отклоняется | [evidence/04-grade4-corrupted.txt](evidence/04-grade4-corrupted.txt) | -| Grade 4: MIME | `shouldCheckMimeByContent` | MIME по содержимому, не по имени | [evidence/05-grade4-mime.txt](evidence/05-grade4-mime.txt) | -| Grade 4: foreign | `shouldRejectForeignAttachmentAccess` | Чужой пользователь не получает доступ | [evidence/06-grade4-foreign.txt](evidence/06-grade4-foreign.txt) | -| Grade 4: concurrent | `shouldHandleConcurrentUploads` | 10 concurrent, 0 failures, 380 MiB RAM | [evidence/12-grade4-concurrent-memory.txt](evidence/12-grade4-concurrent-memory.txt) | -| Track 2: stale repair | `shouldDetectAndRepairStaleAttachment` | UPLOADING → BROKEN за 15 мин | [evidence/07-track2-stale-repair.txt](evidence/07-track2-stale-repair.txt) | -| Track 2: dry-run | `shouldNotModifyInDryRun` | Ничего не меняет | [evidence/08-track2-dry-run.txt](evidence/08-track2-dry-run.txt) | -| Track 2: fresh safe | `shouldNotDeleteFreshAttachments` | READY вложения не удаляются | [evidence/09-track2-fresh-not-deleted.txt](evidence/09-track2-fresh-not-deleted.txt) | -| Track 2: traversal | `shouldMeasureTraversalCost` | 0.90ms на вложение (50 шт) | [evidence/10-track2-traversal-cost.txt](evidence/10-track2-traversal-cost.txt) | -| Track 2: concurrent | `shouldHandleConcurrentUploads` | 10 concurrent, 0 failures, 234ms | [evidence/11-track2-concurrent.txt](evidence/11-track2-concurrent.txt) | +| Сборка, unit | docker compose build api; gradlew.bat test | Java build, fixture/filename/cursor | [build](evidence/build.txt), [unit](evidence/unit.txt) | +| Smoke, ACL, checksum, файл в сообщении | py tests/smoke.py | Сквозной контрактный сценарий | [smoke](evidence/smoke.txt) | +| Размер, corrupt, MIME, filename | extended.py | 413/415/400, без пользовательских 500 | [extended](evidence/extended.txt) | +| Чужие/повторные ID, другой чат, пустой текст с файлом | extended.py | 404/400; допустимое сообщение с файлом | [extended](evidence/extended.txt) | +| Конкурентная идемпотентность | extended.py | 24 повтора, 2 instance ID, одно сообщение | [extended](evidence/extended.txt) | +| Failover с оригиналом | resilience.py --mode failover | Реальная остановка обслужившей API | [failover](evidence/failover.txt), [до](evidence/stop-one-before.txt), [после](evidence/stop-one-after.txt), [hook](scripts/stop-one.ps1) | +| Restart всех зависимостей | resilience.py --mode persistence | Прежние сессия, сообщения, оригинал | [persistence](evidence/persistence.txt), [до](evidence/restart-before.txt), [после](evidence/restart-after.txt) | +| Private bucket / app user | faults.py и bootstrap | Direct GET существующего ключа → 403 | [faults](evidence/faults.txt), [policy](policy.json), [Compose](docker-compose.yml) | +| Отказ S3 и БД после PUT | faults.py | S3 → 503; DB reject без 201 и фиксации метаданных | [faults](evidence/faults.txt) | +| Конкурентные upload / RAM | extended.py | 20 × 8 674 413 bytes, concurrency=10, без ошибок | [extended](evidence/extended.txt) | +| Трек 2: orphan/missing, dry-run/apply/repeat, свежий объект | track2.py | Dry-run без изменений; orphan удалён, missing → BROKEN, свежий сохранён, повтор идемпотентен | [track2](evidence/track2.txt) | +| Окно PUT → COMMIT и прерывание клиента | locking.py; aborted.py | Исключительная блокировка недоступна до COMMIT; частичный запрос не создаёт данные | [locking](evidence/locking.txt), [aborted](evidence/aborted.txt) | +| Стоимость обхода | scheduled reconcile | elapsedMs цикла | [apply](evidence/reconciler-apply-log.txt), [dry-run](evidence/reconciler-dry-run-log.txt), [объём/лимиты](evidence/environment.json) | ## Эксперименты и отказы -### Эксперимент 1: ≥10 одновременных загрузок +Гипотеза: общая транзакционная блокировка защищает активный PUT, повторяемая сверка устраняет рассогласования без удаления существующих используемых объектов. Инъекции: объект без метаданных; удаление bytes READY-записи; временный DB trigger отклоняет INSERT после PUT. Методики: [track2.py](tests/track2.py), [faults.py](tests/faults.py). Ускоренный опыт: interval=3s, grace=1min; defaults — 1h/15min. Перед apply выдерживается grace, свежий объект создаётся отдельно. Оба scheduler координируются через БД. -**Гипотеза:** При 10 одновременных загрузках пик RAM API не превысит 512 MiB благодаря `REQUIRES_NEW` и отсутствию удержания соединения БД на время S3. +Docker: 8 CPU, 8 212 668 416 bytes RAM. API ×2: CPU=1, RAM=512MiB, Xmx320m каждая; БД CPU=1/RAM=1GiB. Closed-loop: 20 PNG 1700×1700 со случайными RGB, 10 клиентов, 12.547s, ошибок 0. Максимум наблюдавшихся RSS samples: 454.1/454.8MiB. До настройки Xmx384m дал наблюдавшийся пик 502.3MiB ([предыдущий профиль](evidence/upload-before-tuning.json)); поэтому heap уменьшен до 320MiB и нагрузка повторена. Итог: [upload-summary.json](evidence/upload-summary.json). Дискретный docker stats может пропустить краткий пик. CPU во времени, p50/p95/p99 и максимальный throughput upload не измерялись: опыт проверяет RAM. Один локальный прогон без длительного прогрева. -**Настройки:** -- Java: `-XX:MaxRAMPercentage=75` -- HikariCP: `maximum-pool-size: 20` -- S3: MinIO локальный +На уровне операций: S3 list, один индексированный exists на объект, HEAD старых метаданных, страницы БД. SQL OFFSET и count страниц добавляют стоимость; линейную асимптотику для большого набора не заявляем. Длительности записанных циклов apply (ms): [2709, 682, 552, 1896, 795, 676]. Это небольшой учебный набор. Объёмы — environment.json. Экстраполяция на большой bucket не проверена. -**Результат:** -| Конкурентность | Успешных | Ошибок | p50 (ms) | p95 (ms) | p99 (ms) | Пик RAM (MiB) | -|---|---|---|---|---|---|---| -| 1 | 10 | 0 | 45 | 60 | 80 | 120 | -| 5 | 50 | 0 | 52 | 78 | 120 | 280 | -| 10 | 100 | 0 | 58 | 95 | 150 | 380 | -| 20 | 200 | 0 | 65 | 110 | 180 | 480 | - -### Эксперимент 2: Отказ S3 во время upload - -**Гипотеза:** При недоступности MinIO во время upload файл не остаётся в S3 (загрузка не началась), БД чиста. - -**Что наблюдали:** `StorageUnavailableException` → 503, в S3 пусто, в БД пусто. - -### Эксперимент 3: Отказ БД после S3 - -**Гипотеза:** При падении PostgreSQL после успешного S3 PUT, `compensateS3Upload()` удалит файл из S3. - -**Что наблюдали:** `DataAccessException` → `compensateS3Upload()` → объект удалён из S3, 500 клиенту. +| Воздействие | Что ожидаем | Что наблюдали | Восстановление / потеря данных | +| --- | --- | --- | --- | +| API stop | другая API обслуживает данные | failover stdout | Метрики восстановления в stdout | +| Restart DB/Redis/S3/API/LB | данные и сессия сохраняются | persistence, StartedAt | Volumes сохранены | +| S3 stop | 503 без ложного 201 | faults.py | После start S3, старые объекты сохранены | +| DB reject после PUT | rollback / orphan / compensation | нет 201 и новых метаданных | Сверка после grace | +| Orphan/missing | безопасный ремонт и повтор | track2.py | Удалённые инъекцией bytes не восстанавливаются | ## Ограничения и следующие шаги -**Выдерживаются:** -- Отказ S3 во время upload (загрузка не началась) -- Отказ БД после S3 (компенсация) -- Мульти-репликация с ShedLock +Локальный стенд без HA для БД/S3/LB/host, без распределённой транзакции БД/S3. Внешний оператор может удалить bytes; сверка обнаруживает потерю, но не восстанавливает их. Блокировки координируют приложение, не сторонние S3 clients. При постоянных загрузках цикл может пропускаться; срок ремонта зависит от свободного окна/grace/интервала. Большой обход держит exclusive lock и задерживает upload. Следующие шаги: per-key fencing/leases, keyset-обход и ограничение persistence context; они пока не реализованы. Ссылка на PR будет доступна после публикации. -**Не выдерживаются:** -- Отказ MinIO во время уже начавшейся загрузки (файл может остаться в S3) -- Падение Redis (сессии теряются, но восстанавливаются через login) -**Учебное упрощение:** нет реального S3 pre-signed upload, нет image processing (требуется lab5). +## Повторная проверка замечаний — 06.11.2026 -**Улучшение:** добавить S3 lifecycle policies для автоматической очистки орфанных объектов. +Проверены все пять замечаний к commit 057143dd на текущем изменённом рабочем дереве. Новые изменения production-кода для этой проверки не потребовались. [Тест](tests/review_regressions.py), [stdout](evidence/review-regressions-2026-11-06.txt), [контекст, команда и SHA-256](evidence/review-regressions-2026-11-06.json). Требуется Pillow для синтетического JPEG. + +Результаты: 10 PNG/JPEG upload → 201, metadata → 200, download идентичен, ровно 10 новых метаданных, обе API-реплики. Message без файлов содержит attachment_ids=[], с файлом и пустым текстом → 201; чужой чат/неизвестный ID/посторонний → 404. Десять sample.png[:33] → 415, filename=../x.png → 400; БД и S3 до/после отклонённых загрузок не изменились. Валидный PNG ровно 1770843 bytes → 201 через nginx, checksum скачанного оригинала совпадает. + +Windows зарезервировала TCP 8057–8156, поэтому host:8080 недоступен для bind Docker. Проверка выполнена через тот же LB на 127.0.0.1:18080: [compose.review.yaml](compose.review.yaml) меняет только публикацию порта. Запуск: docker compose -f docker-compose.yml -f compose.review.yaml up -d; затем BASE_URL=http://127.0.0.1:18080 py tests/review_regressions.py. Для PowerShell задавать BASE_URL через $env:BASE_URL. Стенд оставлен на этом порту. Исправления рабочего дерева не меняют содержимое удалённого commit 057143dd. diff --git a/compose.review.yaml b/compose.review.yaml new file mode 100644 index 0000000..2755ebf --- /dev/null +++ b/compose.review.yaml @@ -0,0 +1,4 @@ +services: + lb: + ports: !override + - "127.0.0.1:18080:80" diff --git a/docker-compose.yml b/docker-compose.yml index 656d578..41c09e2 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -15,7 +15,8 @@ services: networks: - messenger-net healthcheck: - test: ["CMD-SHELL", "pg_isready -U $$(cat /run/secrets/postgres_user) -d $$(cat /run/secrets/postgres_db)"] + # Добавлена очистка от \n и \r для надежности на Windows + test: ["CMD-SHELL", "pg_isready -U $$(cat /run/secrets/postgres_user | tr -d '\\n\\r') -d $$(cat /run/secrets/postgres_db | tr -d '\\n\\r')"] interval: 5s timeout: 5s retries: 10 @@ -25,18 +26,11 @@ services: limits: cpus: '1.0' memory: 1G - reservations: - cpus: '0.25' - memory: 256M redis: image: redis:7.4-alpine restart: unless-stopped - command: > - redis-server - --appendonly yes - --appendfsync everysec - --save "" + command: redis-server --appendonly yes --appendfsync everysec --save "" volumes: - redisdata:/data networks: @@ -46,45 +40,39 @@ services: interval: 5s timeout: 3s retries: 10 - start_period: 5s - deploy: - resources: - limits: - cpus: '0.5' - memory: 256M - reservations: - cpus: '0.1' - memory: 64M migrate: + profiles: [migration] build: context: . dockerfile: Dockerfile - entrypoint: ["sh", "-c"] - command: - - "exec java -Dfile.encoding=UTF-8 -Dspring.main.web-application-type=none -jar app.jar" + entrypoint: + - "sh" + - "-c" + - | + export SPRING_DATASOURCE_USERNAME="$$(cat /run/secrets/postgres_user)"; + export SPRING_DATASOURCE_PASSWORD="$$(cat /run/secrets/postgres_password)"; + export S3_ACCESS_KEY_ID="$$(cat /run/secrets/s3_access_key_id)"; + export S3_SECRET_ACCESS_KEY="$$(cat /run/secrets/s3_secret_access_key)"; + exec java -Dfile.encoding=UTF-8 -Dspring.main.web-application-type=none -Dspring.main.keep-alive=false -jar app.jar environment: SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/messenger - SPRING_DATASOURCE_USERNAME_FILE: /run/secrets/postgres_user - SPRING_DATASOURCE_PASSWORD_FILE: /run/secrets/postgres_password SPRING_FLYWAY_ENABLED: "true" + RECONCILER_ENABLED: "false" + SPRING_TASK_SCHEDULING_ENABLED: "false" # Отключает фоновые задачи, чтобы контейнер успешно завершился secrets: - postgres_user - postgres_password + - s3_access_key_id + - s3_secret_access_key depends_on: db: condition: service_healthy restart: "no" networks: - messenger-net - deploy: - resources: - limits: - cpus: '0.5' - memory: 512M - reservations: - cpus: '0.1' - memory: 128M + healthcheck: + disable: true api: stop_grace_period: 40s @@ -93,29 +81,44 @@ services: dockerfile: Dockerfile restart: unless-stopped depends_on: - migrate: - condition: service_completed_successfully + db: + condition: service_healthy redis: condition: service_healthy + minio-init: + condition: service_completed_successfully entrypoint: - "sh" - "-c" - - "export INSTANCE_ID=\"$$(cat /etc/hostname)\"; exec java -Dfile.encoding=UTF-8 $$JAVA_OPTS -jar app.jar" + - | + export INSTANCE_ID="$$(cat /etc/hostname)"; + export SPRING_DATASOURCE_USERNAME="$$(cat /run/secrets/postgres_user)"; + export SPRING_DATASOURCE_PASSWORD="$$(cat /run/secrets/postgres_password)"; + export S3_ACCESS_KEY_ID="$$(cat /run/secrets/s3_access_key_id)"; + export S3_SECRET_ACCESS_KEY="$$(cat /run/secrets/s3_secret_access_key)"; + exec java -Dfile.encoding=UTF-8 $$JAVA_OPTS -jar app.jar environment: + RECONCILER_INTERVAL_MS: ${RECONCILER_INTERVAL_MS:-3600000} + RECONCILER_DRY_RUN: ${RECONCILER_DRY_RUN:-false} + RECONCILER_GRACE_PERIOD_MINUTES: ${RECONCILER_GRACE_PERIOD_MINUTES:-15} + JAVA_OPTS: -Xmx320m -Xss512k RATE_PER_MINUTE: ${RATE_PER_MINUTE:-100000} SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/messenger - SPRING_DATASOURCE_USERNAME_FILE: /run/secrets/postgres_user - SPRING_DATASOURCE_PASSWORD_FILE: /run/secrets/postgres_password SPRING_PROFILES_ACTIVE: ${APP_ENV:-lab} - SPRING_FLYWAY_ENABLED: "false" + SPRING_FLYWAY_ENABLED: "true" SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-400} SESSION_STORE: ${SESSION_STORE:-postgres} SPRING_DATA_REDIS_HOST: redis SPRING_DATA_REDIS_PORT: 6379 MAX_MESSAGES_PER_CHAT: ${MAX_MESSAGES_PER_CHAT:-10000} + S3_ENDPOINT: http://minio:9000 + S3_BUCKET: messenger-attachments + S3_REGION: us-east-1 secrets: - postgres_user - postgres_password + - s3_access_key_id + - s3_secret_access_key networks: - messenger-net deploy: @@ -124,9 +127,6 @@ services: limits: cpus: '1.0' memory: 512M - reservations: - cpus: '0.25' - memory: 256M healthcheck: test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:8080/health/ready | grep -q '\"status\":\"ok\"'"] interval: 5s @@ -146,17 +146,9 @@ services: condition: service_healthy networks: - messenger-net - deploy: - resources: - limits: - cpus: '0.5' - memory: 128M - reservations: - cpus: '0.1' - memory: 32M minio: - image: quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z + image: pgsty/silo@sha256:7bb2aff6b89efe1cb1303c10fc21d579447434ed06649ac9ec05a5e2dbf30ad7 command: server /data --console-address ":9001" environment: MINIO_ROOT_USER_FILE: /run/secrets/minio_root_user @@ -177,12 +169,12 @@ services: start_period: 10s minio-init: - image: quay.io/minio/mc:RELEASE.2025-08-13T08-35-41Z + image: pgsty/mc@sha256:ce62c9053c69ce12a527a322345b5ad9f44b5f8a7b8189f82e516f2e3f30f1d8 depends_on: minio: condition: service_healthy entrypoint: > - /bin/sh -c " + /bin/sh -ec " export MINIO_ROOT_USER=$$(cat /run/secrets/minio_root_user); export MINIO_ROOT_PASSWORD=$$(cat /run/secrets/minio_root_password); export S3_ACCESS_KEY_ID=$$(cat /run/secrets/s3_access_key_id); @@ -190,12 +182,12 @@ services: until mc alias set local http://minio:9000 $$MINIO_ROOT_USER $$MINIO_ROOT_PASSWORD; do echo 'Waiting for MinIO...' && sleep 2; done; - mc mb --ignore-existing local/your-bucket-name; - mc anonymous set none local/your-bucket-name; + mc mb --ignore-existing local/messenger-attachments; + mc anonymous set none local/messenger-attachments; mc admin user add local $$S3_ACCESS_KEY_ID $$S3_SECRET_ACCESS_KEY; mc admin policy create local app-uploads-policy /tmp/policy.json; mc admin policy attach local app-uploads-policy --user $$S3_ACCESS_KEY_ID; - echo 'MinIO initialization completed'; + echo 'MinIO initialization completed (bucket: messenger-attachments)'; " volumes: - ./policy.json:/tmp/policy.json:ro @@ -231,4 +223,4 @@ secrets: s3_access_key_id: file: ./secrets/s3_access_key_id.txt s3_secret_access_key: - file: ./secrets/s3_secret_access_key.txt \ No newline at end of file + file: ./secrets/s3_secret_access_key.txt diff --git a/evidence/01-compile-java.txt b/evidence/01-compile-java.txt deleted file mode 100644 index fda3a0c..0000000 --- a/evidence/01-compile-java.txt +++ /dev/null @@ -1,14 +0,0 @@ -# Команда -./gradlew compileJava compileTestJava --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE - -BUILD SUCCESSFUL in 10s -3 actionable tasks: 3 up-to-date diff --git a/evidence/02-docker-compose-config.txt b/evidence/02-docker-compose-config.txt deleted file mode 100644 index 191b022..0000000 --- a/evidence/02-docker-compose-config.txt +++ /dev/null @@ -1,5 +0,0 @@ -# Команда -docker compose config --quiet - -# Вывод -(пусто — конфигурация валидна) diff --git a/evidence/03-grade4-oversized.txt b/evidence/03-grade4-oversized.txt deleted file mode 100644 index 144db4f..0000000 --- a/evidence/03-grade4-oversized.txt +++ /dev/null @@ -1,20 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldRejectOversizedFile" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Grade 4: Reject file exceeding max size (10 MiB) PASSED - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/04-grade4-corrupted.txt b/evidence/04-grade4-corrupted.txt deleted file mode 100644 index 2b37141..0000000 --- a/evidence/04-grade4-corrupted.txt +++ /dev/null @@ -1,20 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldRejectCorruptedImage" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Grade 4: Reject corrupted image (valid header, corrupted body) PASSED - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/05-grade4-mime.txt b/evidence/05-grade4-mime.txt deleted file mode 100644 index 732aaa0..0000000 --- a/evidence/05-grade4-mime.txt +++ /dev/null @@ -1,20 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldCheckMimeByContent" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Grade 4: MIME checked by content not filename PASSED - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/06-grade4-foreign.txt b/evidence/06-grade4-foreign.txt deleted file mode 100644 index b08745b..0000000 --- a/evidence/06-grade4-foreign.txt +++ /dev/null @@ -1,20 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldRejectForeignAttachmentAccess" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Grade 4: Reject foreign attachment access (user not in chat) PASSED - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/07-track2-stale-repair.txt b/evidence/07-track2-stale-repair.txt deleted file mode 100644 index e845a17..0000000 --- a/evidence/07-track2-stale-repair.txt +++ /dev/null @@ -1,20 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldDetectAndRepairStaleAttachment" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Track 2: Reconciler detects and repairs stale UPLOADING attachment PASSED - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/08-track2-dry-run.txt b/evidence/08-track2-dry-run.txt deleted file mode 100644 index 2ba4b2c..0000000 --- a/evidence/08-track2-dry-run.txt +++ /dev/null @@ -1,20 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldNotModifyInDryRun" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Track 2: Reconciler dry-run does not modify anything PASSED - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/09-track2-fresh-not-deleted.txt b/evidence/09-track2-fresh-not-deleted.txt deleted file mode 100644 index fb2d119..0000000 --- a/evidence/09-track2-fresh-not-deleted.txt +++ /dev/null @@ -1,20 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldNotDeleteFreshAttachments" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Track 2: Reconciler does not delete fresh attachments PASSED - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/10-track2-traversal-cost.txt b/evidence/10-track2-traversal-cost.txt deleted file mode 100644 index a7e3514..0000000 --- a/evidence/10-track2-traversal-cost.txt +++ /dev/null @@ -1,21 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldMeasureTraversalCost" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Track 2: Measure reconciler traversal cost PASSED -2026-10-30T12:00:00.000Z INFO [reconciler] ru.mtusi.messenger.service.AttachmentReconciler - Reconciler traversal cost: 45ms for 50 attachments (0.90ms per attachment) - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/11-track2-concurrent.txt b/evidence/11-track2-concurrent.txt deleted file mode 100644 index 4e05608..0000000 --- a/evidence/11-track2-concurrent.txt +++ /dev/null @@ -1,21 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldHandleConcurrentUploads" --no-daemon - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Track 2: Concurrent uploads do not corrupt state PASSED -2026-10-30T12:00:00.000Z INFO [reconciler] ru.mtusi.messenger.service.AttachmentIntegrationTest - Concurrent uploads: 10 success, 0 failures, 234ms - -BUILD SUCCESSFUL in 14s -4 actionable tasks: 1 executed, 3 up-to-date diff --git a/evidence/12-grade4-concurrent-memory.txt b/evidence/12-grade4-concurrent-memory.txt deleted file mode 100644 index b63f473..0000000 --- a/evidence/12-grade4-concurrent-memory.txt +++ /dev/null @@ -1,30 +0,0 @@ -# Команда -./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldHandleConcurrentUploads" --no-daemon - -# Настройки -Java: -XX:MaxRAMPercentage=75 -HikariCP: maximum-pool-size: 20 -S3: MinIO локальный -Concurrency: 10 - -# Вывод -To honour the JVM settings for this build a single-use Daemon process will be forked. -Daemon will be stopped at the end of the build - -> Task :compileJava UP-TO-DATE -> Task :processResources UP-TO-DATE -> Task :classes UP-TO-DATE -> Task :compileTestJava UP-TO-DATE -> Task :processTestResources NO-SOURCE -> Task :testClasses UP-TO-DATE - -> Task :test - -AttachmentIntegrationTest > Track 2: Concurrent uploads do not corrupt state PASSED -2026-10-30T12:00:00.000Z INFO [reconciler] ru.mtusi.messenger.service.AttachmentIntegrationTest - Concurrent uploads: 10 success, 0 failures, 234ms - -# Наблюдаемый результат -- 10 одновременных загрузок: 10 success, 0 failures -- Время выполнения: 234ms -- Пик RAM: 380 MiB (не превышен лимит 512 MiB) -- Соединения БД не удерживаются на время S3-вызова diff --git a/evidence/aborted.txt b/evidence/aborted.txt new file mode 100644 index 0000000..9f88408 --- /dev/null +++ b/evidence/aborted.txt @@ -0,0 +1 @@ +PASS interrupted upload: connection closed before full body; metadata and S3 object listing unchanged diff --git a/evidence/build.txt b/evidence/build.txt new file mode 100644 index 0000000..28ec644 --- /dev/null +++ b/evidence/build.txt @@ -0,0 +1,119 @@ + Image mtusi2026lab4-api Building +#1 [internal] load local bake definitions +#1 reading from stdin 507B 0.0s done +#1 DONE 0.0s + +#2 [internal] load build definition from Dockerfile +#2 transferring dockerfile: 1.09kB 0.0s done +#2 DONE 0.1s + +#3 [internal] load metadata for docker.io/library/eclipse-temurin:21-jdk-jammy +#3 ... + +#4 [internal] load metadata for docker.io/library/eclipse-temurin:21-jre-jammy +#4 DONE 0.8s + +#3 [internal] load metadata for docker.io/library/eclipse-temurin:21-jdk-jammy +#3 DONE 1.1s + +#5 [internal] load .dockerignore +#5 transferring context: 237B done +#5 DONE 0.1s + +#6 [internal] load build context +#6 DONE 0.0s + +#7 [builder 1/11] FROM docker.io/library/eclipse-temurin:21-jdk-jammy@sha256:4b1a74fbffbe6b4e7b6f0171a472478282fb463790a51edf10e465d578f22b69 +#7 resolve docker.io/library/eclipse-temurin:21-jdk-jammy@sha256:4b1a74fbffbe6b4e7b6f0171a472478282fb463790a51edf10e465d578f22b69 +#7 ... + +#8 [runner 1/5] FROM docker.io/library/eclipse-temurin:21-jre-jammy@sha256:1eac8b8fee11ca7d6725a48bc5d41992ed1561504ebae25490d2d062847e8269 +#8 resolve docker.io/library/eclipse-temurin:21-jre-jammy@sha256:1eac8b8fee11ca7d6725a48bc5d41992ed1561504ebae25490d2d062847e8269 0.5s done +#8 DONE 0.5s + +#7 [builder 1/11] FROM docker.io/library/eclipse-temurin:21-jdk-jammy@sha256:4b1a74fbffbe6b4e7b6f0171a472478282fb463790a51edf10e465d578f22b69 +#7 resolve docker.io/library/eclipse-temurin:21-jdk-jammy@sha256:4b1a74fbffbe6b4e7b6f0171a472478282fb463790a51edf10e465d578f22b69 0.5s done +#7 DONE 0.6s + +#6 [internal] load build context +#6 transferring context: 42.52kB 0.0s done +#6 DONE 0.2s + +#9 [builder 7/11] RUN unzip -q /tmp/gradle.zip -d /opt && rm /tmp/gradle.zip +#9 CACHED + +#10 [builder 8/11] RUN gradle dependencies --no-daemon +#10 CACHED + +#11 [builder 4/11] COPY gradle gradle +#11 CACHED + +#12 [builder 6/11] COPY gradle-9.7.1-bin.zip /tmp/gradle.zip +#12 CACHED + +#13 [builder 2/11] WORKDIR /build +#13 CACHED + +#14 [builder 3/11] RUN apt-get update && apt-get install -y --no-install-recommends unzip && rm -rf /var/lib/apt/lists/* +#14 CACHED + +#15 [builder 5/11] COPY gradlew settings.gradle build.gradle ./ +#15 CACHED + +#16 [builder 9/11] COPY contracts contracts +#16 CACHED + +#17 [builder 10/11] COPY src src +#17 DONE 5.7s + +#18 [builder 11/11] RUN gradle bootJar -x test --no-daemon +#18 1.359 To honour the JVM settings for this build a single-use Daemon process will be forked. For more on this, please refer to https://docs.gradle.org/9.7.1/userguide/gradle_daemon.html#sec:disabling_the_daemon in the Gradle documentation. +#18 2.956 Daemon will be stopped at the end of the build +#18 30.06 +#18 30.06 > Task :compileJava +#18 30.06 Note: /build/src/main/java/ru/mtusi/messenger/controller/controllerAdvice/GlobalExceptionHandler.java uses or overrides a deprecated API. +#18 30.06 Note: Recompile with -Xlint:deprecation for details. +#18 40.25 +#18 40.25 > Task :processResources +#18 40.25 > Task :classes +#18 40.25 > Task :resolveMainClassName +#18 45.05 > Task :bootJar +#18 45.11 +#18 45.11 [Incubating] Problems report is available at: file:///build/build/reports/problems/problems-report.html +#18 45.11 +#18 45.11 BUILD SUCCESSFUL in 44s +#18 45.11 4 actionable tasks: 4 executed +#18 45.11 Consider enabling configuration cache to speed up this build: https://docs.gradle.org/9.7.1/userguide/configuration_cache_enabling.html +#18 DONE 48.4s + +#19 [runner 2/5] WORKDIR /app +#19 CACHED + +#20 [runner 3/5] RUN groupadd --system spring && useradd --system --gid spring --no-create-home spring +#20 CACHED + +#21 [runner 4/5] COPY --from=builder /build/build/libs/*.jar app.jar +#21 DONE 4.0s + +#22 [runner 5/5] RUN chown spring:spring app.jar +#22 DONE 7.3s + +#23 exporting to image +#23 exporting layers +#23 exporting layers 9.8s done +#23 exporting manifest sha256:f00e792050e174a75cde5f4b956125ba7d36a8cfe33609b105e482472d5ce9d6 0.1s done +#23 exporting config sha256:92c76641eab13bc11abd71c0a35a9c9247dae83f41ae7a4c325b51bcfb04fcc7 +#23 exporting config sha256:92c76641eab13bc11abd71c0a35a9c9247dae83f41ae7a4c325b51bcfb04fcc7 0.1s done +#23 exporting attestation manifest sha256:8b2c56fd96a5b9747b6844bb0a6aef04df87bc6409e6b9ec109dce156a22ff6e +#23 exporting attestation manifest sha256:8b2c56fd96a5b9747b6844bb0a6aef04df87bc6409e6b9ec109dce156a22ff6e 0.1s done +#23 exporting manifest list sha256:47cb763f87e76f1d565b338531a1014058b598be3a544934f297a31a83a80cbb +#23 exporting manifest list sha256:47cb763f87e76f1d565b338531a1014058b598be3a544934f297a31a83a80cbb 0.1s done +#23 naming to docker.io/library/mtusi2026lab4-api:latest +#23 naming to docker.io/library/mtusi2026lab4-api:latest 0.0s done +#23 unpacking to docker.io/library/mtusi2026lab4-api:latest +#23 unpacking to docker.io/library/mtusi2026lab4-api:latest 1.6s done +#23 DONE 12.0s + +#24 resolving provenance for metadata file +#24 DONE 0.3s + Image mtusi2026lab4-api Built diff --git a/evidence/contract.txt b/evidence/contract.txt new file mode 100644 index 0000000..b62af9c --- /dev/null +++ b/evidence/contract.txt @@ -0,0 +1 @@ +PASS: offline skeleton structure, local refs and Python syntax; application NOT tested. diff --git a/evidence/environment.json b/evidence/environment.json new file mode 100644 index 0000000..38fe26e --- /dev/null +++ b/evidence/environment.json @@ -0,0 +1,19 @@ +{ + "date": "2026-11-01 Europe/Moscow", + "base_commit": "057143dd915f0fe75b72e253db103800940eeb38", + "working_tree": "modified; source hashes recorded separately", + "docker": "8 CPUs 8212668416 bytes", + "counts_format": "attachments|messages|users", + "api_limits": "2 replicas, each CPU=1 RAM=512MiB heap=320MiB", + "api_image": "sha256:47cb763f87e76f1d565b338531a1014058b598be3a544934f297a31a83a80cbb", + "reconcile_elapsed_ms": [ + 2709, + 682, + 552, + 1896, + 795, + 676 + ], + "reconcile_dataset_counts": "80|196|66", + "final_counts": "104|301|78" +} \ No newline at end of file diff --git a/evidence/extended.txt b/evidence/extended.txt new file mode 100644 index 0000000..d29fe9f --- /dev/null +++ b/evidence/extended.txt @@ -0,0 +1,12 @@ +PASS negative: MIME, corrupt, oversized, filename, duplicate/unknown/cross-chat attachments +PASS 24 simultaneous idempotent requests across 2 replicas; one message +image_bytes= 8674413 memory_samples= ['mtusi2026lab4-api-2 425.5MiB / 512MiB\nmtusi2026lab4-api-1 427.2MiB / 512MiB\nmtusi2026lab4-lb-1 29.76MiB / 7.649GiB\nmtusi2026lab4-minio-1 245.1MiB / 7.649GiB\nmtusi2026lab4-db-1 54.5MiB / 1GiB\nmtusi2026lab4-redis-1 5.648MiB / 7.649GiB\n', 'mtusi2026lab4-api-2 438.1MiB / 512MiB\nmtusi2026lab4-api-1 433.1MiB / 512MiB\nmtusi2026lab4-lb-1 32.69MiB / 7.649GiB\nmtusi2026lab4-minio-1 246.3MiB / 7.649GiB\nmtusi2026lab4-db-1 63.9MiB / 1GiB\nmtusi2026lab4-redis-1 5.648MiB / 7.649GiB\n', 'mtusi2026lab4-api-2 447.6MiB / 512MiB\nmtusi2026lab4-api-1 453.1MiB / 512MiB\nmtusi2026lab4-lb-1 33.38MiB / 7.649GiB\nmtusi2026lab4-minio-1 247MiB / 7.649GiB\nmtusi2026lab4-db-1 64.46MiB / 1GiB\nmtusi2026lab4-redis-1 5.898MiB / 7.649GiB\n', 'mtusi2026lab4-api-2 454.1MiB / 512MiB\nmtusi2026lab4-api-1 454.8MiB / 512MiB\nmtusi2026lab4-lb-1 32.6MiB / 7.649GiB\nmtusi2026lab4-minio-1 247.8MiB / 7.649GiB\nmtusi2026lab4-db-1 64.63MiB / 1GiB\nmtusi2026lab4-redis-1 5.648MiB / 7.649GiB\n', 'mtusi2026lab4-api-2 453.7MiB / 512MiB\nmtusi2026lab4-api-1 449.7MiB / 512MiB\nmtusi2026lab4-lb-1 20.16MiB / 7.649GiB\nmtusi2026lab4-minio-1 248.8MiB / 7.649GiB\nmtusi2026lab4-db-1 65.42MiB / 1GiB\nmtusi2026lab4-redis-1 5.652MiB / 7.649GiB\n', 'mtusi2026lab4-api-2 453.9MiB / 512MiB\nmtusi2026lab4-api-1 443.8MiB / 512MiB\nmtusi2026lab4-lb-1 9.195MiB / 7.649GiB\nmtusi2026lab4-minio-1 249.6MiB / 7.649GiB\nmtusi2026lab4-db-1 65.41MiB / 1GiB\nmtusi2026lab4-redis-1 5.898MiB / 7.649GiB\n'] +PASS 20 uploads concurrency=10 elapsed_seconds= 12.547 +mtusi2026lab4-api-2 453.6MiB / 512MiB +mtusi2026lab4-api-1 443.8MiB / 512MiB +mtusi2026lab4-lb-1 9.188MiB / 7.649GiB +mtusi2026lab4-minio-1 249.6MiB / 7.649GiB +mtusi2026lab4-db-1 65.65MiB / 1GiB +mtusi2026lab4-redis-1 5.898MiB / 7.649GiB + +PASS logout across replicas diff --git a/evidence/failover.txt b/evidence/failover.txt new file mode 100644 index 0000000..d4e0c5b --- /dev/null +++ b/evidence/failover.txt @@ -0,0 +1,3 @@ +Executing explicit failover hook; hook output suppressed to avoid leaking secrets. +{"observed_instances_before": 2, "successful_writes": 99, "failed_attempts": 0, "first_recovery_seconds": 2.578} +PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence. diff --git a/evidence/faults.txt b/evidence/faults.txt new file mode 100644 index 0000000..5e7c192 --- /dev/null +++ b/evidence/faults.txt @@ -0,0 +1,3 @@ +PASS direct anonymous GET existing object returns 403 +PASS S3 outage: readiness and upload return 503 +PASS DB failure: no success response, no committed metadata; orphan cleanup covered by track2 diff --git a/evidence/final-state.txt b/evidence/final-state.txt new file mode 100644 index 0000000..f66f7d6 --- /dev/null +++ b/evidence/final-state.txt @@ -0,0 +1,8 @@ +2026-11-01 Europe/Moscow +GET /health/ready: 200; two distinct X-Instance-Id observed +api Up 55 seconds (healthy) +api Up 55 seconds (healthy) +db Up 55 seconds (healthy) +lb Up 55 seconds +minio Up 55 seconds (healthy) +redis Up 55 seconds (healthy) diff --git a/evidence/locking.txt b/evidence/locking.txt new file mode 100644 index 0000000..0584e31 --- /dev/null +++ b/evidence/locking.txt @@ -0,0 +1 @@ +PASS PUT/COMMIT barrier: uncommitted metadata invisible; exclusive reconcile lock denied during upload; released after commit; other member downloads identical bytes diff --git a/evidence/persistence.txt b/evidence/persistence.txt new file mode 100644 index 0000000..749fd29 --- /dev/null +++ b/evidence/persistence.txt @@ -0,0 +1,2 @@ +Executing explicit persistence hook; hook output suppressed to avoid leaking secrets. +PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence. diff --git a/evidence/private-bucket.txt b/evidence/private-bucket.txt new file mode 100644 index 0000000..e1a29c1 --- /dev/null +++ b/evidence/private-bucket.txt @@ -0,0 +1 @@ +403 diff --git a/evidence/reconciler-apply-log.txt b/evidence/reconciler-apply-log.txt new file mode 100644 index 0000000..0b493e5 --- /dev/null +++ b/evidence/reconciler-apply-log.txt @@ -0,0 +1,9 @@ +2026-10-31 23:05:08,625 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=false key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:05:08,753 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=false key=proof-3146543cd1fe4a9baa759dd244768dfe +2026-10-31 23:05:09,646 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=false id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:05:09,833 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=false elapsedMs=2709 +2026-10-31 23:05:13,718 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=false elapsedMs=682 +2026-10-31 23:05:17,277 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=false elapsedMs=552 +2026-10-31 23:05:12,894 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=false elapsedMs=1896 +2026-10-31 23:05:16,707 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=false elapsedMs=795 +2026-10-31 23:05:20,451 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=false elapsedMs=676 \ No newline at end of file diff --git a/evidence/reconciler-dry-run-log.txt b/evidence/reconciler-dry-run-log.txt new file mode 100644 index 0000000..4dc68cc --- /dev/null +++ b/evidence/reconciler-dry-run-log.txt @@ -0,0 +1,175 @@ +2026-10-31 23:02:31,306 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:32,211 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=2613 +2026-10-31 23:02:36,553 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:37,581 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=3100 +2026-10-31 23:02:40,774 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:42,152 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=1474 +2026-10-31 23:02:48,262 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:49,457 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=1290 +2026-10-31 23:02:52,480 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:53,058 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=595 +2026-10-31 23:03:02,111 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:02,363 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=269 +2026-10-31 23:03:05,402 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:05,708 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=338 +2026-10-31 23:03:14,732 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:15,168 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=449 +2026-10-31 23:03:18,187 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:18,460 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=284 +2026-10-31 23:03:21,473 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:38,331 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:38,848 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=540 +2026-10-31 23:03:21,778 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=314 +2026-10-31 23:03:24,805 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:24,989 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:25,023 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=238 +2026-10-31 23:03:28,036 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:45,002 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:46,786 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=1884 +2026-10-31 23:03:28,203 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:28,284 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=257 +2026-10-31 23:03:31,295 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:02:49,808 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:31,376 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:31,401 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=113 +2026-10-31 23:02:50,698 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=905 +2026-10-31 23:03:34,413 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:34,497 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:02,757 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:34,523 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=118 +2026-10-31 23:03:37,535 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:03,215 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=497 +2026-10-31 23:03:37,635 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:37,664 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=139 +2026-10-31 23:03:40,688 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:40,755 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:40,774 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=104 +2026-10-31 23:03:43,787 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:43,905 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:43,947 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=169 +2026-10-31 23:03:46,971 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:47,056 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:47,077 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=125 +2026-10-31 23:03:06,231 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:50,101 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:06,526 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=306 +2026-10-31 23:03:50,175 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:50,194 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=111 +2026-10-31 23:03:15,545 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:53,206 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:53,302 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:53,365 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=168 +2026-10-31 23:03:56,381 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:56,460 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:56,494 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=123 +2026-10-31 23:03:59,509 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:15,923 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=389 +2026-10-31 23:03:59,610 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:59,650 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=151 +2026-10-31 23:04:02,675 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:18,964 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:03,062 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:03,119 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=463 +2026-10-31 23:04:06,130 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:19,280 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=351 +2026-10-31 23:04:06,278 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:06,316 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=194 +2026-10-31 23:04:09,340 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:22,298 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:09,605 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:22,615 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=328 +2026-10-31 23:03:25,629 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:25,831 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:25,904 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=286 +2026-10-31 23:03:28,920 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:29,039 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:29,106 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=198 +2026-10-31 23:03:32,132 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:09,638 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=319 +2026-10-31 23:03:32,314 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:12,654 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:12,850 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:32,341 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=228 +2026-10-31 23:04:12,880 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=238 +2026-10-31 23:04:15,901 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:15,991 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:35,365 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:16,074 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=189 +2026-10-31 23:04:19,088 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:35,459 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:35,502 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=157 +2026-10-31 23:03:38,535 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:19,188 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:19,218 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=138 +2026-10-31 23:04:22,236 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:38,654 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:38,682 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=176 +2026-10-31 23:03:41,696 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:22,371 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:41,803 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:22,399 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=176 +2026-10-31 23:04:25,413 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:25,455 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=proof-3146543cd1fe4a9baa759dd244768dfe +2026-10-31 23:04:25,596 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:41,836 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=150 +2026-10-31 23:04:25,675 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=273 +2026-10-31 23:04:28,690 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:28,714 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=proof-3146543cd1fe4a9baa759dd244768dfe +2026-10-31 23:04:28,773 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:44,859 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:28,799 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=120 +2026-10-31 23:03:44,942 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:44,993 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=152 +2026-10-31 23:03:48,005 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:31,809 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:31,837 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=proof-3146543cd1fe4a9baa759dd244768dfe +2026-10-31 23:04:31,904 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:48,091 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:48,113 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=115 +2026-10-31 23:04:31,965 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=162 +2026-10-31 23:03:51,135 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:51,206 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:51,223 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=105 +2026-10-31 23:03:54,252 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:54,423 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:54,508 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=279 +2026-10-31 23:03:57,534 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:03:57,708 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:03:57,735 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=221 +2026-10-31 23:04:00,758 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:00,928 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:01,092 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=351 +2026-10-31 23:04:04,112 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:04,214 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:04,290 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=193 +2026-10-31 23:04:07,311 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:07,394 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:07,427 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=136 +2026-10-31 23:04:10,449 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:10,611 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:10,646 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=214 +2026-10-31 23:04:13,671 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:14,026 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:14,076 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=425 +2026-10-31 23:04:17,086 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:17,189 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:17,215 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=135 +2026-10-31 23:04:20,236 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:20,338 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:20,363 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=143 +2026-10-31 23:04:23,387 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:23,437 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=proof-3146543cd1fe4a9baa759dd244768dfe +2026-10-31 23:04:23,618 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:23,718 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=349 +2026-10-31 23:04:26,746 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:26,799 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=proof-3146543cd1fe4a9baa759dd244768dfe +2026-10-31 23:04:26,893 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:26,939 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=216 +2026-10-31 23:04:29,960 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:29,999 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=proof-3146543cd1fe4a9baa759dd244768dfe +2026-10-31 23:04:30,054 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:30,091 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=147 +2026-10-31 23:04:33,108 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=096fb076-173f-43ab-9779-87748b3f5901 +2026-10-31 23:04:33,145 INFO [] r.m.m.s.AttachmentReconciler - Reconcile orphan dryRun=true key=proof-3146543cd1fe4a9baa759dd244768dfe +2026-10-31 23:04:33,280 INFO [] r.m.m.s.AttachmentReconciler - Reconcile missing dryRun=true id=aba148cc-25a4-45b5-9e9b-e1932d97ac4b +2026-10-31 23:04:33,321 INFO [] r.m.m.s.AttachmentReconciler - Reconcile complete dryRun=true elapsedMs=226 \ No newline at end of file diff --git a/evidence/restart-after.txt b/evidence/restart-after.txt new file mode 100644 index 0000000..70af5ff --- /dev/null +++ b/evidence/restart-after.txt @@ -0,0 +1,6 @@ +/mtusi2026lab4-api-1 2026-10-31T23:07:53.373414795Z +/mtusi2026lab4-api-2 2026-10-31T23:07:53.246086496Z +/mtusi2026lab4-db-1 2026-10-31T23:07:53.514536294Z +/mtusi2026lab4-lb-1 2026-10-31T23:07:53.295382239Z +/mtusi2026lab4-minio-1 2026-10-31T23:07:52.724899356Z +/mtusi2026lab4-redis-1 2026-10-31T23:07:52.593327772Z diff --git a/evidence/restart-before.txt b/evidence/restart-before.txt new file mode 100644 index 0000000..d9c6d0b --- /dev/null +++ b/evidence/restart-before.txt @@ -0,0 +1,6 @@ +/mtusi2026lab4-api-1 2026-10-31T23:05:39.650988072Z +/mtusi2026lab4-api-2 2026-10-31T23:07:01.788885681Z +/mtusi2026lab4-db-1 2026-10-31T22:56:53.864795289Z +/mtusi2026lab4-lb-1 2026-10-31T23:02:01.211664851Z +/mtusi2026lab4-minio-1 2026-10-31T22:56:53.561984546Z +/mtusi2026lab4-redis-1 2026-10-31T22:56:53.117528456Z diff --git a/evidence/review-regressions-2026-11-06.json b/evidence/review-regressions-2026-11-06.json new file mode 100644 index 0000000..5ddfcee --- /dev/null +++ b/evidence/review-regressions-2026-11-06.json @@ -0,0 +1,24 @@ +{ + "date": "2026-11-06 Europe/Moscow", + "base_commit": "057143dd915f0fe75b72e253db103800940eeb38", + "scope": "current uncommitted working tree, not remote commit", + "base_url": "http://127.0.0.1:18080", + "port_reason": "Windows excluded TCP range 8057-8156 includes 8080; separate Compose override changes only host publication", + "command": "docker compose -f docker-compose.yml -f compose.review.yaml up -d; BASE_URL=http://127.0.0.1:18080 py tests/review_regressions.py", + "jpeg_generator": "Pillow 12.3.0; synthetic RGB image", + "generated_mapper_mime_lines": [ + "attachmentDTO.setContentType( mapContentType( entity.getContentType() ) );", + "attachment.setContentType( contentType );" + ], + "mime_enum_valueOf_present": false, + "sha256": { + "src/main/java/ru/mtusi/messenger/mapper/AttachmentMapper.java": "04d419c72a402b8c48462a6a38c243837e3163b42165d478afe844d3aad0ab2d", + "src/main/java/ru/mtusi/messenger/dto/CreateMessageDTO.java": "3dda594258f94aed0322bedb76cd031faf325b52ec49d4ad9d6316e38570a9cb", + "src/main/java/ru/mtusi/messenger/dto/MessageDTO.java": "c136940d1f406797220bf7b6f65e604694bca64f6a9553ce1c30d653a0b9745d", + "src/main/java/ru/mtusi/messenger/validation/AttachmentValidator.java": "bc0a335f7d32b6b659c42cdd24858b8817e1558b338f5f8c075f8375c4ee6b15", + "nginx.conf": "5d7753018d2e6a3c141963d932571382abdffe45869c1e98bc58dda015e6fd0f", + "tests/review_regressions.py": "511cd9ca5ba4905a21d4dda2ac79bd16a29edb940c4a29d9c1f78ff2de2c57b2", + "compose.review.yaml": "66b48810a6eb1e78c2e6c3bb3be53c504c8d9e237cdf8c2204d3b922f89bdc78" + }, + "containers": "api Up 6 minutes (healthy) 8080/tcp\napi Up 6 minutes (healthy) 8080/tcp\ndb Up 6 minutes (healthy) 5432/tcp\nlb Up 56 seconds 127.0.0.1:18080->80/tcp\nminio Up 4 minutes (healthy) 9000/tcp\nredis Up 6 minutes (healthy) 6379/tcp\n" +} \ No newline at end of file diff --git a/evidence/review-regressions-2026-11-06.txt b/evidence/review-regressions-2026-11-06.txt new file mode 100644 index 0000000..5652bb2 --- /dev/null +++ b/evidence/review-regressions-2026-11-06.txt @@ -0,0 +1,6 @@ +2026-11-06 Europe/Moscow; current working tree; synthetic data only +PASS PNG/JPEG: 10 uploads -> 201; 10 metadata GET -> 200; 10 downloads identical; DB +10; instances=2 +PASS messages: plain attachment_ids=[]; file-only POST -> 201; cross-chat/unknown/outsider -> 404 +PASS sample.png[:33]: 10 x 415; ../x.png: 400; DB and S3 unchanged +PASS valid PNG exactly 1770843 bytes: POST -> 201 through LB; download checksum matches +PASS all five review regressions diff --git a/evidence/smoke.txt b/evidence/smoke.txt new file mode 100644 index 0000000..c554fd6 --- /dev/null +++ b/evidence/smoke.txt @@ -0,0 +1 @@ +PASS: lab4 public API scenario; infrastructure and grading evidence remain separate. diff --git a/evidence/source-sha256.json b/evidence/source-sha256.json new file mode 100644 index 0000000..2efb0dd --- /dev/null +++ b/evidence/source-sha256.json @@ -0,0 +1,134 @@ +{ + "src/main/java/ru/mtusi/messenger/MessengerLab1Application.java": "ebff5c9496dda66c8137788564f170ece5ac5bebb2e1e1b13b0bc351057c4c01", + "src/main/java/ru/mtusi/messenger/cache/CacheHealthTracker.java": "01247b304b36da4115f64d1850cf417e6c050d4d833faf0477fe9e8ec4726d47", + "src/main/java/ru/mtusi/messenger/config/JacksonConfig.java": "d81cf57029887349a496e96c31a856726f4e376075723425777f2e04a1d4d4c4", + "src/main/java/ru/mtusi/messenger/config/MinioConfig.java": "4fa20cd0bbd6f7923fbba5dce0f6faace584d797e50bbac2c0e9c0bb6f4341b2", + "src/main/java/ru/mtusi/messenger/config/OpenApiConfig.java": "214152ac0da558a2953c3b8ed5a6378917bc904eaf89a6d248af5ec27b2afb66", + "src/main/java/ru/mtusi/messenger/config/RateLimitConfig.java": "a529b58411902caff30048449e806ad3435be537257d7c7c66b733d6f29a7c8d", + "src/main/java/ru/mtusi/messenger/config/RedisConfig.java": "7c46490bbdc8a71fd7bf1932a4b0473effb2c29ef9e8e0d5eee2535b623912b7", + "src/main/java/ru/mtusi/messenger/config/SecurityConfig.java": "11a7b514390a004fa0facf67767b7cb33363fb5bc9a00e6e63dde3ff48f12b4a", + "src/main/java/ru/mtusi/messenger/config/ShedLockConfig.java": "7b316cc874133b9d0cb765662db093ffa500a095cbed66733921d27918c22886", + "src/main/java/ru/mtusi/messenger/config/properties/AppProperties.java": "6b6bb675240794dd9e23ca0f0c00c1643be63568f317edd0abffa2fdfbdaa6eb", + "src/main/java/ru/mtusi/messenger/config/properties/IdempotencyProperties.java": "acf9c9d25f1f5685dbed0e7a34bfc90a690d7bd964f4c65a25a9e5bdaaabf2ae", + "src/main/java/ru/mtusi/messenger/config/properties/InstanceProperties.java": "6948d3846adddad1882b6f5351ee5e3a3bffb62f6ead726fdd1226a272545c8f", + "src/main/java/ru/mtusi/messenger/config/properties/RateLimitProperties.java": "cc0903db00b44e79b32d58c1507f96b10c4a8bf950eaa2702947aed8b9e82b17", + "src/main/java/ru/mtusi/messenger/config/properties/ReconcilerProperties.java": "b15f02fe0dd4a42b5f20d96a2be293095955d959afe6493f3c02d77399164573", + "src/main/java/ru/mtusi/messenger/config/properties/RedisProperties.java": "28a9da022e39f639d46e6a3d64d55065058db7cccda2f46c54506e93c954909d", + "src/main/java/ru/mtusi/messenger/config/properties/S3Properties.java": "810b5a44bb487c4a4e0f5790f5011c8575becd43f06694d46aed2a417053ba50", + "src/main/java/ru/mtusi/messenger/config/properties/SessionProperties.java": "412cd9380e9962c14f179d8d95103a6a1efead47be08913085e13dc478858639", + "src/main/java/ru/mtusi/messenger/config/properties/UploadProperties.java": "e28d7084351928728a15527dc1ddd3ff3386c2b91a948265fe73308e8cda27fa", + "src/main/java/ru/mtusi/messenger/controller/AttachmentController.java": "c714109087d223387fcb30afe5d719a94d8acd4fd36c6208c471921b1d28086a", + "src/main/java/ru/mtusi/messenger/controller/AttachmentsApi.java": "cd24b1773b7c12d146bf929510c7421e84ff41e31390717b491586ae1c245e96", + "src/main/java/ru/mtusi/messenger/controller/AuthApi.java": "cc5879043e166008d6d5d899339d91473f1a95dac729999ab3af809926a3157f", + "src/main/java/ru/mtusi/messenger/controller/AuthController.java": "06c823e3718e387922e05eebe75f44a8321661d8f57d2636f1a6b5b5f1490fe1", + "src/main/java/ru/mtusi/messenger/controller/ChatController.java": "91c280c54767d88c55a17d6b8bf1be1fc1a209582b6f99afe1ceb4370613202a", + "src/main/java/ru/mtusi/messenger/controller/ChatsApi.java": "d2b467ddb281e742e02925e52f50ec8f45bcb3d9371e2445ef34a6b2637e6a67", + "src/main/java/ru/mtusi/messenger/controller/HealthApi.java": "f13f8333478cbb92f2c669f031b5860d28d6c5aa2fc72678f98eb881fc4c20c3", + "src/main/java/ru/mtusi/messenger/controller/HealthController.java": "5975098719aa56f14db3528defd64acec2e4bd8e8bf791c68ac3463ebbb1def2", + "src/main/java/ru/mtusi/messenger/controller/MessageController.java": "307891082d0e658e1cc7cfc184501fcf15724b2cbffb0a77905953c54831c0d8", + "src/main/java/ru/mtusi/messenger/controller/MessagesApi.java": "b0751aad2ac3a5928659a62e3815d7246f0706a1a38498d05d2cefc48f6ab670", + "src/main/java/ru/mtusi/messenger/controller/UserController.java": "ea2f842e70694cf4135b9803eb19ecea3dd21e268fd140e8a7ac196c93aac199", + "src/main/java/ru/mtusi/messenger/controller/UsersApi.java": "01fcddf9d694706cbdbba94b3a1bad479939e845948b1b7f9ad61f8dcaf5d00e", + "src/main/java/ru/mtusi/messenger/controller/controllerAdvice/GlobalExceptionHandler.java": "d3c65a3f6c7becf2a0acc33f76a58fa24fb99810525c03821b19359a31e3fd4e", + "src/main/java/ru/mtusi/messenger/dto/AttachmentDTO.java": "c99729a1c6f84979a53a196970cc8d20fbdde13adacd1606294fc9d9dc8bb98f", + "src/main/java/ru/mtusi/messenger/dto/ChatDTO.java": "33ce24cf9d1265f2ec0f7f4e490dd6d143e93385a91289c926bf8fbe20382d3f", + "src/main/java/ru/mtusi/messenger/dto/ChatPageDTO.java": "dec21d65e2eea9f5c3250b0ee8c8c83c8de7c48db2de2757982dbb0ece6dd67a", + "src/main/java/ru/mtusi/messenger/dto/CreateChatDTO.java": "e2fb6c482022e89b22d790d7c25769411e2277768f40bc7d99cc513dc4b272f3", + "src/main/java/ru/mtusi/messenger/dto/CreateMessageDTO.java": "3dda594258f94aed0322bedb76cd031faf325b52ec49d4ad9d6316e38570a9cb", + "src/main/java/ru/mtusi/messenger/dto/CreateUserDTO.java": "77be22f51ed669cb26b27c2d848050b8d9a54748b51b4e2d45dda9ab925bc02c", + "src/main/java/ru/mtusi/messenger/dto/ErrorDTO.java": "93cf9f964cb70f8a95293c0c0014a703178d478d67380ccfe4c5b220e780b2a1", + "src/main/java/ru/mtusi/messenger/dto/ErrorErrorDTO.java": "1b5431f78b23bdf27a327ae3bf89d52f66a5394f0a5902688245538bfae0262e", + "src/main/java/ru/mtusi/messenger/dto/HealthDTO.java": "ba735da4c3bb594a2022449813426fd1f7b63dc1e5fca1f19b9dc3a20268abd0", + "src/main/java/ru/mtusi/messenger/dto/MessageDTO.java": "c136940d1f406797220bf7b6f65e604694bca64f6a9553ce1c30d653a0b9745d", + "src/main/java/ru/mtusi/messenger/dto/MessagePageDTO.java": "fdf92495fe8911e21c44bce2e17871a19ac3225dca2ef35228dfb3cce312c631", + "src/main/java/ru/mtusi/messenger/dto/SessionDTO.java": "18df9b2402c31cd9f29eeb62dbf50369de449f8af9abb1454425290254f7fda8", + "src/main/java/ru/mtusi/messenger/dto/UserDTO.java": "b0c0b9e39df18d86c04ef3cec08af3b8eabb14a963b8a440df0407cbfb5c77a3", + "src/main/java/ru/mtusi/messenger/dto/VariantDTO.java": "2271032b5450f486dc315b1fe8adc554e48835a3febad7024e73fd7ba483f11e", + "src/main/java/ru/mtusi/messenger/exception/ApiErrorFactory.java": "decfec03fcc44355356239a5c967d8fc5e94d0244e6f629ba5e25a2e727491ad", + "src/main/java/ru/mtusi/messenger/exception/ConflictException.java": "e4a73d92c86ea8ca411db8f8995a0ff4be4ba422898fda7d5d4b98dfc79681ce", + "src/main/java/ru/mtusi/messenger/exception/InvalidRequestException.java": "382c8fb84ebc3d591cbf0633f65e57df9aa5652ecd2068673ff2c0d2f55630f5", + "src/main/java/ru/mtusi/messenger/exception/NotFoundException.java": "e167e13943712563093ce4ec988b9cc291d872059e4125cb5b825b5d0e6579f1", + "src/main/java/ru/mtusi/messenger/exception/PayloadTooLargeException.java": "9b9be34b4d897619e65666efc11ddde6ca0e330a14d20ede25dc22694cdd33e1", + "src/main/java/ru/mtusi/messenger/exception/RateLimitedException.java": "c88f435c66c7e5db6076d0a5706e2be49606fe300d816e08cdc88fd6da7c404d", + "src/main/java/ru/mtusi/messenger/exception/ServiceUnavailableException.java": "110b237c0cce9d57b8520b5ba5345e65552481dde0700a88b7646363d9dc90b4", + "src/main/java/ru/mtusi/messenger/exception/StaleAttachmentException.java": "903877e733a5a9717d6b19d5f0635d9284bc59c30e67f3778780e7e794dfaddb", + "src/main/java/ru/mtusi/messenger/exception/StorageUnavailableException.java": "8a5780f38d5e47505b247874c9418bfba9e4841bae2a41938700a180a349b1e8", + "src/main/java/ru/mtusi/messenger/exception/UnauthorizedException.java": "2c3b0dce5c04947a1e1d5fd50130433b204b8155315ef49e7915775f65451213", + "src/main/java/ru/mtusi/messenger/exception/UnsupportedMediaTypeException.java": "28889bc84475c4319f13f3e76dd3b9939325837afb7bb25d5d93f8ff0fe985d6", + "src/main/java/ru/mtusi/messenger/filter/InstanceIdFilter.java": "1927510c6524e7925a44ad6f27863dc88872ca429c1dd2d88b84e8cdddfce336", + "src/main/java/ru/mtusi/messenger/filter/RateLimitFilter.java": "794ae35e8bf7654bd2822316159a0026a371e1f64c0428dbc0d947c2baf45a6b", + "src/main/java/ru/mtusi/messenger/filter/RequestIdFilter.java": "4a0deae7a6dda1cce176caebb86f0dbeae06bc021cb283084cdd8ceb453de374", + "src/main/java/ru/mtusi/messenger/mapper/AttachmentMapper.java": "04d419c72a402b8c48462a6a38c243837e3163b42165d478afe844d3aad0ab2d", + "src/main/java/ru/mtusi/messenger/mapper/ChatMapper.java": "44a598e24d18302cf0252a8784b54f861387f0c64e2317a4cd108e759e277689", + "src/main/java/ru/mtusi/messenger/mapper/MessageMapper.java": "ccbf70b59f9b6d574849462b00e7be4d5ff13fe14cc35dbf7fc63ebacc383f1b", + "src/main/java/ru/mtusi/messenger/mapper/SessionMapper.java": "2102ae913a14ac0e564470b6ae429f721fbb0bc2473ae0e15d2b23cd184671a3", + "src/main/java/ru/mtusi/messenger/mapper/UserMapper.java": "62e859c1263976b652d32a0ffd4d0a8c2d0f38df0ae94c627c55cc57cd90b7eb", + "src/main/java/ru/mtusi/messenger/model/Attachment.java": "08558ff1903ec23da66e105b54fc591e493dc230752f263296c322d0372735b6", + "src/main/java/ru/mtusi/messenger/model/AttachmentStatus.java": "c0b2814d1b59aefdb52938caf52207b507313d2afedc9b2440f8cea96b235db1", + "src/main/java/ru/mtusi/messenger/model/Chat.java": "71edfe3387e148d59e2c991e01611d4ad632291ca55c990ec05b4497993cbc04", + "src/main/java/ru/mtusi/messenger/model/IdempotencyKey.java": "ba71e214c392487503c7376af03cc109a5d364cf580a5c755686ec4a53240036", + "src/main/java/ru/mtusi/messenger/model/Message.java": "487a3552f72d68d56d3193c36ee04857e6f0807454ddb8860e639dbcabc122d4", + "src/main/java/ru/mtusi/messenger/model/Session.java": "dbffe079041d5819ac1f80f62d671b84d9e679c28894a14c122dbe734ba81e73", + "src/main/java/ru/mtusi/messenger/model/User.java": "99fe24ee3adb30effc690792276c8473b5d2878ef140845db6af7d04594f93e3", + "src/main/java/ru/mtusi/messenger/repository/AttachmentRepository.java": "6d257db7b1f61e9610f3b1061c7dbdc57b9dce0b4ce9d2aad8f35e586e81f157", + "src/main/java/ru/mtusi/messenger/repository/ChatRepository.java": "cea1f215afc44a5a9fc434166e562b1f463872994ef0a5881fa9570922d88e7c", + "src/main/java/ru/mtusi/messenger/repository/IdempotencyRepository.java": "224bf69df8f6b7c5902296849c3a20cf025deb3893d7a7343ac0b76962abf9a5", + "src/main/java/ru/mtusi/messenger/repository/MessageRepository.java": "a9b3e6e2fba5b346c4769eff5b7242bd938ccb34a3f5a9e677e37ba6b66d1b33", + "src/main/java/ru/mtusi/messenger/repository/SessionRepository.java": "a3243410e63515462f7a3c01388dddb0641c26aa5d20549cdb56872713b8df83", + "src/main/java/ru/mtusi/messenger/repository/UserRepository.java": "3fcd0b14687fbdb638662e5cfa0c005817ccb3e0590e97cfd87ce3f1cabdb13f", + "src/main/java/ru/mtusi/messenger/security/ApiAuthenticationEntryPoint.java": "424e3edbd796b442f820fc48ec9b55c0045fc60490b18138b62856ddd5116eaf", + "src/main/java/ru/mtusi/messenger/security/BasicCredentials.java": "025355bb417d99bc54b4d25bfeba1c42f1eda2b3e121832875ba7db5a95d8c55", + "src/main/java/ru/mtusi/messenger/security/SessionAuthenticationFilter.java": "4372c93961b8ddde8cc8d9c337a9d595e1b07366b58aab951f928de570258182", + "src/main/java/ru/mtusi/messenger/security/SessionAuthenticationProvider.java": "24c1f7ebb83e514480db0b9be0430c500850dedb9a8f9a8982d008eba9a00384", + "src/main/java/ru/mtusi/messenger/security/SessionAuthenticationToken.java": "ddc3755f8859e585146c721f996da757bd5dca53dd781bfbf4297fef7ba1b574", + "src/main/java/ru/mtusi/messenger/security/SessionPrincipal.java": "9581156a00d7f6124c80c034cd210506e1b040abdba229e1200bb07a61474801", + "src/main/java/ru/mtusi/messenger/service/AttachmentReconciler.java": "fe1084a7ea72c0598b9a6fb51bf9bd38b787cab1bfeacd3aa136581c4bd10d0e", + "src/main/java/ru/mtusi/messenger/service/AttachmentService.java": "d9a4d2b6d7eab490357d5306905d2b0f36ef0ee3da012bde1e76aae4d6e3fe63", + "src/main/java/ru/mtusi/messenger/service/AuthService.java": "f885882f4700a1751e30a7839af7dac6c3b6baa2bbdf445ce9b5271131748508", + "src/main/java/ru/mtusi/messenger/service/ChatService.java": "b1fca78031db1a57af8ce8ee7f00b340169548d1ffb5857cfb9aa87afce10652", + "src/main/java/ru/mtusi/messenger/service/HealthService.java": "3a870d6848c784a5f53caa65eeb442bd3c4591a842b7ed9dcd2381739cf89195", + "src/main/java/ru/mtusi/messenger/service/IdempotencyService.java": "3b3d2b874c4c2684eab8fd8d65c5866defec1d52e2cd384bf014494da19657b0", + "src/main/java/ru/mtusi/messenger/service/MessageService.java": "b118cd2d46d26b05ce5c8f388d9e9fdd952ffa300a64f3a5882253b35a950e4a", + "src/main/java/ru/mtusi/messenger/service/SessionCacheService.java": "bb15211bd76353ee5782f5c7bc2afcdc8a68bf4f07d3ec6914d10218c318e150", + "src/main/java/ru/mtusi/messenger/service/SessionService.java": "75177e71b443be546cc5222549e0b0eb0f66d92b06453310d90c63b305c25ca6", + "src/main/java/ru/mtusi/messenger/service/UserService.java": "bb0dd55d35637bf0c62c9c5a568eb4280bfe6d4da40d303855035ded714bc9e2", + "src/main/java/ru/mtusi/messenger/util/CursorCodec.java": "3875011a04eb47fe28d709c4ccf5ade53ffa9cbe813a68539d7161cc2bb90947", + "src/main/java/ru/mtusi/messenger/util/TokenGenerator.java": "08297b5805b2d71c5af62d93ac3c805c1096ec025ae182485e406bdd5c04e037", + "src/main/java/ru/mtusi/messenger/util/TokenHasher.java": "564ae39c89ebf48478460fa9cc326930f4ba6a8f976d856dd75a44c3f886eacc", + "src/main/java/ru/mtusi/messenger/util/converter/IdempotencyPayloadConverter.java": "6f706047c050f5cbfdf61082a1ce6c74595c0e5416b2c5b32ef588c92173aef6", + "src/main/java/ru/mtusi/messenger/validation/AttachmentValidator.java": "bc0a335f7d32b6b659c42cdd24858b8817e1558b338f5f8c075f8375c4ee6b15", + "src/main/java/ru/mtusi/messenger/validation/ChatValidator.java": "e2379205ab9e96db74187d3353d4192c05bead99c7fbf509ff6fe5d033109f18", + "src/main/java/ru/mtusi/messenger/validation/IdempotencyValidator.java": "1cd9c1af4d2ba0f56eedccfb5ddc4d27928da57b0cf2918b16099df26bfe7cb4", + "src/test/java/ru/mtusi/messenger/AttachmentIntegrationTest.java": "71c9d0e6e34e16b45d0add29a3b2aabaec88980c9723ae5cfdb671327789e9ef", + "src/test/java/ru/mtusi/messenger/MessengerLab1ApplicationTests.java": "e949d72c057d7457b3747dd962df7732eb092074c014008ee69bdc8e36123ca2", + "src/main/resources/application.yaml": "ef414162f182cd68410acb1f07f39d8200c3a489ae1daa6b307c3768be26e6bf", + "src/main/resources/db/migration/V1__init.sql": "9f15aaf1bc1949bc19dec830314c976477d9af4d55b8554f1913f84ccb4dae58", + "src/main/resources/db/migration/V2__attachment_integrity.sql": "232232d2845c5aca2977dc504e682bb90c2ab112f264de2f2a54ad337ea8c982", + "scripts/bootstrap.ps1": "219760fe8b02c0cbdcb740da306cf1c47247b5b0455fd6912dd335f3ef65426c", + "scripts/check_contract.py": "61e300fe5c1074360418e24b4653b12f96808247a2beaf2a64859aa360f81493", + "scripts/restart.cmd": "a8b170bd67efd9fd2cbd0db837984f826cf0f09c9b73754798b79ff481b1d589", + "scripts/restart.example.sh": "7cbfc7f518b975ee3ca37a645359ff734ab0686789b795a9ac845789eb46fbd9", + "scripts/restart.ps1": "72eb5e70fd4cf2e1891dddffd85ca560ae82f066f77d953f532d145be1800f8a", + "scripts/restart.sh": "d17a08a9d4cc7af1c594757f3f2bfbec7c15bf11d36393070fc915af30fa8dcc", + "scripts/stop-one.cmd": "b636287c36f4c8f52e7840513506565bab0a1d480bb9187fdb40d1c0101616fa", + "scripts/stop-one.example.sh": "6467c6e1515dee72982f85bced3f3862601ae0dbfafd4fd4818ef420f1dea040", + "scripts/stop-one.ps1": "c60f70430dbea3916c68011ad5ccbaf329a44a43f1e2d55f33597e4a495cc0b6", + "scripts/stop-one.sh": "fec1a774f4387676b8b6adf2398a41e7b0108a0bc745103a3079b23acfe6f668", + "scripts/validate_openapi.py": "cd7a33221b1860c5d2330e52afa6cb222d375db671ae9ab379ba51a0d3a77946", + "tests/aborted.py": "ef6ce22c66ccefc80d3c09322f4041624653194a2a119b7a21e6826899d8a196", + "tests/client.py": "b318084e2a4c380db91a55d082190bf33979fa556a4c4ca87e01f46ae275775c", + "tests/extended.py": "c816ba34a8421c70f5aa7e80d40bf229704481d9d548f540882c897d59d5fcd1", + "tests/faults.py": "0732d7a87c208351f49b887e4ef92a304c344da9252c11bef15db6994e82a259", + "tests/image_probe.py": "ef48d8882ab138b9699a173e7419e5814c22a10b01b6b0ddbb50ea49481a5baa", + "tests/load.py": "9a9aff05e109630e243c3b092d7bd564344e4872b29e38b6618647a08460a72e", + "tests/locking.py": "cf607f8bb078e1b8ca0fac81844ff7eb0a170a409d110ebf020355966c6892f4", + "tests/resilience.py": "4ba30b9c81a0664127ddab2b0bea5e76aa2087be8bb25aa13cd5a8e69be23fdd", + "tests/smoke.py": "354e168e1e644145c3ff50bbbe189fd217cce5659aa125f588098025ffab0fdf", + "tests/track2.py": "03c3eec04d71f6a8ebc4a9bd665f0d3c972955fe78c27b58ce4419c101544473", + "Dockerfile": "f460b0ff20112ea354aa406514a883dccd19f811503ae2e31695490bdf08345f", + "docker-compose.yml": "030357a3883f2c2fba3fc131c15d085a2fa2f62da16a3d8b4c63fe6a75ae043a", + "nginx.conf": "5d7753018d2e6a3c141963d932571382abdffe45869c1e98bc58dda015e6fd0f", + "build.gradle": "ac066d5f1d3b537db1eb401e88a97cfd9e2d9b86ed9e612cc44f7a74ccd5079c", + "settings.gradle": "eb9d73971105b2fae07689265bee6764673d5cfe1bf6d24a2561120fef94ad00", + "policy.json": "df8ef175cfca7a4d20d835fd605db8c7de1e96eea2a1b1fa4b8eb8b091ef2f66" +} \ No newline at end of file diff --git a/evidence/stop-one-after.txt b/evidence/stop-one-after.txt new file mode 100644 index 0000000..f76da00 --- /dev/null +++ b/evidence/stop-one-after.txt @@ -0,0 +1 @@ +/mtusi2026lab4-api-2 3217054c4e74 exited diff --git a/evidence/stop-one-before.txt b/evidence/stop-one-before.txt new file mode 100644 index 0000000..8fd6c1d --- /dev/null +++ b/evidence/stop-one-before.txt @@ -0,0 +1 @@ +/mtusi2026lab4-api-2 3217054c4e74 running diff --git a/evidence/track2.txt b/evidence/track2.txt new file mode 100644 index 0000000..8711c14 --- /dev/null +++ b/evidence/track2.txt @@ -0,0 +1,3 @@ +PASS dry-run: old orphan and missing metadata detected; no mutations +PASS apply: orphan removed, missing marked BROKEN, fresh object retained +PASS repeat is idempotent; referenced attachment and bytes retained diff --git a/evidence/unit-summary.json b/evidence/unit-summary.json new file mode 100644 index 0000000..7f1be69 --- /dev/null +++ b/evidence/unit-summary.json @@ -0,0 +1,18 @@ +[ + { + "name": "ru.mtusi.messenger.AttachmentIntegrationTest", + "tests": "3", + "failures": "0", + "errors": "0", + "skipped": "0", + "time": "5.344" + }, + { + "name": "ru.mtusi.messenger.MessengerLab1ApplicationTests", + "tests": "1", + "failures": "0", + "errors": "0", + "skipped": "0", + "time": "0.179" + } +] \ No newline at end of file diff --git a/evidence/unit.txt b/evidence/unit.txt new file mode 100644 index 0000000..c16a3f6 --- /dev/null +++ b/evidence/unit.txt @@ -0,0 +1,13 @@ +To honour the JVM settings for this build a single-use Daemon process will be forked. For more on this, please refer to https://docs.gradle.org/9.3.0/userguide/gradle_daemon.html#sec:disabling_the_daemon in the Gradle documentation. +Daemon will be stopped at the end of the build +> Task :compileJava +> Task :processResources UP-TO-DATE +> Task :classes +> Task :compileTestJava UP-TO-DATE +> Task :processTestResources NO-SOURCE +> Task :testClasses UP-TO-DATE +> Task :test + +BUILD SUCCESSFUL in 34s +4 actionable tasks: 2 executed, 2 up-to-date +Consider enabling configuration cache to speed up this build: https://docs.gradle.org/9.3.0/userguide/configuration_cache_enabling.html diff --git a/evidence/upload-before-tuning.json b/evidence/upload-before-tuning.json new file mode 100644 index 0000000..5b4ec4c --- /dev/null +++ b/evidence/upload-before-tuning.json @@ -0,0 +1,11 @@ +{ + "uploads": 20, + "concurrency": 10, + "image_bytes": 8674413, + "sampled_peak_MiB": { + "mtusi2026lab4-api-1": 502.3, + "mtusi2026lab4-api-2": 491.4 + }, + "elapsed_seconds": 10.5, + "limitation": "discrete docker stats samples; true peak may be higher" +} \ No newline at end of file diff --git a/evidence/upload-before-tuning.txt b/evidence/upload-before-tuning.txt new file mode 100644 index 0000000..6b9c21e --- /dev/null +++ b/evidence/upload-before-tuning.txt @@ -0,0 +1,12 @@ +PASS negative: MIME, corrupt, oversized, filename, duplicate/unknown/cross-chat attachments +PASS 24 simultaneous idempotent requests across 2 replicas; one message +image_bytes= 8674413 memory_samples= ['mtusi2026lab4-api-1 485.9MiB / 512MiB\nmtusi2026lab4-api-2 453.3MiB / 512MiB\nmtusi2026lab4-lb-1 38.87MiB / 7.649GiB\nmtusi2026lab4-minio-1 124.2MiB / 7.649GiB\nmtusi2026lab4-db-1 55.26MiB / 1GiB\nmtusi2026lab4-redis-1 5.652MiB / 7.649GiB\n', 'mtusi2026lab4-api-1 496MiB / 512MiB\nmtusi2026lab4-api-2 491.4MiB / 512MiB\nmtusi2026lab4-lb-1 35.27MiB / 7.649GiB\nmtusi2026lab4-minio-1 125.5MiB / 7.649GiB\nmtusi2026lab4-db-1 56.24MiB / 1GiB\nmtusi2026lab4-redis-1 5.652MiB / 7.649GiB\n', 'mtusi2026lab4-api-1 502.3MiB / 512MiB\nmtusi2026lab4-api-2 489.2MiB / 512MiB\nmtusi2026lab4-lb-1 24.9MiB / 7.649GiB\nmtusi2026lab4-minio-1 127MiB / 7.649GiB\nmtusi2026lab4-db-1 56.78MiB / 1GiB\nmtusi2026lab4-redis-1 5.652MiB / 7.649GiB\n', 'mtusi2026lab4-api-1 494.9MiB / 512MiB\nmtusi2026lab4-api-2 484.5MiB / 512MiB\nmtusi2026lab4-lb-1 9.027MiB / 7.649GiB\nmtusi2026lab4-minio-1 128.7MiB / 7.649GiB\nmtusi2026lab4-db-1 56.84MiB / 1GiB\nmtusi2026lab4-redis-1 5.652MiB / 7.649GiB\n', 'mtusi2026lab4-api-1 492.6MiB / 512MiB\nmtusi2026lab4-api-2 485.2MiB / 512MiB\nmtusi2026lab4-lb-1 8.957MiB / 7.649GiB\nmtusi2026lab4-minio-1 128.8MiB / 7.649GiB\nmtusi2026lab4-db-1 57.18MiB / 1GiB\nmtusi2026lab4-redis-1 5.652MiB / 7.649GiB\n'] +PASS 20 uploads concurrency=10 elapsed_seconds= 10.5 +mtusi2026lab4-api-1 492.6MiB / 512MiB +mtusi2026lab4-api-2 484.8MiB / 512MiB +mtusi2026lab4-lb-1 8.957MiB / 7.649GiB +mtusi2026lab4-minio-1 128.8MiB / 7.649GiB +mtusi2026lab4-db-1 57.17MiB / 1GiB +mtusi2026lab4-redis-1 5.902MiB / 7.649GiB + +PASS logout across replicas diff --git a/evidence/upload-summary.json b/evidence/upload-summary.json new file mode 100644 index 0000000..3751819 --- /dev/null +++ b/evidence/upload-summary.json @@ -0,0 +1,12 @@ +{ + "uploads": 20, + "concurrency": 10, + "image_bytes": 8674413, + "heap_MiB": 320, + "sampled_peak_MiB": { + "mtusi2026lab4-api-2": 454.1, + "mtusi2026lab4-api-1": 454.8 + }, + "elapsed_seconds": 12.547, + "limitation": "discrete docker stats; true peak may be higher" +} \ No newline at end of file diff --git a/nginx.conf b/nginx.conf index b8be5fc..a290974 100644 --- a/nginx.conf +++ b/nginx.conf @@ -21,11 +21,13 @@ http { listen 80; server_name _; - client_max_body_size 1m; + client_max_body_size 0; + client_body_timeout 15s; location / { proxy_pass http://api_backend; proxy_http_version 1.1; + proxy_request_buffering off; proxy_set_header Connection ""; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-Host $http_host; @@ -34,7 +36,7 @@ http { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; - proxy_next_upstream error timeout http_502 http_503 http_504 non_idempotent; + proxy_next_upstream error timeout http_502 http_504; proxy_next_upstream_tries 3; proxy_connect_timeout 1s; diff --git a/scripts/bootstrap.ps1 b/scripts/bootstrap.ps1 new file mode 100644 index 0000000..f6b6258 --- /dev/null +++ b/scripts/bootstrap.ps1 @@ -0,0 +1,12 @@ +$ErrorActionPreference = 'Stop' +New-Item -ItemType Directory -Force secrets | Out-Null +$values = @{postgres_db='messenger'; postgres_user='messenger'; postgres_password=''; minio_root_user='labadmin'; minio_root_password=''; s3_access_key_id='labapi'; s3_secret_access_key=''} +foreach ($name in $values.Keys) { + $path = Join-Path secrets ($name + '.txt') + if (!(Test-Path -LiteralPath $path)) { + $value = $values[$name] + if (!$value) { $value = [Convert]::ToBase64String([System.Security.Cryptography.RandomNumberGenerator]::GetBytes(32)) } + [System.IO.File]::WriteAllText((Join-Path (Get-Location) $path), $value) + } +} +if (!(Test-Path .env)) { Copy-Item .env.example .env } diff --git a/scripts/restart.cmd b/scripts/restart.cmd new file mode 100644 index 0000000..e71fe90 --- /dev/null +++ b/scripts/restart.cmd @@ -0,0 +1 @@ +@powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0restart.ps1" diff --git a/scripts/restart.ps1 b/scripts/restart.ps1 new file mode 100644 index 0000000..369254a --- /dev/null +++ b/scripts/restart.ps1 @@ -0,0 +1,9 @@ +$ErrorActionPreference = 'Stop' +$ids = docker compose ps -q +$before = docker inspect --format '{{.Name}} {{.State.StartedAt}}' $ids +$before | Set-Content evidence/restart-before.txt +docker compose restart db redis minio api lb +if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE } +$ids = docker compose ps -q +$after = docker inspect --format '{{.Name}} {{.State.StartedAt}}' $ids +$after | Set-Content evidence/restart-after.txt diff --git a/scripts/restart.sh b/scripts/restart.sh new file mode 100644 index 0000000..cbc094c --- /dev/null +++ b/scripts/restart.sh @@ -0,0 +1,3 @@ +#!/bin/sh +set -eu +docker compose restart db redis minio api lb diff --git a/scripts/stop-one.cmd b/scripts/stop-one.cmd new file mode 100644 index 0000000..dcdcb84 --- /dev/null +++ b/scripts/stop-one.cmd @@ -0,0 +1 @@ +@powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0stop-one.ps1" diff --git a/scripts/stop-one.ps1 b/scripts/stop-one.ps1 new file mode 100644 index 0000000..27ce510 --- /dev/null +++ b/scripts/stop-one.ps1 @@ -0,0 +1,13 @@ +$ErrorActionPreference = 'Stop' +$replicas = docker compose ps -q api +foreach ($replica in $replicas) { + $alias = docker inspect --format '{{.Config.Hostname}}' $replica + if ($alias -eq $env:TARGET_INSTANCE_ID) { + docker inspect --format "{{.Name}} {{.Config.Hostname}} {{.State.Status}}" $replica | Set-Content evidence/stop-one-before.txt + docker stop --time 2 $replica + $result = $LASTEXITCODE + docker inspect --format "{{.Name}} {{.Config.Hostname}} {{.State.Status}}" $replica | Set-Content evidence/stop-one-after.txt + exit $result + } +} +exit 1 diff --git a/scripts/stop-one.sh b/scripts/stop-one.sh new file mode 100644 index 0000000..60c335f --- /dev/null +++ b/scripts/stop-one.sh @@ -0,0 +1,9 @@ +#!/bin/sh +set -eu +for id in $(docker compose ps -q api); do + if [ "$(docker inspect --format '{{.Config.Hostname}}' "$id")" = "$TARGET_INSTANCE_ID" ]; then + docker stop -t 2 "$id" + exit 0 + fi +done +exit 1 diff --git a/src/main/java/ru/mtusi/messenger/config/JacksonConfig.java b/src/main/java/ru/mtusi/messenger/config/JacksonConfig.java index 585c537..197c495 100644 --- a/src/main/java/ru/mtusi/messenger/config/JacksonConfig.java +++ b/src/main/java/ru/mtusi/messenger/config/JacksonConfig.java @@ -19,6 +19,7 @@ public class JacksonConfig { return builder -> builder .enable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) .withCoercionConfigDefaults(config -> { + config.setCoercion(CoercionInputShape.Boolean, CoercionAction.Fail); config.setCoercion(CoercionInputShape.Integer, CoercionAction.Fail); config.setCoercion(CoercionInputShape.Float, CoercionAction.Fail); }); diff --git a/src/main/java/ru/mtusi/messenger/config/MinioConfig.java b/src/main/java/ru/mtusi/messenger/config/MinioConfig.java index 390bff3..2efe971 100644 --- a/src/main/java/ru/mtusi/messenger/config/MinioConfig.java +++ b/src/main/java/ru/mtusi/messenger/config/MinioConfig.java @@ -15,8 +15,13 @@ public class MinioConfig { public MinioClient minioClient() { return MinioClient.builder() .endpoint(props.endpoint()) + .httpClient(new okhttp3.OkHttpClient.Builder() + .connectTimeout(java.time.Duration.ofSeconds(2)) + .readTimeout(java.time.Duration.ofSeconds(10)) + .writeTimeout(java.time.Duration.ofSeconds(10)) + .callTimeout(java.time.Duration.ofSeconds(20)).build()) .region(props.region()) - .credentials(props.accessKey(), props.secretKey()) + .credentials(props.resolvedAccessKey(), props.resolvedSecretKey()) .build(); } } \ No newline at end of file diff --git a/src/main/java/ru/mtusi/messenger/config/properties/S3Properties.java b/src/main/java/ru/mtusi/messenger/config/properties/S3Properties.java index 61d075c..a474334 100644 --- a/src/main/java/ru/mtusi/messenger/config/properties/S3Properties.java +++ b/src/main/java/ru/mtusi/messenger/config/properties/S3Properties.java @@ -8,9 +8,32 @@ public record S3Properties( String endpoint, String bucket, @DefaultValue("us-east-1") String region, - String accessKey, - String secretKey, + @DefaultValue("") String accessKey, + @DefaultValue("") String secretKey, @DefaultValue("true") boolean pathStyleAccess, @DefaultValue("300") int presignExpirySeconds ) { + public String resolvedAccessKey() { + var file = System.getenv("S3_ACCESS_KEY_ID_FILE"); + if (file != null && !file.isBlank()) { + try { + return new String(java.nio.file.Files.readAllBytes(java.nio.file.Paths.get(file))).strip(); + } catch (Exception e) { + throw new RuntimeException("Failed to read S3_ACCESS_KEY_ID_FILE: " + file, e); + } + } + return accessKey; + } + + public String resolvedSecretKey() { + var file = System.getenv("S3_SECRET_ACCESS_KEY_FILE"); + if (file != null && !file.isBlank()) { + try { + return new String(java.nio.file.Files.readAllBytes(java.nio.file.Paths.get(file))).strip(); + } catch (Exception e) { + throw new RuntimeException("Failed to read S3_SECRET_ACCESS_KEY_FILE: " + file, e); + } + } + return secretKey; + } } \ No newline at end of file diff --git a/src/main/java/ru/mtusi/messenger/controller/AttachmentController.java b/src/main/java/ru/mtusi/messenger/controller/AttachmentController.java index 2e0a0b2..df9ed7a 100644 --- a/src/main/java/ru/mtusi/messenger/controller/AttachmentController.java +++ b/src/main/java/ru/mtusi/messenger/controller/AttachmentController.java @@ -1,5 +1,6 @@ package ru.mtusi.messenger.controller; +import jakarta.servlet.http.HttpServletRequest; import lombok.RequiredArgsConstructor; import org.springframework.http.HttpHeaders; import org.springframework.http.MediaType; @@ -19,8 +20,8 @@ public class AttachmentController implements AttachmentsApi { private final AttachmentService service; @Override - public ResponseEntity uploadAttachment(UUID chatId, String filename, byte[] body, @AuthenticationPrincipal SessionPrincipal principal) { - return ResponseEntity.status(201).body(service.upload(chatId, principal.userId(), filename, body)); + public ResponseEntity uploadAttachment(UUID chatId, String filename, HttpServletRequest body, String contentType, @AuthenticationPrincipal SessionPrincipal principal) { + return ResponseEntity.status(201).body(service.upload(chatId, principal.userId(), filename, body, contentType)); } @Override @@ -30,19 +31,6 @@ public class AttachmentController implements AttachmentsApi { @Override public ResponseEntity downloadAttachment(UUID attachmentId, String variant, SessionPrincipal principal) { - var downloaded = service.download(attachmentId, principal.userId(), variant); - - byte[] bytes; - try (var in = downloaded.stream()) { - bytes = in.readAllBytes(); - } catch (Exception e) { - throw new RuntimeException("Failed to read attachment stream", e); - } - - return ResponseEntity.ok() - .contentType(MediaType.parseMediaType(downloaded.contentType())) - .header(HttpHeaders.CONTENT_DISPOSITION, - "attachment; filename=\"" + downloaded.filename() + "\"") - .body(bytes); + return service.downloadResponse(attachmentId, principal.userId(), variant); } -} \ No newline at end of file +} diff --git a/src/main/java/ru/mtusi/messenger/controller/AttachmentsApi.java b/src/main/java/ru/mtusi/messenger/controller/AttachmentsApi.java index 59b5ac9..fd8981e 100644 --- a/src/main/java/ru/mtusi/messenger/controller/AttachmentsApi.java +++ b/src/main/java/ru/mtusi/messenger/controller/AttachmentsApi.java @@ -1,5 +1,6 @@ package ru.mtusi.messenger.controller; +import jakarta.servlet.http.HttpServletRequest; import io.swagger.v3.oas.annotations.Operation; import io.swagger.v3.oas.annotations.Parameter; import io.swagger.v3.oas.annotations.enums.ParameterIn; @@ -71,7 +72,8 @@ public interface AttachmentsApi { @PathVariable("chat_id") UUID chatId, @Parameter(name = "filename", required = true, in = ParameterIn.QUERY) @Size(min = 1, max = 255) @RequestParam("filename") String filename, - @RequestBody byte[] body, + HttpServletRequest body, + @RequestHeader("Content-Type") String contentType, @AuthenticationPrincipal SessionPrincipal principal ); @@ -154,8 +156,7 @@ public interface AttachmentsApi { ) @RequestMapping( method = RequestMethod.GET, - value = "/api/v1/attachments/{attachment_id}/content", - produces = { "image/png", "image/jpeg", "image/webp" } + value = "/api/v1/attachments/{attachment_id}/content" ) ResponseEntity downloadAttachment( @Parameter(name = "attachment_id", required = true, in = ParameterIn.PATH) diff --git a/src/main/java/ru/mtusi/messenger/controller/controllerAdvice/GlobalExceptionHandler.java b/src/main/java/ru/mtusi/messenger/controller/controllerAdvice/GlobalExceptionHandler.java index 36a3eaa..ccad7a8 100644 --- a/src/main/java/ru/mtusi/messenger/controller/controllerAdvice/GlobalExceptionHandler.java +++ b/src/main/java/ru/mtusi/messenger/controller/controllerAdvice/GlobalExceptionHandler.java @@ -26,6 +26,7 @@ import ru.mtusi.messenger.exception.*; @RestControllerAdvice @RequiredArgsConstructor public class GlobalExceptionHandler { + private static final org.slf4j.Logger log = org.slf4j.LoggerFactory.getLogger(GlobalExceptionHandler.class); private final RateLimitProperties limitProperties; private final ApiErrorFactory errors; @@ -113,6 +114,7 @@ public class GlobalExceptionHandler { @ExceptionHandler(Exception.class) public ResponseEntity fallback(Exception e, HttpServletRequest req) { + log.error("Unexpected API failure type={}", e.getClass().getName()); return respond(HttpStatus.INTERNAL_SERVER_ERROR, ErrorErrorDTO.CodeEnum.INTERNAL_ERROR, "Internal error", req); } diff --git a/src/main/java/ru/mtusi/messenger/dto/CreateMessageDTO.java b/src/main/java/ru/mtusi/messenger/dto/CreateMessageDTO.java index 718cb0f..42238b6 100644 --- a/src/main/java/ru/mtusi/messenger/dto/CreateMessageDTO.java +++ b/src/main/java/ru/mtusi/messenger/dto/CreateMessageDTO.java @@ -6,13 +6,18 @@ import jakarta.validation.constraints.NotNull; import jakarta.validation.constraints.Size; import org.hibernate.validator.constraints.CodePointLength; +import java.util.ArrayList; +import java.util.List; import java.util.Objects; +import java.util.UUID; @Schema(name = "CreateMessage") public class CreateMessageDTO { private String text; + private List attachmentIds = new ArrayList<>(); + public CreateMessageDTO() {} public CreateMessageDTO text(String text) { @@ -21,20 +26,26 @@ public class CreateMessageDTO { } @NotNull - @CodePointLength(min = 1, max = 4000) @Schema(name = "text", requiredMode = Schema.RequiredMode.REQUIRED) @JsonProperty("text") public String getText() { return text; } public void setText(String text) { this.text = text; } + @Schema(name = "attachment_ids", requiredMode = Schema.RequiredMode.NOT_REQUIRED) + @JsonProperty("attachment_ids") + public List getAttachmentIds() { return attachmentIds; } + + public void setAttachmentIds(List attachmentIds) { this.attachmentIds = attachmentIds; } + @Override public boolean equals(Object o) { if (this == o) return true; if (!(o instanceof CreateMessageDTO d)) return false; - return Objects.equals(text, d.text); + return Objects.equals(text, d.text) && + Objects.equals(attachmentIds, d.attachmentIds); } @Override - public int hashCode() { return Objects.hash(text); } + public int hashCode() { return Objects.hash(text, attachmentIds); } } \ No newline at end of file diff --git a/src/main/java/ru/mtusi/messenger/dto/MessageDTO.java b/src/main/java/ru/mtusi/messenger/dto/MessageDTO.java index 703b3ac..03d44d4 100644 --- a/src/main/java/ru/mtusi/messenger/dto/MessageDTO.java +++ b/src/main/java/ru/mtusi/messenger/dto/MessageDTO.java @@ -9,6 +9,8 @@ import org.hibernate.validator.constraints.CodePointLength; import java.time.Instant; import java.util.UUID; +import java.util.ArrayList; +import java.util.List; /** * MessageDTO @@ -26,6 +28,8 @@ public class MessageDTO { private Instant createdAt; + private List attachmentIds = new ArrayList<>(); + public MessageDTO() { super(); } @@ -41,6 +45,15 @@ public class MessageDTO { this.createdAt = createdAt; } + public MessageDTO(UUID id, UUID chatId, UUID senderId, String text, Instant createdAt, List attachmentIds) { + this.id = id; + this.chatId = chatId; + this.senderId = senderId; + this.text = text; + this.createdAt = createdAt; + this.attachmentIds = attachmentIds != null ? attachmentIds : new ArrayList<>(); + } + public MessageDTO id(UUID id) { this.id = id; return this; @@ -142,6 +155,17 @@ public class MessageDTO { this.createdAt = createdAt; } + @Valid + @Schema(name = "attachment_ids", requiredMode = Schema.RequiredMode.REQUIRED) + @JsonProperty("attachment_ids") + public List getAttachmentIds() { + return attachmentIds; + } + + public void setAttachmentIds(List attachmentIds) { + this.attachmentIds = attachmentIds; + } + @Override public boolean equals(Object o) { if (this == o) { @@ -155,12 +179,13 @@ public class MessageDTO { Objects.equals(this.chatId, messageDTO.chatId) && Objects.equals(this.senderId, messageDTO.senderId) && Objects.equals(this.text, messageDTO.text) && - Objects.equals(this.createdAt, messageDTO.createdAt); + Objects.equals(this.createdAt, messageDTO.createdAt) && + Objects.equals(this.attachmentIds, messageDTO.attachmentIds); } @Override public int hashCode() { - return Objects.hash(id, chatId, senderId, text, createdAt); + return Objects.hash(id, chatId, senderId, text, createdAt, attachmentIds); } @Override @@ -172,6 +197,7 @@ public class MessageDTO { sb.append(" senderId: ").append(toIndentedString(senderId)).append("\n"); sb.append(" text: ").append(toIndentedString(text)).append("\n"); sb.append(" createdAt: ").append(toIndentedString(createdAt)).append("\n"); + sb.append(" attachmentIds: ").append(toIndentedString(attachmentIds)).append("\n"); sb.append("}"); return sb.toString(); } diff --git a/src/main/java/ru/mtusi/messenger/mapper/AttachmentMapper.java b/src/main/java/ru/mtusi/messenger/mapper/AttachmentMapper.java index 1c28796..b5b63cc 100644 --- a/src/main/java/ru/mtusi/messenger/mapper/AttachmentMapper.java +++ b/src/main/java/ru/mtusi/messenger/mapper/AttachmentMapper.java @@ -17,9 +17,9 @@ public interface AttachmentMapper { @Mapping(target = "ownerId", source = "owner.id") @Mapping(target = "sha256", source = "checksum") @Mapping(target = "status", source = "status", qualifiedByName = "mapStatus") - @Mapping(target = "contentType", source = "contentType") + @Mapping(target = "contentType", source = "contentType", qualifiedByName = "mapContentType") @Mapping(target = "variants", expression = "java(emptyVariants())") - @Mapping(target = "errorCode", constant = "null") + @Mapping(target = "errorCode", expression = "java((String) null)") AttachmentDTO toDto(Attachment entity); @Named("mapStatus") @@ -27,7 +27,15 @@ public interface AttachmentMapper { if (status == AttachmentStatus.READY) { return AttachmentDTO.StatusEnum.READY; } - return null; + return AttachmentDTO.StatusEnum.READY; + } + + @Named("mapContentType") + default AttachmentDTO.ContentTypeEnum mapContentType(String contentType) { + if (contentType == null) { + return null; + } + return AttachmentDTO.ContentTypeEnum.fromValue(contentType); } default java.util.List emptyVariants() { diff --git a/src/main/java/ru/mtusi/messenger/mapper/MessageMapper.java b/src/main/java/ru/mtusi/messenger/mapper/MessageMapper.java index 393f23e..2eb5ddf 100644 --- a/src/main/java/ru/mtusi/messenger/mapper/MessageMapper.java +++ b/src/main/java/ru/mtusi/messenger/mapper/MessageMapper.java @@ -13,11 +13,18 @@ public interface MessageMapper { @Mapping(target = "chat", ignore = true) @Mapping(target = "sender", ignore = true) @Mapping(target = "createdAt", ignore = true) + @Mapping(target = "attachmentIds", ignore = true) Message toMessage(MessageDTO dto); @Mapping(target = "chatId", source = "chat.id") @Mapping(target = "senderId", source = "sender.id") + @Mapping(target = "attachmentIds", source = "attachmentIds") MessageDTO toMessageDTO(Message entity); + @Mapping(target = "id", ignore = true) + @Mapping(target = "chat", ignore = true) + @Mapping(target = "sender", ignore = true) + @Mapping(target = "createdAt", ignore = true) + @Mapping(target = "attachmentIds", source = "attachmentIds") Message toMessage(CreateMessageDTO dto); } \ No newline at end of file diff --git a/src/main/java/ru/mtusi/messenger/model/Message.java b/src/main/java/ru/mtusi/messenger/model/Message.java index b4a368d..c02c175 100644 --- a/src/main/java/ru/mtusi/messenger/model/Message.java +++ b/src/main/java/ru/mtusi/messenger/model/Message.java @@ -5,6 +5,8 @@ import lombok.*; import java.time.Instant; import java.time.temporal.ChronoUnit; +import java.util.ArrayList; +import java.util.List; import java.util.UUID; @Entity @@ -33,6 +35,12 @@ public class Message { @Column(name = "created_at", nullable = false, updatable = false) private Instant createdAt; + @ElementCollection + @CollectionTable(name = "message_attachments", joinColumns = @JoinColumn(name = "message_id")) + @Column(name = "attachment_id") + @OrderColumn(name = "attachment_order") + private List attachmentIds = new ArrayList<>(); + @PrePersist private void onCreate() { this.createdAt = Instant.now().truncatedTo(ChronoUnit.MILLIS); diff --git a/src/main/java/ru/mtusi/messenger/repository/AttachmentRepository.java b/src/main/java/ru/mtusi/messenger/repository/AttachmentRepository.java index 15b8a0f..eede767 100644 --- a/src/main/java/ru/mtusi/messenger/repository/AttachmentRepository.java +++ b/src/main/java/ru/mtusi/messenger/repository/AttachmentRepository.java @@ -16,6 +16,7 @@ import java.util.List; import java.util.UUID; public interface AttachmentRepository extends JpaRepository { + boolean existsByObjectKey(String objectKey); @Lock(LockModeType.PESSIMISTIC_WRITE) @QueryHints({ diff --git a/src/main/java/ru/mtusi/messenger/service/AttachmentReconciler.java b/src/main/java/ru/mtusi/messenger/service/AttachmentReconciler.java index 0cd40bf..b789b7e 100644 --- a/src/main/java/ru/mtusi/messenger/service/AttachmentReconciler.java +++ b/src/main/java/ru/mtusi/messenger/service/AttachmentReconciler.java @@ -1,27 +1,19 @@ package ru.mtusi.messenger.service; import io.minio.*; -import io.minio.messages.Item; -import io.minio.Result; +import io.minio.errors.ErrorResponseException; import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; -import net.javacrumbs.shedlock.spring.annotation.SchedulerLock; import org.springframework.data.domain.PageRequest; import org.springframework.scheduling.annotation.Scheduled; import org.springframework.stereotype.Component; +import org.springframework.transaction.annotation.Transactional; import ru.mtusi.messenger.config.properties.ReconcilerProperties; import ru.mtusi.messenger.config.properties.S3Properties; import ru.mtusi.messenger.model.Attachment; import ru.mtusi.messenger.model.AttachmentStatus; import ru.mtusi.messenger.repository.AttachmentRepository; - import java.time.Instant; -import java.time.temporal.ChronoUnit; -import java.util.ArrayList; -import java.util.HashSet; -import java.util.List; -import java.util.Set; -import java.util.UUID; @Component @RequiredArgsConstructor @@ -31,274 +23,63 @@ public class AttachmentReconciler { private final MinioClient minio; private final S3Properties s3Props; private final ReconcilerProperties reconcilerProps; + private final org.springframework.jdbc.core.JdbcTemplate jdbc; @Scheduled(fixedDelayString = "${reconciler.interval-ms:3600000}") - @SchedulerLock(name = "reconciler", - lockAtMostFor = "${reconciler.lock-at-most-for:30m}", - lockAtLeastFor = "${reconciler.lock-at-least-for:1m}" - ) + @Transactional public void reconcile() { - if (!reconcilerProps.enabled()) { - log.debug("Reconciler is disabled, skipping"); - return; - } - executeReconciliation(); + if (!reconcilerProps.enabled()) return; + Boolean acquired = jdbc.queryForObject("select pg_try_advisory_xact_lock(741204)", Boolean.class); + if (!Boolean.TRUE.equals(acquired)) return; + long started = System.nanoTime(); + scanObjects(); + scanMetadata(); + log.info("Reconcile complete dryRun={} elapsedMs={}", reconcilerProps.dryRun(), (System.nanoTime()-started)/1000000); } - private void executeReconciliation() { - if (reconcilerProps.dryRun()) { - log.info("Reconciler DRY-RUN: would process stale attachments, orphaned S3 objects, and verify READY"); - return; - } - log.info("Reconciliation cycle started"); - long start = System.currentTimeMillis(); - reconcileStaleAttachments(); - reconcileOrphanedS3Objects(); - verifyReadyAttachments(); - - long duration = System.currentTimeMillis() - start; - log.info("Reconciliation cycle completed in {}ms", duration); + private Instant cutoff() { + return Instant.now().minusSeconds(reconcilerProps.gracePeriodMinutes() * 60L); } - void reconcileStaleAttachments() { - Instant staleBefore = Instant.now() - .truncatedTo(ChronoUnit.MILLIS) - .minus(reconcilerProps.gracePeriodMinutes(), ChronoUnit.MINUTES); - - List stale = findStaleAttachments(staleBefore); - if (stale.isEmpty()) { - log.info("No stale UPLOADING attachments found"); - return; - } - - log.info("Found {} stale UPLOADING attachments", stale.size()); - for (Attachment attachment : stale) { - processStaleAttachment(attachment); + private void scanObjects() { + try { + for (var result : minio.listObjects(ListObjectsArgs.builder().bucket(s3Props.bucket()).recursive(true).build())) { + var item = result.get(); + if (item.lastModified().toInstant().isBefore(cutoff()) + && !attachmentRepository.existsByObjectKey(item.objectName())) { + log.info("Reconcile orphan dryRun={} key={}", reconcilerProps.dryRun(), item.objectName()); + if (!reconcilerProps.dryRun()) remove(item.objectName()); + } + } + } catch (Exception e) { + throw new ru.mtusi.messenger.exception.StorageUnavailableException("Reconcile storage unavailable", e); } } - private List findStaleAttachments(Instant before) { - List result = new ArrayList<>(); - int pageSize = reconcilerProps.batchSize(); + private void scanMetadata() { int page = 0; - - List batch = attachmentRepository.findStaleForUpdateSkipLocked(AttachmentStatus.UPLOADING, before, PageRequest.of(page, pageSize)); - - while (!batch.isEmpty()) { - result.addAll(batch); - log.debug("Loaded batch #{} of {} stale attachments", page + 1, batch.size()); - - if (batch.size() < pageSize) { - break; - } - page++; - batch = attachmentRepository - .findStaleForUpdateSkipLocked(AttachmentStatus.UPLOADING, before, - org.springframework.data.domain.PageRequest.of(page, pageSize)); - } - return result; + org.springframework.data.domain.Page batch; + do { + batch = attachmentRepository.findAll(PageRequest.of(page++, reconcilerProps.batchSize(), org.springframework.data.domain.Sort.by("id"))); + batch.forEach(this::verify); + } while (batch.hasNext()); } - /** - * Обрабатывает одну зависшую запись: удаляет из S3 и помечает как BROKEN. - */ - void processStaleAttachment(Attachment attachment) { - String objectKey = attachment.getObjectKey(); - log.warn("Processing stale attachment id={} objectKey={} chatId={}", - attachment.getId(), objectKey, attachment.getChat().getId()); - - removeObjectFromS3(objectKey, attachment.getId()); - markAsBroken(attachment.getId(), AttachmentStatus.UPLOADING); - } - - // --------------------------------------------------------------- - // Step 2: orphaned S3 objects - // --------------------------------------------------------------- - - /** - * Находит файлы в S3, которых нет в БД, и удаляет их. - * - *

Алгоритм: - * 1. Получаем все objectKey из БД. - * 2. Перечисляем все объекты в S3-бакете. - * 3. Объекты, которых нет в БД и старше порога — удаляем.

- */ - void reconcileOrphanedS3Objects() { - String bucket = s3Props.bucket(); - log.info("Checking for orphaned S3 objects in bucket={}", bucket); - - Set knownObjectKeys = collectKnownObjectKeys(); - log.info("Known object keys in DB: {}", knownObjectKeys.size()); - - int removed = 0; + private void verify(Attachment attachment) { + if (attachment.getStatus() == AttachmentStatus.BROKEN) return; + if (!attachment.getCreatedAt().isBefore(cutoff())) return; try { - Iterable> items = minio.listObjects( - ListObjectsArgs.builder() - .bucket(bucket) - .recursive(true) - .build()); - - for (Result result : items) { - Item item = result.get(); - if (isOrphaned(item, knownObjectKeys)) { - removeOrphanedObject(bucket, item.objectName()); - removed++; - } - } + minio.statObject(StatObjectArgs.builder().bucket(s3Props.bucket()).object(attachment.getObjectKey()).build()); + } catch (ErrorResponseException e) { + if (!"NoSuchKey".equals(e.errorResponse().code())) throw new IllegalStateException("S3 verification failed", e); + log.info("Reconcile missing dryRun={} id={}", reconcilerProps.dryRun(), attachment.getId()); + if (!reconcilerProps.dryRun()) attachment.setStatus(AttachmentStatus.BROKEN); } catch (Exception e) { - log.error("Error scanning S3 for orphaned objects: {}", e.getMessage(), e); - return; - } - - log.info("Removed {} orphaned S3 objects", removed); - } - - private Set collectKnownObjectKeys() { - Set keys = new HashSet<>(); - List all = attachmentRepository.findAll(); - for (Attachment a : all) { - keys.add(a.getObjectKey()); - } - return keys; - } - - private boolean isOrphaned(Item item, Set knownKeys) { - String objectName = item.objectName(); - if (knownKeys.contains(objectName)) { - return false; - } - Instant objectAge = item.lastModified().toInstant(); - Instant staleThreshold = Instant.now() - .minusSeconds(reconcilerProps.gracePeriodMinutes() * 60L); - return objectAge.isBefore(staleThreshold); - } - - private void removeOrphanedObject(String bucket, String objectName) { - try { - minio.removeObject(RemoveObjectArgs.builder() - .bucket(bucket) - .object(objectName) - .build()); - log.info("Removed orphaned S3 object: {}", objectName); - } catch (Exception e) { - log.error("Failed to remove orphaned S3 object {}: {}", - objectName, e.getMessage()); + throw new IllegalStateException("S3 verification failed", e); } } - // --------------------------------------------------------------- - // Step 3: verify READY attachments exist in S3 - // --------------------------------------------------------------- - - /** - * Проверяет, что все READY-записи имеют соответствующие файлы в S3. - * Если файл отсутствует — меняет статус на BROKEN. - */ - void verifyReadyAttachments() { - log.info("Verifying READY attachments against S3"); - - int pageSize = reconcilerProps.batchSize(); - int page = 0; - int verified = 0; - int broken = 0; - - while (true) { - List batch = attachmentRepository - .findReadyForReconciliation(AttachmentStatus.READY, - org.springframework.data.domain.PageRequest.of(page, pageSize)); - - if (batch.isEmpty()) { - break; - } - - for (Attachment attachment : batch) { - if (verifySingleAttachment(attachment)) { - verified++; - } else { - broken++; - } - } - - if (batch.size() < pageSize) { - break; - } - page++; - } - - log.info("Verification complete: verified={}, broken={}", verified, broken); - } - - /** - * Проверяет одну запись: существует ли файл в S3. - * Возвращает true если файл найден, false если нет - * (и помечает как BROKEN). - */ - boolean verifySingleAttachment(Attachment attachment) { - String objectKey = attachment.getObjectKey(); - String bucket = s3Props.bucket(); - - try { - minio.statObject(StatObjectArgs.builder() - .bucket(bucket) - .object(objectKey) - .build()); - return true; - } catch (io.minio.errors.ErrorResponseException e) { - if ("NoSuchKey".equals(e.errorResponse().code())) { - log.warn("S3 object missing for READY attachment id={} objectKey={}", - attachment.getId(), objectKey); - markAsBroken(attachment.getId(), AttachmentStatus.READY); - return false; - } - log.error("Error checking S3 object {}: {}", objectKey, e.getMessage()); - return true; - } catch (Exception e) { - log.error("Error checking S3 object {}: {}", objectKey, e.getMessage()); - return true; - } - } - - // --------------------------------------------------------------- - // Shared helpers - // --------------------------------------------------------------- - - /** - * Безопасно удаляет объект из S3 (best-effort, max retries). - */ - private void removeObjectFromS3(String objectKey, UUID attachmentId) { - String bucket = s3Props.bucket(); - int maxRetries = reconcilerProps.maxS3Retries(); - for (int attempt = 1; attempt <= maxRetries; attempt++) { - try { - minio.removeObject(RemoveObjectArgs.builder() - .bucket(bucket) - .object(objectKey) - .build()); - log.info("Removed stale S3 object attempt={}/{} objectKey={} attachmentId={}", - attempt, maxRetries, objectKey, attachmentId); - return; - } catch (Exception e) { - log.warn("S3 remove attempt {}/{} failed for objectKey={}: {}", - attempt, maxRetries, objectKey, e.getMessage()); - if (attempt == maxRetries) { - log.error("All {} attempts failed for objectKey={}, attachment will remain as orphan", - maxRetries, objectKey); - } - } - } - } - - /** - * Меняет статус на BROKEN с optimistic check (WHERE status = oldStatus). - */ - private void markAsBroken(UUID id, AttachmentStatus currentStatus) { - int updated = attachmentRepository.updateStatus( - id, currentStatus, AttachmentStatus.BROKEN); - if (updated > 0) { - log.info("Marked attachment {} as BROKEN (was {})", id, currentStatus); - } else { - log.warn("Attachment {} already transitioned out of {} status, skipping", - id, currentStatus); - } + private void remove(String key) throws Exception { + minio.removeObject(RemoveObjectArgs.builder().bucket(s3Props.bucket()).object(key).build()); } } diff --git a/src/main/java/ru/mtusi/messenger/service/AttachmentService.java b/src/main/java/ru/mtusi/messenger/service/AttachmentService.java index 149bd9a..5576739 100644 --- a/src/main/java/ru/mtusi/messenger/service/AttachmentService.java +++ b/src/main/java/ru/mtusi/messenger/service/AttachmentService.java @@ -1,11 +1,25 @@ package ru.mtusi.messenger.service; +import ru.mtusi.messenger.exception.ConflictException; +import ru.mtusi.messenger.exception.ServiceUnavailableException; +import ru.mtusi.messenger.exception.UnsupportedMediaTypeException; +import ru.mtusi.messenger.exception.PayloadTooLargeException; +import ru.mtusi.messenger.exception.InvalidRequestException; +import java.io.InputStream; +import java.io.IOException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.Semaphore; +import java.nio.charset.StandardCharsets; +import org.springframework.http.ContentDisposition; +import org.springframework.http.MediaType; +import org.springframework.http.ResponseEntity; +import jakarta.servlet.http.HttpServletRequest; +import org.springframework.jdbc.core.JdbcTemplate; import io.minio.*; import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; import org.springframework.dao.DataAccessException; import org.springframework.stereotype.Service; -import org.springframework.transaction.annotation.Propagation; import org.springframework.transaction.annotation.Transactional; import ru.mtusi.messenger.config.properties.S3Properties; import ru.mtusi.messenger.dto.AttachmentDTO; @@ -27,6 +41,8 @@ import java.util.UUID; @RequiredArgsConstructor @Slf4j public class AttachmentService { + private final Semaphore uploads = new Semaphore(1, true); + private final JdbcTemplate jdbc; private final UserService userService; private final ChatService chatService; private final MinioClient minio; @@ -35,16 +51,49 @@ public class AttachmentService { private final AttachmentValidator validator; private final AttachmentMapper attachmentMapper; - public AttachmentDTO upload(UUID chatId, UUID caller, String filename, byte[] body) { + @Transactional + public AttachmentDTO upload(UUID chatId, UUID caller, String filename, HttpServletRequest request, String declaredType) { + acquireUploadSlot(); + try { + return performUpload(chatId, caller, filename, request, declaredType); + } finally { + uploads.release(); + } + } + + private void acquireUploadSlot() { + try { + if (!uploads.tryAcquire(30, TimeUnit.SECONDS)) + throw new ServiceUnavailableException("Upload capacity exhausted"); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new ServiceUnavailableException("Upload interrupted"); + } + } + + private AttachmentDTO performUpload(UUID chatId, UUID caller, String filename, + HttpServletRequest request, String declaredType) { + jdbc.execute("select pg_advisory_xact_lock_shared(741204)"); Chat chat = chatService.getChat(caller, chatId); - String contentType = validator.validate(body, filename); String safeFilename = validator.sanitizeFilename(filename); + byte[] body = readBounded(request); + String contentType = validator.validate(body, safeFilename); + if (!contentType.equals(declaredType)) throw new UnsupportedMediaTypeException("MIME mismatch"); String objectKey = UUID.randomUUID().toString(); String checksum = uploadToS3(body, objectKey, contentType); return saveAttachmentReady(objectKey, safeFilename, contentType, body, checksum, chat, caller); } - @Transactional(propagation = Propagation.REQUIRES_NEW) + private byte[] readBounded(HttpServletRequest request) { + try { + byte[] bytes = request.getInputStream().readNBytes(10485761); + if (bytes.length > 10485760) throw new PayloadTooLargeException("File exceeds limit"); + return bytes; + } catch (IOException e) { + throw new InvalidRequestException("Incomplete upload"); + } + } + AttachmentDTO saveAttachmentReady(String objectKey, String safeFilename, String contentType, byte[] body, String checksum, Chat chat, UUID caller) { try { Attachment attachment = buildAttachment(objectKey, safeFilename, contentType, body, checksum, chat, caller); @@ -60,12 +109,14 @@ public class AttachmentService { @Transactional(readOnly = true) public AttachmentDTO getMetadata(UUID attachmentId, UUID userId) { Attachment attachment = requireAttachmentForMember(attachmentId, userId); + if (attachment.getStatus() != AttachmentStatus.READY) throw new ConflictException("Attachment unavailable"); return attachmentMapper.toDto(attachment); } public DownloadedAttachment download(UUID attachmentId, UUID userId, String variant) { Attachment attachment = requireAttachmentForMember(attachmentId, userId); assertOriginalVariant(variant); + if (attachment.getStatus() != AttachmentStatus.READY) throw new ConflictException("Attachment not ready"); return streamFromS3(attachment); } @@ -109,14 +160,14 @@ public class AttachmentService { private Attachment requireAttachmentForMember(UUID attachmentId, UUID userId) { Attachment attachment = attachmentRepository.findById(attachmentId) - .orElseThrow(() -> new ru.mtusi.messenger.exception.NotFoundException("Attachment not found")); + .orElseThrow(() -> new NotFoundException("Attachment not found")); assertMember(attachment.getChat().getId(), userId); return attachment; } private void assertOriginalVariant(String variant) { if (!"original".equals(variant)) { - throw new ru.mtusi.messenger.exception.NotFoundException("Variant not found"); + throw new NotFoundException("Variant not found"); } } @@ -150,5 +201,19 @@ public class AttachmentService { } } - public record DownloadedAttachment(java.io.InputStream stream, String contentType, String filename) {} + @Transactional(readOnly = true) + public ResponseEntity downloadResponse(UUID id, UUID user, String variant) { + var file = download(id, user, variant); + try (var in = file.stream()) { + return ResponseEntity.ok() + .contentType(MediaType.parseMediaType(file.contentType())) + .header("Content-Disposition", ContentDisposition.attachment() + .filename(file.filename(), StandardCharsets.UTF_8).build().toString()) + .body(in.readNBytes(10485761)); + } catch (IOException e) { + throw new StorageUnavailableException("Download failed", e); + } + } + + public record DownloadedAttachment(InputStream stream, String contentType, String filename) {} } \ No newline at end of file diff --git a/src/main/java/ru/mtusi/messenger/service/HealthService.java b/src/main/java/ru/mtusi/messenger/service/HealthService.java index 60decf5..8a43a79 100644 --- a/src/main/java/ru/mtusi/messenger/service/HealthService.java +++ b/src/main/java/ru/mtusi/messenger/service/HealthService.java @@ -15,10 +15,18 @@ public class HealthService { private final JdbcTemplate jdbcTemplate; private final DataSource dataSource; + private final io.minio.MinioClient minio; + private final ru.mtusi.messenger.config.properties.S3Properties s3; public HealthDTO readiness() { requirePoolAvailable(); requireDatabaseReachable(); + try { + if (!minio.bucketExists(io.minio.BucketExistsArgs.builder().bucket(s3.bucket()).build())) + throw new ServiceUnavailableException("Storage unavailable"); + } catch (Exception e) { + throw new ServiceUnavailableException("Storage unavailable"); + } return ok(); } diff --git a/src/main/java/ru/mtusi/messenger/service/MessageService.java b/src/main/java/ru/mtusi/messenger/service/MessageService.java index d6d2ea0..01f9029 100644 --- a/src/main/java/ru/mtusi/messenger/service/MessageService.java +++ b/src/main/java/ru/mtusi/messenger/service/MessageService.java @@ -1,5 +1,8 @@ package ru.mtusi.messenger.service; +import ru.mtusi.messenger.model.AttachmentStatus; +import ru.mtusi.messenger.repository.AttachmentRepository; +import ru.mtusi.messenger.exception.ConflictException; import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; import org.springframework.dao.DataIntegrityViolationException; @@ -28,6 +31,7 @@ import java.util.UUID; @Slf4j @RequiredArgsConstructor public class MessageService { + private final AttachmentRepository attachments; private final IdempotencyService idempotencyService; private final MessageRepository messageRepository; private final ChatService chatService; @@ -57,9 +61,21 @@ public class MessageService { } private void validateCreate(UUID chatId, CreateMessageDTO dto) { - if (dto.getText() == null || dto.getText().isBlank()) { + if (dto.getText() == null || (dto.getText().isBlank() && (dto.getAttachmentIds() == null || dto.getAttachmentIds().isEmpty()))) { throw new InvalidRequestException("Empty text"); } + var ids = dto.getAttachmentIds(); + if (ids == null || ids.size() > 10 || ids.contains(null) || new java.util.HashSet<>(ids).size() != ids.size() + || dto.getText().codePointCount(0, dto.getText().length()) > 4000) { + throw new InvalidRequestException("Invalid message"); + } + for (UUID id : ids) { + var attachment = attachments.findById(id).filter(a -> a.getChat().getId().equals(chatId)) + .orElseThrow(() -> new NotFoundException("Attachment not found")); + if (attachment.getStatus() != AttachmentStatus.READY) { + throw new ConflictException("Attachment not ready"); + } + } if (messageRepository.countByChatId(chatId) >= appProperties.maxMessagesPerChat()) { throw new PayloadTooLargeException("Chat history limit reached"); } diff --git a/src/main/java/ru/mtusi/messenger/service/SessionCacheService.java b/src/main/java/ru/mtusi/messenger/service/SessionCacheService.java index 8573e42..4499259 100644 --- a/src/main/java/ru/mtusi/messenger/service/SessionCacheService.java +++ b/src/main/java/ru/mtusi/messenger/service/SessionCacheService.java @@ -22,7 +22,6 @@ public class SessionCacheService { private final CacheManager cacheManager; private final CacheHealthTracker cacheHealthTracker; - @Cacheable(value = SESSIONS, key = "#hex", unless = "#result == null") @Transactional(readOnly = true) public SessionService.SessionRecord lookupByHash(String hex) { byte[] hash = hexToBytes(hex); diff --git a/src/main/java/ru/mtusi/messenger/validation/AttachmentValidator.java b/src/main/java/ru/mtusi/messenger/validation/AttachmentValidator.java index edc94e5..79f6cf9 100644 --- a/src/main/java/ru/mtusi/messenger/validation/AttachmentValidator.java +++ b/src/main/java/ru/mtusi/messenger/validation/AttachmentValidator.java @@ -2,7 +2,6 @@ package ru.mtusi.messenger.validation; import lombok.RequiredArgsConstructor; import org.springframework.stereotype.Component; -import org.springframework.web.HttpMediaTypeNotSupportedException; import ru.mtusi.messenger.config.properties.UploadProperties; import ru.mtusi.messenger.exception.InvalidRequestException; import ru.mtusi.messenger.exception.PayloadTooLargeException; @@ -10,10 +9,10 @@ import ru.mtusi.messenger.exception.UnsupportedMediaTypeException; import javax.imageio.ImageIO; import javax.imageio.ImageReader; -import javax.imageio.stream.ImageInputStream; +import javax.imageio.stream.MemoryCacheImageInputStream; import java.io.ByteArrayInputStream; import java.io.IOException; -import java.util.Iterator; +import java.util.Locale; @Component @RequiredArgsConstructor @@ -24,49 +23,54 @@ public class AttachmentValidator { if (bytes.length > props.maxBytes()) { throw new PayloadTooLargeException("File exceeds max size"); } - sanitizeFilename(filename); - - try (ByteArrayInputStream bais = new ByteArrayInputStream(bytes); - ImageInputStream iis = ImageIO.createImageInputStream(bais)) { - - Iterator readers = ImageIO.getImageReaders(iis); + try (var input = new MemoryCacheImageInputStream(new ByteArrayInputStream(bytes))) { + var readers = ImageIO.getImageReaders(input); if (!readers.hasNext()) { throw new UnsupportedMediaTypeException("Only PNG/JPEG allowed"); } - ImageReader reader = readers.next(); - reader.setInput(iis, true, true); - - String format = reader.getFormatName().toLowerCase(); - String contentType = switch (format) { - case "png" -> "image/png"; - case "jpeg", "jpg" -> "image/jpeg"; - default -> throw new UnsupportedMediaTypeException("Only PNG/JPEG allowed"); - }; - - long pixels = (long) reader.getWidth(0) * reader.getHeight(0); - if (pixels > props.maxPixels()) { - throw new PayloadTooLargeException("Image dimensions exceed limit"); + try { + reader.setInput(input, true, true); + return inspect(reader); + } finally { + reader.dispose(); } - - return contentType; - } catch (IOException e) { throw new UnsupportedMediaTypeException("Corrupted or unsupported image"); } } - public String sanitizeFilename(String filename) { - if (filename == null || filename.isBlank()) { - throw new InvalidRequestException("Filename is blank"); - } - String cleaned = filename - .replaceAll("[\\x00-\\x1F/\\\\]", "_") - .replaceAll("^\\.+", ""); + private String inspect(ImageReader reader) throws IOException { + String mime = contentType(reader.getFormatName()); + validateDimensions(reader.getWidth(0), reader.getHeight(0)); + reader.read(0); + return mime; + } - if (cleaned.isBlank() || cleaned.codePointCount(0, cleaned.length()) > 255) { + private String contentType(String format) { + return switch (format.toLowerCase(Locale.ROOT)) { + case "png" -> "image/png"; + case "jpeg", "jpg" -> "image/jpeg"; + default -> throw new UnsupportedMediaTypeException("Only PNG/JPEG allowed"); + }; + } + + private void validateDimensions(int width, int height) { + if (width <= 0 || height <= 0) { + throw new UnsupportedMediaTypeException("Invalid image dimensions"); + } + if ((long) width * height > props.maxPixels()) { + throw new PayloadTooLargeException("Image dimensions exceed limit"); + } + } + + public String sanitizeFilename(String filename) { + if (filename == null || filename.isBlank() + || filename.codePointCount(0, filename.length()) > 255 + || filename.contains("/") || filename.contains("\\") + || filename.codePoints().anyMatch(Character::isISOControl)) { throw new InvalidRequestException("Invalid filename"); } - return cleaned; + return filename; } -} \ No newline at end of file +} diff --git a/src/main/resources/application.yaml b/src/main/resources/application.yaml index 49d0322..d3618d1 100644 --- a/src/main/resources/application.yaml +++ b/src/main/resources/application.yaml @@ -6,7 +6,7 @@ spring: lifecycle: timeout-per-shutdown-phase: 30s jackson: - default-property-inclusion: non_null + default-property-inclusion: always datasource: driver-class-name: org.postgresql.Driver url: ${SPRING_DATASOURCE_URL:jdbc:postgresql://localhost:5432/messenger} @@ -120,4 +120,4 @@ logging: level: org.springframework.cache: TRACE org.springframework.cache.interceptor: TRACE - software.amazon.awssdk: DEBUG \ No newline at end of file + software.amazon.awssdk: DEBUG diff --git a/src/main/resources/db/migration/V1__init.sql b/src/main/resources/db/migration/V1__init.sql index 11be68b..2bd17a7 100644 --- a/src/main/resources/db/migration/V1__init.sql +++ b/src/main/resources/db/migration/V1__init.sql @@ -109,4 +109,13 @@ CREATE INDEX idx_attachments_object_key ON attachments(object_key); CREATE INDEX idx_attachments_chat_id ON attachments(chat_id); CREATE INDEX idx_attachments_owner_id ON attachments(owner_id); CREATE INDEX idx_attachments_status ON attachments(status); -CREATE INDEX idx_attachments_created_at ON attachments(created_at); \ No newline at end of file +CREATE INDEX idx_attachments_created_at ON attachments(created_at); + +CREATE TABLE message_attachments ( + message_id UUID NOT NULL REFERENCES messages(id) ON DELETE CASCADE, + attachment_id UUID NOT NULL, + attachment_order INTEGER NOT NULL, + PRIMARY KEY (message_id, attachment_order) +); + +CREATE INDEX idx_message_attachments_attachment_id ON message_attachments(attachment_id); \ No newline at end of file diff --git a/src/main/resources/db/migration/V2__attachment_integrity.sql b/src/main/resources/db/migration/V2__attachment_integrity.sql new file mode 100644 index 0000000..4bb8e4b --- /dev/null +++ b/src/main/resources/db/migration/V2__attachment_integrity.sql @@ -0,0 +1,2 @@ +ALTER TABLE messages DROP CONSTRAINT IF EXISTS ck_messages_text_not_blank; +ALTER TABLE message_attachments ADD CONSTRAINT fk_message_attachment FOREIGN KEY (attachment_id) REFERENCES attachments(id); diff --git a/src/test/java/ru/mtusi/messenger/AttachmentIntegrationTest.java b/src/test/java/ru/mtusi/messenger/AttachmentIntegrationTest.java index f9cfd8b..9b34b86 100644 --- a/src/test/java/ru/mtusi/messenger/AttachmentIntegrationTest.java +++ b/src/test/java/ru/mtusi/messenger/AttachmentIntegrationTest.java @@ -1,317 +1,38 @@ -package ru.mtusi.messenger.service; +package ru.mtusi.messenger; -import io.minio.*; -import lombok.extern.slf4j.Slf4j; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import ru.mtusi.messenger.config.properties.ReconcilerProperties; -import ru.mtusi.messenger.config.properties.S3Properties; -import ru.mtusi.messenger.dto.AttachmentDTO; -import ru.mtusi.messenger.model.Attachment; -import ru.mtusi.messenger.model.AttachmentStatus; -import ru.mtusi.messenger.repository.AttachmentRepository; - -import java.io.ByteArrayInputStream; -import java.time.Duration; -import java.time.Instant; +import ru.mtusi.messenger.config.properties.UploadProperties; +import ru.mtusi.messenger.validation.AttachmentValidator; +import ru.mtusi.messenger.exception.*; import java.util.List; -import java.util.UUID; -import java.util.concurrent.CountDownLatch; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; +import java.nio.file.Files; +import java.nio.file.Path; +import static org.assertj.core.api.Assertions.*; -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -/** - * Интеграционные тесты для Grade 4 и Track 2. - * - *

Grade 4: тесты на переполнение, повреждённые файлы, подмену MIME, - * чужие attachment_id, привязку к чату.

- * - *

Track 2: error injection, reconciler detection/repair, - * proof fresh attachments aren't deleted, traversal cost measurement.

- */ -@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) -@Slf4j class AttachmentIntegrationTest { - - @Autowired - private AttachmentService attachmentService; - - @Autowired - private AttachmentRepository attachmentRepository; - - @Autowired - private MinioClient minio; - - @Autowired - private S3Properties s3Props; - - private String bucket; - - @BeforeEach - void setUp() { - bucket = s3Props.bucket(); - attachmentRepository.deleteAll(); - } - - // ================================================================ - // Grade 4: Validation and error handling tests - // ================================================================ + private final AttachmentValidator validator = new AttachmentValidator( + new UploadProperties(10485760, 40000000, List.of("image/png", "image/jpeg"))); @Test - @DisplayName("Grade 4: Reject file exceeding max size (10 MiB)") - void shouldRejectOversizedFile() { - byte[] oversized = new byte[11 * 1024 * 1024]; - System.arraycopy(new byte[]{(byte) 0x89, 0x50, 0x4E, 0x47}, 0, oversized, 0, 4); - - assertThatThrownBy(() -> - attachmentService.upload( - UUID.randomUUID(), UUID.randomUUID(), "big.png", oversized)) - .isInstanceOf(Exception.class); + void acceptsRealFixture() throws Exception { + assertThat(validator.validate(Files.readAllBytes(Path.of("tests/fixtures/sample.png")), "sample.png")) + .isEqualTo("image/png"); } @Test - @DisplayName("Grade 4: Reject corrupted image (valid header, corrupted body)") - void shouldRejectCorruptedImage() { - byte[] corrupted = new byte[1024]; - System.arraycopy(new byte[]{(byte) 0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A}, 0, corrupted, 0, 8); - - assertThatThrownBy(() -> - attachmentService.upload( - UUID.randomUUID(), UUID.randomUUID(), "corrupted.png", corrupted)) - .isInstanceOf(Exception.class); - } - - @Test - @DisplayName("Grade 4: MIME checked by content not filename") - void shouldCheckMimeByContent() { - byte[] jpegData = new byte[]{ - (byte) 0xFF, (byte) 0xD8, (byte) 0xFF, (byte) 0xE0, - 0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00, 0x01, - 0x01, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00 - }; - - AttachmentDTO att = attachmentService.upload( - UUID.randomUUID(), UUID.randomUUID(), "fake.png", jpegData); - assertThat(att.getContentType()).isEqualTo("image/jpeg"); - } - - @Test - @DisplayName("Grade 4: Reject foreign attachment access (user not in chat)") - void shouldRejectForeignAttachmentAccess() { - UUID chatId = UUID.randomUUID(); - UUID ownerId = UUID.randomUUID(); - UUID outsiderId = UUID.randomUUID(); - - byte[] pngData = createMinimalPng(); - AttachmentDTO att = attachmentService.upload(chatId, ownerId, "test.png", pngData); - - assertThatThrownBy(() -> - attachmentService.getMetadata(att.getId(), outsiderId)) - .isInstanceOf(Exception.class); - } - - // ================================================================ - // Track 2: Error injection and reconciler tests - // ================================================================ - - @Test - @DisplayName("Track 2: Reconciler detects and repairs stale UPLOADING attachment") - void shouldDetectAndRepairStaleAttachment() { - // Inject: create attachment with UPLOADING status and old timestamp - Attachment stale = new Attachment(); - String orphanKey = UUID.randomUUID().toString(); - stale.setObjectKey(orphanKey); - stale.setFilename("stale.png"); - stale.setContentType("image/png"); - stale.setSizeBytes(100L); - stale.setChecksum("abc123"); - stale.setStatus(AttachmentStatus.UPLOADING); - stale.setCreatedAt(Instant.now().minus(Duration.ofMinutes(20))); - stale.setUpdatedAt(stale.getCreatedAt()); - attachmentRepository.save(stale); - - // Inject: create orphan in S3 - try { - minio.putObject(PutObjectArgs.builder() - .bucket(bucket) - .object(orphanKey) - .stream(new ByteArrayInputStream(new byte[100]), 100L, -1L) - .build()); - } catch (Exception e) { - log.warn("Failed to inject orphan: {}", e.getMessage()); - } - - // Run reconciler - AttachmentReconciler reconciler = new AttachmentReconciler( - attachmentRepository, minio, s3Props, - new ReconcilerProperties(true, 3600000L, 15, false, 100, 3, "30m", "1m")); - - reconciler.reconcileStaleAttachments(); - - // Verify: stale attachment is now BROKEN - Attachment updated = attachmentRepository.findAll().stream() - .filter(a -> a.getObjectKey().equals(orphanKey)) - .findFirst().orElse(null); - if (updated != null) { - assertThat(updated.getStatus()).isEqualTo(AttachmentStatus.BROKEN); + void rejectsControlCharactersWithoutSilentlyRenaming() { + for (String name : List.of("a/b.png", "a\\b.png", "a\n.png", "a\u007f.png")) { + assertThatThrownBy(() -> validator.sanitizeFilename(name)).isInstanceOf(InvalidRequestException.class); } + assertThat(validator.sanitizeFilename("photo..png")).isEqualTo("photo..png"); } @Test - @DisplayName("Track 2: Reconciler dry-run does not modify anything") - void shouldNotModifyInDryRun() { - Attachment stale = new Attachment(); - stale.setObjectKey(UUID.randomUUID().toString()); - stale.setFilename("stale.png"); - stale.setContentType("image/png"); - stale.setSizeBytes(100L); - stale.setChecksum("abc"); - stale.setStatus(AttachmentStatus.UPLOADING); - stale.setCreatedAt(Instant.now().minus(Duration.ofMinutes(20))); - stale.setUpdatedAt(stale.getCreatedAt()); - attachmentRepository.save(stale); - - int beforeCount = (int) attachmentRepository.count(); - - // Run reconciler in dry-run mode - AttachmentReconciler reconciler = new AttachmentReconciler( - attachmentRepository, minio, s3Props, - new ReconcilerProperties(true, 3600000L, 15, true, 100, 3, "30m", "1m")); - - reconciler.reconcileStaleAttachments(); - - // Verify: nothing changed - assertThat(attachmentRepository.count()).isEqualTo(beforeCount); - assertThat(stale.getStatus()).isEqualTo(AttachmentStatus.UPLOADING); - } - - @Test - @DisplayName("Track 2: Reconciler does not delete fresh attachments") - void shouldNotDeleteFreshAttachments() { - UUID chatId = UUID.randomUUID(); - UUID userId = UUID.randomUUID(); - byte[] pngData = createMinimalPng(); - - AttachmentDTO fresh = attachmentService.upload(chatId, userId, "fresh.png", pngData); - - // Run reconciler - AttachmentReconciler reconciler = new AttachmentReconciler( - attachmentRepository, minio, s3Props, - new ReconcilerProperties(true, 3600000L, 15, false, 100, 3, "30m", "1m")); - - reconciler.verifyReadyAttachments(); - - // Verify: fresh attachment still exists and is READY - Attachment updated = attachmentRepository.findById(fresh.getId()).orElse(null); - assertThat(updated).isNotNull(); - assertThat(updated.getStatus()).isEqualTo(AttachmentStatus.READY); - } - - @Test - @DisplayName("Track 2: Measure reconciler traversal cost") - void shouldMeasureTraversalCost() { - UUID chatId = UUID.randomUUID(); - UUID userId = UUID.randomUUID(); - int attachmentCount = 50; - - byte[] pngData = createMinimalPng(); - for (int i = 0; i < attachmentCount; i++) { - attachmentService.upload(chatId, userId, "test-" + i + ".png", pngData); - } - - AttachmentReconciler reconciler = new AttachmentReconciler( - attachmentRepository, minio, s3Props, - new ReconcilerProperties(true, 3600000L, 15, false, 100, 3, "30m", "1m")); - - long start = System.currentTimeMillis(); - reconciler.verifyReadyAttachments(); - long duration = System.currentTimeMillis() - start; - - log.info("Reconciler traversal cost: {}ms for {} attachments ({:.2f}ms per attachment)", - duration, attachmentCount, (double) duration / attachmentCount); - - // Verify: all attachments still READY - List all = attachmentRepository.findAll(); - assertThat(all).hasSize(attachmentCount); - assertThat(all.stream().allMatch(a -> a.getStatus() == AttachmentStatus.READY)).isTrue(); - } - - @Test - @DisplayName("Track 2: Concurrent uploads do not corrupt state") - void shouldHandleConcurrentUploads() throws InterruptedException { - UUID chatId = UUID.randomUUID(); - UUID userId = UUID.randomUUID(); - byte[] pngData = createMinimalPng(); - int threadCount = 10; - CountDownLatch startLatch = new CountDownLatch(1); - CountDownLatch doneLatch = new CountDownLatch(threadCount); - AtomicInteger successCount = new AtomicInteger(0); - AtomicInteger failureCount = new AtomicInteger(0); - - ExecutorService executor = Executors.newFixedThreadPool(threadCount); - - for (int i = 0; i < threadCount; i++) { - final int index = i; - executor.submit(() -> { - try { - startLatch.await(); - attachmentService.upload(chatId, userId, "concurrent-" + index + ".png", pngData); - successCount.incrementAndGet(); - } catch (Exception e) { - failureCount.incrementAndGet(); - log.error("Upload {} failed: {}", index, e.getMessage()); - } finally { - doneLatch.countDown(); - } - }); - } - - long before = System.currentTimeMillis(); - startLatch.countDown(); - doneLatch.await(30, TimeUnit.SECONDS); - long duration = System.currentTimeMillis() - before; - - executor.shutdown(); - executor.awaitTermination(10, TimeUnit.SECONDS); - - log.info("Concurrent uploads: {} success, {} failures, {}ms", - successCount.get(), failureCount.get(), duration); - - assertThat(failureCount.get()).isLessThanOrEqualTo(1); - assertThat(attachmentRepository.count()).isGreaterThanOrEqualTo(threadCount - 1); - } - - // ================================================================ - // Helpers - // ================================================================ - - private byte[] createMinimalPng() { - return new byte[]{ - (byte) 0x89, (byte) 0x50, (byte) 0x4E, (byte) 0x47, - (byte) 0x0D, (byte) 0x0A, (byte) 0x1A, (byte) 0x0A, - (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x0D, - (byte) 0x49, (byte) 0x48, (byte) 0x44, (byte) 0x52, - (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x01, - (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x01, - (byte) 0x08, (byte) 0x06, (byte) 0x00, (byte) 0x00, - (byte) 0x00, (byte) 0x1F, (byte) 0x15, (byte) 0xC4, - (byte) 0x89, (byte) 0x00, (byte) 0x00, (byte) 0x00, - (byte) 0x0A, (byte) 0x49, (byte) 0x44, (byte) 0x41, - (byte) 0x54, (byte) 0x78, (byte) 0x9C, (byte) 0x63, - (byte) 0x00, (byte) 0x01, (byte) 0x00, (byte) 0x00, - (byte) 0x05, (byte) 0x00, (byte) 0x01, (byte) 0x0D, - (byte) 0x0A, (byte) 0x2D, (byte) 0xB4, - (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00, - (byte) 0x49, (byte) 0x45, (byte) 0x4E, (byte) 0x44, - (byte) 0xAE, (byte) 0x42, (byte) 0x60, (byte) 0x82 - }; + void rejectsTruncatedImageAndOversizedBody() throws Exception { + byte[] png = Files.readAllBytes(Path.of("tests/fixtures/sample.png")); + assertThatThrownBy(() -> validator.validate(java.util.Arrays.copyOf(png, 40), "x.png")) + .isInstanceOf(UnsupportedMediaTypeException.class); + assertThatThrownBy(() -> validator.validate(new byte[10485761], "x.png")) + .isInstanceOf(PayloadTooLargeException.class); } } diff --git a/src/test/java/ru/mtusi/messenger/MessengerLab1ApplicationTests.java b/src/test/java/ru/mtusi/messenger/MessengerLab1ApplicationTests.java index 2da5d51..33eda5d 100644 --- a/src/test/java/ru/mtusi/messenger/MessengerLab1ApplicationTests.java +++ b/src/test/java/ru/mtusi/messenger/MessengerLab1ApplicationTests.java @@ -1,13 +1,18 @@ package ru.mtusi.messenger; import org.junit.jupiter.api.Test; -import org.springframework.boot.test.context.SpringBootTest; +import ru.mtusi.messenger.util.CursorCodec; +import java.time.Instant; +import java.util.UUID; +import static org.assertj.core.api.Assertions.*; -@SpringBootTest class MessengerLab1ApplicationTests { - - @Test - void contextLoads() { - } - + @Test + void cursorPreservesScopeAndBoundary() { + UUID scope = UUID.randomUUID(); + var boundary = new CursorCodec.PageBoundary(Instant.parse("2026-01-01T00:00:00Z"), UUID.randomUUID()); + var decoded = CursorCodec.decode(CursorCodec.encode(scope, boundary)); + assertThat(decoded.scopeId()).isEqualTo(scope); + assertThat(decoded.boundary()).isEqualTo(boundary); + } } diff --git a/tests/README.md b/tests/README.md index 0dfa3f8..4128c89 100644 --- a/tests/README.md +++ b/tests/README.md @@ -55,3 +55,20 @@ make test-security BASE_URL=https://localhost:8443 CA_FILE=/absolute/path/to/ca. ## Что ещё остаётся доказать Smoke не доказывает persistence, число реальных контейнеров, durability брокера, outbox, отсутствие утечек/уязвимостей, полноту telemetry или выполнение уровня 4/5. Конкурентные/нагрузочные/негативные проверки своего решения добавляйте отдельно. Список защиты текущей лабы находится в README.md. Проверки рассчитаны на localhost-стенд, не на production. + +## Дополнительные проверки реализации lab4 + +Windows: используйте `py` вместо `python3`. Все команды выполняются из корня. + +- `py tests/extended.py`: MIME, corrupt/oversize, filename, принадлежность вложений, 24 конкурентных повтора сообщения через две реплики, 20 PNG по ~8.3 MiB при concurrency=10, RSS samples и logout. +- `py tests/resilience.py --mode persistence --action-script scripts/restart.cmd`: перезапуск PostgreSQL, Redis, MinIO, обеих API и LB; прежняя сессия и checksum. На Linux используйте `scripts/restart.sh`. +- `py tests/resilience.py --mode failover --action-script scripts/stop-one.cmd`: остановка конкретной реплики; после проверки `docker compose up -d api`. На Linux используйте `.sh`. +- `py tests/track2.py`: только изолированный стенд. Перед запуском задайте `RECONCILER_INTERVAL_MS=3000`, `RECONCILER_GRACE_PERIOD_MINUTES=1`, `RECONCILER_DRY_RUN=true` и пересоздайте API (`docker compose up -d --no-deps api`). Скрипт создаёт orphan и missing, выдерживает grace, проверяет dry-run, пересоздаёт API с apply и проверяет повтор и сохранность свежего объекта. Вывод не содержит токенов. Тест требует Docker CLI и права оператора. После проверки верните переменные к defaults и пересоздайте API. + +Модульные проверки: JDK 21, `./gradlew test` (`gradlew.bat test` в Windows). Они не требуют работающих хранилищ. Black-box проверки работают с реальным PostgreSQL/S3, не с моками. `BASE_URL=http://127.0.0.1:8080` исключает задержки разрешения localhost на Windows. + +`py tests/faults.py`: private bucket, остановка/возврат S3, отказ INSERT метаданных через временный DB trigger; trigger удаляется в finally. Требует изолированного стенда и прав оператора. + +`py tests/locking.py`: задержка INSERT на 6s через временный DB trigger, проверка невидимости метаданных и запрета exclusive advisory lock в реальном окне PUT/COMMIT. Затем сверка bytes. `py tests/aborted.py`: частичное тело HTTP и разрыв соединения, сверка отсутствия изменений БД/S3. Запускать последовательно без других инъекций/нагрузки. Оба сценария предназначены для изолированного стенда. + +Пять замечаний review (06.11.2026): `py tests/review_regressions.py`, требуется Pillow. Синтетические PNG/JPEG, точные 33 bytes PNG, filename traversal, Message/ACL и PNG ровно 1770843 bytes; сравнение БД/S3 при отказе. BASE_URL задаётся окружением. При резервировании Windows порта 8080 используйте `docker compose -f docker-compose.yml -f compose.review.yaml up -d` и `$env:BASE_URL='http://127.0.0.1:18080'`. diff --git a/tests/aborted.py b/tests/aborted.py new file mode 100644 index 0000000..f6a1dfb --- /dev/null +++ b/tests/aborted.py @@ -0,0 +1,18 @@ +"""Interrupted client body must not produce metadata or S3 writes.""" +from client import Client, check, run +from track2 import sql, mc +import socket,time + +def main(): + c=Client(); a,b,e,chat=c.bootstrap() + before=sql('select count(*) from attachments;') + objects=mc('mc ls proof/messenger-attachments/') + with socket.create_connection(('127.0.0.1',8080),timeout=5) as connection: + headers=('POST /api/v1/chats/'+chat['id']+'/attachments?filename=partial.png HTTP/1.1\r\nHost: localhost\r\nAuthorization: Bearer '+a['token']+'\r\nContent-Type: image/png\r\nContent-Length: 1048576\r\n\r\n') + connection.sendall(headers.encode()+b'partial') + time.sleep(2) + check(sql('select count(*) from attachments;')==before,'partial metadata committed') + check(mc('mc ls proof/messenger-attachments/')==objects,'partial object stored') + print('PASS interrupted upload: connection closed before full body; metadata and S3 object listing unchanged') + +if __name__ == '__main__': run(main) diff --git a/tests/extended.py b/tests/extended.py new file mode 100644 index 0000000..10cf412 --- /dev/null +++ b/tests/extended.py @@ -0,0 +1,59 @@ +"""Additional contract and cross-replica concurrency tests. No secrets in output.""" +from client import Client, check, run +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path +import uuid, json, time, subprocess, threading, zlib, struct, os + +def main(): + c=Client(timeout=30) + a,b,e,chat=c.bootstrap() + c.call('POST','/api/v1/chats/'+chat['id']+'/messages',400,actor=a,body={'text':True},headers={'Idempotency-Key':uuid.uuid4().hex}) + data=Path('tests/fixtures/sample.png').read_bytes() + path='/api/v1/chats/'+chat['id']+'/attachments?filename=test.png' + def upload(raw=data, mime='image/png', expected=201, target=path): + return c.call('POST',target,expected,actor=a,raw=raw,headers={'Content-Type':mime},schema='Attachment' if expected==201 else None) + upload(data,'image/jpeg',415) + upload(data[:40],expected=415) + ihdr=struct.pack('>IIBBBBB',7000,7000,8,2,0,0,0) + bomb=b'\x89PNG\r\n\x1a\n'+struct.pack('>I',13)+b'IHDR'+ihdr+struct.pack('>I',zlib.crc32(b'IHDR'+ihdr)&0xffffffff) + upload(bomb,expected=413) + upload(b'x'*10485761,expected=413) + upload(target=path.replace('test.png','bad%7F.png'),expected=400) + att=upload()['json'] + other=c.call('POST','/api/v1/chats',201,actor=a,body={'title':'other','member_ids':[a['id']]},schema='Chat')['json'] + c.call('POST','/api/v1/chats/'+other['id']+'/messages',404,actor=a,body={'text':'x','attachment_ids':[att['id']]},headers={'Idempotency-Key':uuid.uuid4().hex}) + for ids in ([att['id']]*2,[str(uuid.uuid4())]): + c.call('POST','/api/v1/chats/'+chat['id']+'/messages',400 if len(ids)==2 else 404,actor=a,body={'text':'x','attachment_ids':ids},headers={'Idempotency-Key':uuid.uuid4().hex}) + print('PASS negative: MIME, corrupt, oversized, filename, duplicate/unknown/cross-chat attachments') + key=uuid.uuid4().hex + def message(_): + return c.call('POST','/api/v1/chats/'+chat['id']+'/messages',201,actor=a,body={'text':'','attachment_ids':[att['id']]},headers={'Idempotency-Key':key},schema='Message') + with ThreadPoolExecutor(max_workers=12) as pool: + replies=list(pool.map(message,range(24))) + check(len({r['json']['id'] for r in replies})==1,'duplicate message') + instances={r['headers']['X-Instance-Id'] for r in replies} + check(len(instances)>=2,'need two replicas') + print('PASS 24 simultaneous idempotent requests across',len(instances),'replicas; one message') + def chunk(kind, body): + return struct.pack('>I',len(body))+kind+body+struct.pack('>I',zlib.crc32(kind+body)&0xffffffff) + width=1700 + large=b'\x89PNG\r\n\x1a\n'+chunk(b'IHDR',struct.pack('>IIBBBBB',width,width,8,2,0,0,0))+chunk(b'IDAT',zlib.compress(b''.join(b'\0'+os.urandom(width*3) for _ in range(width))))+chunk(b'IEND',b'') + samples=[]; done=threading.Event() + def sample(): + while not done.is_set(): + samples.append(subprocess.check_output(['docker','stats','--no-stream','--format','{{.Name}} {{.MemUsage}}'],text=True)) + sampler=threading.Thread(target=sample); sampler.start() + with ThreadPoolExecutor(max_workers=10) as pool: + started=time.monotonic(); results=list(pool.map(lambda _:upload(large),range(20))) + check(len({r['json']['id'] for r in results})==20,'uploads collided') + done.set(); sampler.join() + print('image_bytes=',len(large),'memory_samples=',samples) + print('PASS 20 uploads concurrency=10 elapsed_seconds=',round(time.monotonic()-started,3)) + print(subprocess.check_output(['docker','stats','--no-stream','--format','{{.Name}} {{.MemUsage}}'],text=True)) + c.call('DELETE','/api/v1/auth/sessions/current',204,actor=a) + for _ in range(10): c.call('GET','/api/v1/users/me',401,actor=a) + print('PASS logout across replicas') + +if __name__ == '__main__': + run(main) + diff --git a/tests/faults.py b/tests/faults.py new file mode 100644 index 0000000..432ca9e --- /dev/null +++ b/tests/faults.py @@ -0,0 +1,33 @@ +"""Dependency and commit failure checks on a local isolated stand.""" +from client import Client, check, run +from track2 import sql, mc, command +import time, uuid + +def main(): + c=Client(timeout=30); a,b,e,chat=c.bootstrap() + att,data=c.attachment(a,chat) + key=sql("select object_key from attachments where id='"+att['id']+"';") + code=command(['docker','compose','exec','-T','minio','sh','-c','curl -s -o /dev/null -w "%{http_code}" http://localhost:9000/messenger-attachments/'+key]) + check(code=='403','bucket public') + print('PASS direct anonymous GET existing object returns 403',flush=True) + command(['docker','compose','stop','minio']) + try: + c.call('GET','/health/ready',503) + c.call('POST','/api/v1/chats/'+chat['id']+'/attachments?filename=x.png',503,actor=a,raw=data,headers={'Content-Type':'image/png'}) + print('PASS S3 outage: readiness and upload return 503',flush=True) + finally: command(['docker','compose','start','minio']) + time.sleep(3) + # DB trigger rejects metadata after successful PUT. Transaction rolls back; S3 compensation is tested. + before=sql('select count(*) from attachments;') + objects=mc('mc ls proof/messenger-attachments/') + sql("create function proof_reject() returns trigger language plpgsql as $$ begin raise exception 'injected'; end; $$; create trigger proof_reject before insert on attachments for each row execute function proof_reject();") + try: + c.call('POST','/api/v1/chats/'+chat['id']+'/attachments?filename=x.png',500,actor=a,raw=data,headers={'Content-Type':'image/png'}) + finally: + sql('drop trigger proof_reject on attachments; drop function proof_reject();') + check(sql('select count(*) from attachments;')==before,'metadata committed') + # saveAndFlush makes rejection observable before compensation, otherwise reconciler cleans after grace. + print('PASS DB failure: no success response, no committed metadata; orphan cleanup covered by track2',flush=True) + +if __name__ == '__main__': + run(main) diff --git a/tests/locking.py b/tests/locking.py new file mode 100644 index 0000000..a3b37dd --- /dev/null +++ b/tests/locking.py @@ -0,0 +1,27 @@ +"""Barrier experiment: probe DB fencing during PUT -> metadata commit window.""" +from client import Client, check, run +from track2 import sql +from concurrent.futures import ThreadPoolExecutor +import time + +def main(): + c=Client(timeout=30); a,b,e,chat=c.bootstrap() + before=sql('select count(*) from attachments;') + sql("create function proof_delay() returns trigger language plpgsql as $$ begin perform pg_sleep(6); return NEW; end; $$; create trigger proof_delay before insert on attachments for each row execute function proof_delay();") + try: + with ThreadPoolExecutor(max_workers=1) as pool: + task=pool.submit(c.attachment,a,chat) + for _ in range(30): + if sql("select count(*) from pg_stat_activity where wait_event='PgSleep';")!='0': break + time.sleep(.2) + check(sql('select count(*) from attachments;')==before,'metadata visible before commit') + check(sql('select pg_try_advisory_xact_lock(741204);')=='f','reconciler could enter active upload') + att,data=task.result() + check(sql('select pg_try_advisory_xact_lock(741204);')=='t','lock leaked after commit') + received=c.call('GET','/api/v1/attachments/'+att['id']+'/content',actor=b)['bytes'] + check(received==data,'bytes lost in commit window') + print('PASS PUT/COMMIT barrier: uncommitted metadata invisible; exclusive reconcile lock denied during upload; released after commit; other member downloads identical bytes') + finally: + sql('drop trigger proof_delay on attachments; drop function proof_delay();') + +if __name__ == '__main__': run(main) diff --git a/tests/review_regressions.py b/tests/review_regressions.py new file mode 100644 index 0000000..3fdf64b --- /dev/null +++ b/tests/review_regressions.py @@ -0,0 +1,64 @@ +"""Five review regressions, real Compose stack. JPEG fixture generation needs Pillow.""" +from client import Client, check, run +from track2 import sql, mc +from pathlib import Path +from PIL import Image +import io, os, struct, zlib, uuid, hashlib, time, urllib.request + + +def png_large(): + def chunk(kind, data): + return struct.pack('>I', len(data))+kind+data+struct.pack('>I',zlib.crc32(kind+data)&0xffffffff) + width=700 + data=b''.join(b'\0'+os.urandom(width*3) for _ in range(width)) + base=b'\x89PNG\r\n\x1a\n'+chunk(b'IHDR',struct.pack('>IIBBBBB',width,width,8,2,0,0,0))+chunk(b'IDAT',zlib.compress(data)) + extra=1770843-len(base)-12-12 + return base+chunk(b'tEXt',b'proof\0'+b'x'*(extra-6))+chunk(b'IEND',b'') + + +def main(): + print('2026-11-06 Europe/Moscow; current working tree; synthetic data only',flush=True) + c=Client(base_url=os.environ.get('BASE_URL','http://127.0.0.1:8080'),timeout=30) + for _ in range(90): + try: + c.call('GET','/health/ready',schema='Health');break + except Exception: time.sleep(1) + else: raise AssertionError('API not ready') + a,b,e,chat=c.bootstrap() + png=Path('tests/fixtures/sample.png').read_bytes() + out=io.BytesIO(); Image.new('RGB',(32,24),(12,50,100)).save(out,format='JPEG'); jpeg=out.getvalue() + path='/api/v1/chats/'+chat['id']+'/attachments?filename=' + def upload(data,mime='image/png',filename='proof.png',expected=201): + return c.call('POST',path+filename,expected,actor=a,raw=data,headers={'Content-Type':mime},schema='Attachment' if expected==201 else None) + def count(): return sql("select count(*) from attachments where chat_id='"+chat['id']+"';") + before=int(count()); attachments=[]; instances=set() + for data,mime,name in [(png,'image/png','proof.png'),(jpeg,'image/jpeg','proof.jpg')]*5: + response=upload(data,mime,name); att=response['json']; attachments.append(att) + instances.add(response['headers']['X-Instance-Id']) + got=c.call('GET','/api/v1/attachments/'+att['id'],actor=b,schema='Attachment')['json'] + check(got['content_type']==mime and got['status']=='ready','bad MIME/status') + check(c.call('GET','/api/v1/attachments/'+att['id']+'/content',actor=b)['bytes']==data,'bytes differ') + check(int(count())-before==10,'unexpected metadata count') + print('PASS PNG/JPEG: 10 uploads -> 201; 10 metadata GET -> 200; 10 downloads identical; DB +10; instances='+str(len(instances)),flush=True) + plain=c.message(a,chat,'plain')['json'];check(plain['attachment_ids']==[],'missing empty attachment_ids') + attached=c.message(a,chat,'',attachments=[attachments[0]['id']])['json'];check(attached['attachment_ids']==[attachments[0]['id']],'attachment lost') + other=c.call('POST','/api/v1/chats',201,actor=a,body={'title':'other','member_ids':[a['id']]},schema='Chat')['json'] + for actor,target,ids in [(a,other,[attachments[0]['id']]),(a,chat,[str(uuid.uuid4())]),(e,chat,[attachments[0]['id']])]: + c.call('POST','/api/v1/chats/'+target['id']+'/messages',404,actor=actor,body={'text':'x','attachment_ids':ids},headers={'Idempotency-Key':uuid.uuid4().hex}) + print('PASS messages: plain attachment_ids=[]; file-only POST -> 201; cross-chat/unknown/outsider -> 404',flush=True) + before=count(); objects=mc('mc ls proof/messenger-attachments/') + for _ in range(10): upload(png[:33],expected=415) + upload(png,filename='..%2Fx.png',expected=400) + check(count()==before,'rejected upload persisted metadata') + check(mc('mc ls proof/messenger-attachments/')==objects,'rejected upload changed objects') + print('PASS sample.png[:33]: 10 x 415; ../x.png: 400; DB and S3 unchanged',flush=True) + large=png_large();check(len(large)==1770843,'fixture size mismatch') + with Image.open(io.BytesIO(large)) as image: image.load() + att=upload(large)['json'] + result=c.call('GET','/api/v1/attachments/'+att['id']+'/content',actor=b) + check(result['bytes']==large,'large PNG differs') + print('PASS valid PNG exactly 1770843 bytes: POST -> 201 through LB; download checksum matches',flush=True) + print('PASS all five review regressions',flush=True) + + +if __name__=='__main__': run(main) diff --git a/tests/track2.py b/tests/track2.py new file mode 100644 index 0000000..0e4f8ea --- /dev/null +++ b/tests/track2.py @@ -0,0 +1,61 @@ +"""Controlled DB/S3 fault injection; use an isolated local Compose stack.""" +from client import Client, check, run +from pathlib import Path +import subprocess, time, uuid, os, json + +def command(args, data=None): + r=subprocess.run(args,input=data,text=True,capture_output=True) + check(r.returncode==0,'operator command failed: '+args[0]) + return r.stdout.strip() + +def sql(query): + return command(['docker','compose','exec','-T','db','sh','-c','psql -U "$(cat /run/secrets/postgres_user)" -d "$(cat /run/secrets/postgres_db)" -At'],query) + +def mc(action): + return command(['docker','compose','exec','-T','minio','sh','-c', + 'mc alias set proof http://localhost:9000 "$(cat /run/secrets/minio_root_user)" "$(cat /run/secrets/minio_root_password)" >/dev/null && '+action]) + +def main(): + c=Client(timeout=30); a,b,e,chat=c.bootstrap() + protected,protected_data=c.attachment(a,chat) + c.message(a,chat,'protected',attachments=[protected['id']]) + protected_key=sql("select object_key from attachments where id='"+protected['id']+"';") + att,data=c.attachment(a,chat) + key=sql("select object_key from attachments where id='"+att['id']+"';") + orphan='proof-'+uuid.uuid4().hex + mc('mc cp /etc/hostname proof/messenger-attachments/'+orphan) + sql("update attachments set created_at=now()-interval '20 minutes' where id='"+att['id']+"';") + mc('mc rm proof/messenger-attachments/'+key) + fresh='fresh-'+uuid.uuid4().hex + # Objects must age past the configured one-minute test grace before scanning. + time.sleep(62) + mc('mc cp /etc/hostname proof/messenger-attachments/'+fresh) + # API replicas run scheduled reconciliation with dry-run=true, interval=3s, grace=1min. + time.sleep(6) + check(sql("select status from attachments where id='"+att['id']+"';")=='READY','dry-run mutated DB') + mc('mc stat proof/messenger-attachments/'+orphan) + logs=command(['docker','compose','logs','--no-log-prefix','api']) + proof='\n'.join(line for line in logs.splitlines() if 'Reconcile' in line) + Path('evidence/reconciler-dry-run-log.txt').write_text(proof) + check('orphan dryRun=true' in proof and 'missing dryRun=true' in proof,'dry-run did not report findings') + print('PASS dry-run: old orphan and missing metadata detected; no mutations',flush=True) + env=dict(os.environ,RECONCILER_INTERVAL_MS='3000',RECONCILER_GRACE_PERIOD_MINUTES='1',RECONCILER_DRY_RUN='false') + r=subprocess.run(['docker','compose','up','-d','--no-deps','api'],env=env,capture_output=True) + check(r.returncode==0,'recreate failed') + for _ in range(90): + if sql("select status from attachments where id='"+att['id']+"';")=='BROKEN': break + time.sleep(1) + check(sql("select status from attachments where id='"+att['id']+"';")=='BROKEN','missing not repaired') + check(orphan not in mc('mc ls proof/messenger-attachments/'),'orphan retained') + mc('mc stat proof/messenger-attachments/'+fresh) + print('PASS apply: orphan removed, missing marked BROKEN, fresh object retained',flush=True) + time.sleep(5) + check(sql("select status from attachments where id='"+att['id']+"';")=='BROKEN','repeat changed result') + mc('mc stat proof/messenger-attachments/'+protected_key) + check(sql("select status from attachments where id='"+protected['id']+"';")=='READY','used attachment damaged') + logs=command(['docker','compose','logs','--no-log-prefix','api']) + Path('evidence/reconciler-apply-log.txt').write_text('\n'.join(line for line in logs.splitlines() if 'Reconcile' in line)) + print('PASS repeat is idempotent; referenced attachment and bytes retained',flush=True) + +if __name__ == '__main__': + run(main)