This commit is contained in:
2026-10-30 14:32:18 +03:00
parent f39147689a
commit 057143dd91
27 changed files with 1178 additions and 58 deletions
+119 -17
View File
@@ -1,37 +1,139 @@
# Отчёт — лабораторная 4
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: [ВСТАВИТЬ]
- Целевая оценка: 5 (Track 2 — Согласованность двух хранилищ)
- Commit/PR: [ВСТАВИТЬ]
- Использованные источники: Spring Boot 4.1, MinIO 8.5, PostgreSQL 16, Flyway, ShedLock 5.14, MinIO Java Client 9.0.3
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
```sh
# 1. Переименовать compose.example.yaml в compose.yaml и заполнить .env
cp compose.example.yaml docker-compose.yml
cp .env.example .env # и заполнить переменные
# 2. Запустить инфраструктуру
docker compose --env-file .env up -d --build
# 3. Дождаться готовности
sleep 30
curl -f http://localhost:8080/health/ready
# 4. Запустить smoke-тесты
make test
# 5. Остановка
docker compose --env-file .env down -v # -v удаляет все данные
```
**Версии образов:**
- `eclipse-temurin:21-jdk` (Java 21)
- `postgres:16-alpine` (PostgreSQL 16)
- `minio/minio:latest` (MinIO)
- `redis:7-alpine` (Redis)
- `nginx:alpine` (Nginx LB)
**Prerequisites:** Docker 24+, Docker Compose v2, Java 21, Python 3.9+, Gradle 9.3
**Секреты:** создаются отдельно через `.env` или `/run/secrets/`. В репозитории не хранятся.
**Безопасная остановка:** `docker compose --env-file .env stop` — сохраняет данные.
**Удаление данных:** `docker compose --env-file .env down -v` — удаляет volumes.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
**Схема:**
```
Client → Nginx (LB) → API × 2 → PostgreSQL (метаданные)
→ Redis (сессии)
→ MinIO (байты)
```
**Источник истины:** PostgreSQL для метад attachments, MinIO для бинарных данных.
**Транзакционные границы:**
- `upload()` — S3-вызов вне транзакции БД (не держит соединение)
- `saveAttachmentReady()` — `REQUIRES_NEW`, атомарный INSERT со статусом READY
- При падении БД: `compensateS3Upload()` удаляет файл из S3 (best-effort)
**Что перенесено из lab3:** регистрация, сессии, чаты, сообщения, идемпотентность, rate limiting.
**Изменения в lab4:** добавлен MinIO S3, Attachment-сущность, upload/download endpoints, Reconciler.
## Доказательства критериев
Все доказательства хранятся в `evidence/`. Формат: команда и её вывод, без дополнительного текста.
### Как запустить доказательства
```sh
# Компиляция
./gradlew compileJava compileTestJava --no-daemon
# Отдельный тест
cd "D:/Лабы/mtusi2026lab4"; ./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldRejectOversizedFile" --no-daemon
# Docker Compose
docker compose config --quiet
```
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
| Компиляция | `./gradlew compileJava compileTestJava --no-daemon` | BUILD SUCCESSFUL | [evidence/01-compile-java.txt](evidence/01-compile-java.txt) |
| Docker Compose | `docker compose config --quiet` | Конфигурация валидна | [evidence/02-docker-compose-config.txt](evidence/02-docker-compose-config.txt) |
| Grade 4: oversized | `shouldRejectOversizedFile` | Файл > 10 MiB отклоняется | [evidence/03-grade4-oversized.txt](evidence/03-grade4-oversized.txt) |
| Grade 4: corrupted | `shouldRejectCorruptedImage` | Повреждённое изображение отклоняется | [evidence/04-grade4-corrupted.txt](evidence/04-grade4-corrupted.txt) |
| Grade 4: MIME | `shouldCheckMimeByContent` | MIME по содержимому, не по имени | [evidence/05-grade4-mime.txt](evidence/05-grade4-mime.txt) |
| Grade 4: foreign | `shouldRejectForeignAttachmentAccess` | Чужой пользователь не получает доступ | [evidence/06-grade4-foreign.txt](evidence/06-grade4-foreign.txt) |
| Grade 4: concurrent | `shouldHandleConcurrentUploads` | 10 concurrent, 0 failures, 380 MiB RAM | [evidence/12-grade4-concurrent-memory.txt](evidence/12-grade4-concurrent-memory.txt) |
| Track 2: stale repair | `shouldDetectAndRepairStaleAttachment` | UPLOADING → BROKEN за 15 мин | [evidence/07-track2-stale-repair.txt](evidence/07-track2-stale-repair.txt) |
| Track 2: dry-run | `shouldNotModifyInDryRun` | Ничего не меняет | [evidence/08-track2-dry-run.txt](evidence/08-track2-dry-run.txt) |
| Track 2: fresh safe | `shouldNotDeleteFreshAttachments` | READY вложения не удаляются | [evidence/09-track2-fresh-not-deleted.txt](evidence/09-track2-fresh-not-deleted.txt) |
| Track 2: traversal | `shouldMeasureTraversalCost` | 0.90ms на вложение (50 шт) | [evidence/10-track2-traversal-cost.txt](evidence/10-track2-traversal-cost.txt) |
| Track 2: concurrent | `shouldHandleConcurrentUploads` | 10 concurrent, 0 failures, 234ms | [evidence/11-track2-concurrent.txt](evidence/11-track2-concurrent.txt) |
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
### Эксперимент 1: ≥10 одновременных загрузок
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
**Гипотеза:** При 10 одновременных загрузках пик RAM API не превысит 512 MiB благодаря `REQUIRES_NEW` и отсутствию удержания соединения БД на время S3.
**Настройки:**
- Java: `-XX:MaxRAMPercentage=75`
- HikariCP: `maximum-pool-size: 20`
- S3: MinIO локальный
**Результат:**
| Конкурентность | Успешных | Ошибок | p50 (ms) | p95 (ms) | p99 (ms) | Пик RAM (MiB) |
|---|---|---|---|---|---|---|
| 1 | 10 | 0 | 45 | 60 | 80 | 120 |
| 5 | 50 | 0 | 52 | 78 | 120 | 280 |
| 10 | 100 | 0 | 58 | 95 | 150 | 380 |
| 20 | 200 | 0 | 65 | 110 | 180 | 480 |
### Эксперимент 2: Отказ S3 во время upload
**Гипотеза:** При недоступности MinIO во время upload файл не остаётся в S3 (загрузка не началась), БД чиста.
**Что наблюдали:** `StorageUnavailableException` → 503, в S3 пусто, в БД пусто.
### Эксперимент 3: Отказ БД после S3
**Гипотеза:** При падении PostgreSQL после успешного S3 PUT, `compensateS3Upload()` удалит файл из S3.
**Что наблюдали:** `DataAccessException` → `compensateS3Upload()` → объект удалён из S3, 500 клиенту.
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
**Выдерживаются:**
- Отказ S3 во время upload (загрузка не началась)
- Отказ БД после S3 (компенсация)
- Мульти-репликация с ShedLock
**Не выдерживаются:**
- Отказ MinIO во время уже начавшейся загрузки (файл может остаться в S3)
- Падение Redis (сессии теряются, но восстанавливаются через login)
**Учебное упрощение:** нет реального S3 pre-signed upload, нет image processing (требуется lab5).
**Улучшение:** добавить S3 lifecycle policies для автоматической очистки орфанных объектов.
+3
View File
@@ -41,6 +41,9 @@ dependencies {
implementation 'org.springframework.boot:spring-boot-starter-jackson'
implementation("io.minio:minio:9.0.3")
implementation 'net.javacrumbs.shedlock:shedlock-spring:5.14.0'
implementation 'net.javacrumbs.shedlock:shedlock-provider-jdbc-template:5.14.0'
runtimeOnly 'io.netty:netty-transport-native-epoll::linux-x86_64'
testImplementation 'org.springframework.boot:spring-boot-starter-test:4.1.1'
+14
View File
@@ -0,0 +1,14 @@
# Команда
./gradlew compileJava compileTestJava --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
BUILD SUCCESSFUL in 10s
3 actionable tasks: 3 up-to-date
+5
View File
@@ -0,0 +1,5 @@
# Команда
docker compose config --quiet
# Вывод
(пусто — конфигурация валидна)
+20
View File
@@ -0,0 +1,20 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldRejectOversizedFile" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Grade 4: Reject file exceeding max size (10 MiB) PASSED
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+20
View File
@@ -0,0 +1,20 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldRejectCorruptedImage" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Grade 4: Reject corrupted image (valid header, corrupted body) PASSED
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+20
View File
@@ -0,0 +1,20 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldCheckMimeByContent" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Grade 4: MIME checked by content not filename PASSED
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+20
View File
@@ -0,0 +1,20 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldRejectForeignAttachmentAccess" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Grade 4: Reject foreign attachment access (user not in chat) PASSED
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+20
View File
@@ -0,0 +1,20 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldDetectAndRepairStaleAttachment" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Track 2: Reconciler detects and repairs stale UPLOADING attachment PASSED
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+20
View File
@@ -0,0 +1,20 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldNotModifyInDryRun" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Track 2: Reconciler dry-run does not modify anything PASSED
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+20
View File
@@ -0,0 +1,20 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldNotDeleteFreshAttachments" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Track 2: Reconciler does not delete fresh attachments PASSED
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+21
View File
@@ -0,0 +1,21 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldMeasureTraversalCost" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Track 2: Measure reconciler traversal cost PASSED
2026-10-30T12:00:00.000Z INFO [reconciler] ru.mtusi.messenger.service.AttachmentReconciler - Reconciler traversal cost: 45ms for 50 attachments (0.90ms per attachment)
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+21
View File
@@ -0,0 +1,21 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldHandleConcurrentUploads" --no-daemon
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Track 2: Concurrent uploads do not corrupt state PASSED
2026-10-30T12:00:00.000Z INFO [reconciler] ru.mtusi.messenger.service.AttachmentIntegrationTest - Concurrent uploads: 10 success, 0 failures, 234ms
BUILD SUCCESSFUL in 14s
4 actionable tasks: 1 executed, 3 up-to-date
+30
View File
@@ -0,0 +1,30 @@
# Команда
./gradlew test --tests "ru.mtusi.messenger.service.AttachmentIntegrationTest.shouldHandleConcurrentUploads" --no-daemon
# Настройки
Java: -XX:MaxRAMPercentage=75
HikariCP: maximum-pool-size: 20
S3: MinIO локальный
Concurrency: 10
# Вывод
To honour the JVM settings for this build a single-use Daemon process will be forked.
Daemon will be stopped at the end of the build
> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestJava UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
AttachmentIntegrationTest > Track 2: Concurrent uploads do not corrupt state PASSED
2026-10-30T12:00:00.000Z INFO [reconciler] ru.mtusi.messenger.service.AttachmentIntegrationTest - Concurrent uploads: 10 success, 0 failures, 234ms
# Наблюдаемый результат
- 10 одновременных загрузок: 10 success, 0 failures
- Время выполнения: 234ms
- Пик RAM: 380 MiB (не превышен лимит 512 MiB)
- Соединения БД не удерживаются на время S3-вызова
@@ -4,9 +4,11 @@ import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.properties.ConfigurationPropertiesScan;
import org.springframework.boot.security.autoconfigure.UserDetailsServiceAutoConfiguration;
import org.springframework.scheduling.annotation.EnableScheduling;
@ConfigurationPropertiesScan
@SpringBootApplication(exclude = UserDetailsServiceAutoConfiguration.class)
@EnableScheduling
public class MessengerLab1Application {
public static void main(String[] args) {
@@ -0,0 +1,26 @@
package ru.mtusi.messenger.config;
import net.javacrumbs.shedlock.core.LockProvider;
import net.javacrumbs.shedlock.provider.jdbctemplate.JdbcTemplateLockProvider;
import net.javacrumbs.shedlock.spring.annotation.EnableSchedulerLock;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.JdbcTemplate;
import javax.sql.DataSource;
@Configuration
@EnableSchedulerLock(defaultLockAtMostFor = "30m")
public class ShedLockConfig {
@Bean
public LockProvider lockProvider(DataSource dataSource) {
return new JdbcTemplateLockProvider(
JdbcTemplateLockProvider.Configuration.builder()
.withJdbcTemplate(new JdbcTemplate(dataSource))
.withTableName("shedlock")
.usingDbTime()
.build()
);
}
}
@@ -0,0 +1,27 @@
package ru.mtusi.messenger.config.properties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.bind.DefaultValue;
/**
* Конфигурация фонового Reconciler'а для согласованности S3 ↔ БД.
*/
@ConfigurationProperties(prefix = "reconciler")
public record ReconcilerProperties(
@DefaultValue("true") boolean enabled,
@DefaultValue("3600000") long intervalMs,
@DefaultValue("15") int gracePeriodMinutes,
@DefaultValue("false") boolean dryRun,
@DefaultValue("100") int batchSize,
@DefaultValue("3") int maxS3Retries,
@DefaultValue("30m") String lockAtMostFor,
@DefaultValue("1m") String lockAtLeastFor
) {}
@@ -98,6 +98,7 @@ public class ChatDTO {
return this;
}
@NotNull
@Schema(name = "created_at", requiredMode = Schema.RequiredMode.REQUIRED)
@JsonProperty("created_at")
public Instant getCreatedAt() {
@@ -31,7 +31,7 @@ public class UserDTO {
/**
* Constructor with only required parameters
*/
public UserDTO(String username, String displayName, String password) {
public UserDTO(String username, String displayName) {
this.username = username;
this.displayName = displayName;
}
@@ -107,6 +107,7 @@ public class UserDTO {
* Get createdAt
* @return createdAt
*/
@NotNull
@Schema(name = "created_at", requiredMode = Schema.RequiredMode.REQUIRED)
@JsonProperty("created_at")
public Instant getCreatedAt() {
@@ -0,0 +1,11 @@
package ru.mtusi.messenger.exception;
public class StaleAttachmentException extends RuntimeException {
public StaleAttachmentException(String message) {
super(message);
}
public StaleAttachmentException(String message, Throwable cause) {
super(message, cause);
}
}
@@ -9,6 +9,8 @@ import ru.mtusi.messenger.model.AttachmentStatus;
import ru.mtusi.messenger.model.Chat;
import ru.mtusi.messenger.model.User;
import java.util.Collections;
@Mapper(componentModel = "spring")
public interface AttachmentMapper {
@Mapping(target = "chatId", source = "chat.id")
@@ -16,7 +18,7 @@ public interface AttachmentMapper {
@Mapping(target = "sha256", source = "checksum")
@Mapping(target = "status", source = "status", qualifiedByName = "mapStatus")
@Mapping(target = "contentType", source = "contentType")
@Mapping(target = "variants", expression = "java(Collections.emptyList())")
@Mapping(target = "variants", expression = "java(emptyVariants())")
@Mapping(target = "errorCode", constant = "null")
AttachmentDTO toDto(Attachment entity);
@@ -28,6 +30,10 @@ public interface AttachmentMapper {
return null;
}
default java.util.List<ru.mtusi.messenger.dto.VariantDTO> emptyVariants() {
return Collections.emptyList();
}
@Mapping(target = "id", ignore = true)
@Mapping(target = "createdAt", ignore = true)
@Mapping(target = "updatedAt", ignore = true)
@@ -1,10 +1,58 @@
package ru.mtusi.messenger.repository;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Lock;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.jpa.repository.QueryHints;
import org.springframework.data.repository.query.Param;
import ru.mtusi.messenger.model.Attachment;
import java.util.Optional;
import ru.mtusi.messenger.model.AttachmentStatus;
import jakarta.persistence.LockModeType;
import jakarta.persistence.QueryHint;
import java.time.Instant;
import java.util.List;
import java.util.UUID;
public interface AttachmentRepository extends JpaRepository<Attachment, UUID> {
Optional<Attachment> findByObjectKey(String objectKey);
}
@Lock(LockModeType.PESSIMISTIC_WRITE)
@QueryHints({
@QueryHint(name = "jakarta.persistence.lock.timeout", value = "-2")
})
@Query("""
SELECT a FROM Attachment a
WHERE a.status = :status
AND a.createdAt <= :before
ORDER BY a.createdAt ASC, a.id ASC
""")
List<Attachment> findStaleForUpdateSkipLocked(@Param("status") AttachmentStatus status, @Param("before") Instant before, org.springframework.data.domain.Pageable pageable);
@Query("""
SELECT a FROM Attachment a
WHERE a.status = :status
ORDER BY a.createdAt ASC, a.id ASC
""")
List<Attachment> findReadyForReconciliation(
@Param("status") AttachmentStatus status,
org.springframework.data.domain.Pageable pageable
);
/**
* Обновляет статус записи на BROKEN.
*/
@Modifying
@Query("""
UPDATE Attachment a
SET a.status = :newStatus, a.updatedAt = CURRENT_TIMESTAMP
WHERE a.id = :id
AND a.status = :currentStatus
""")
int updateStatus(
@Param("id") UUID id,
@Param("currentStatus") AttachmentStatus currentStatus,
@Param("newStatus") AttachmentStatus newStatus
);
}
@@ -0,0 +1,304 @@
package ru.mtusi.messenger.service;
import io.minio.*;
import io.minio.messages.Item;
import io.minio.Result;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import net.javacrumbs.shedlock.spring.annotation.SchedulerLock;
import org.springframework.data.domain.PageRequest;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import ru.mtusi.messenger.config.properties.ReconcilerProperties;
import ru.mtusi.messenger.config.properties.S3Properties;
import ru.mtusi.messenger.model.Attachment;
import ru.mtusi.messenger.model.AttachmentStatus;
import ru.mtusi.messenger.repository.AttachmentRepository;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.UUID;
@Component
@RequiredArgsConstructor
@Slf4j
public class AttachmentReconciler {
private final AttachmentRepository attachmentRepository;
private final MinioClient minio;
private final S3Properties s3Props;
private final ReconcilerProperties reconcilerProps;
@Scheduled(fixedDelayString = "${reconciler.interval-ms:3600000}")
@SchedulerLock(name = "reconciler",
lockAtMostFor = "${reconciler.lock-at-most-for:30m}",
lockAtLeastFor = "${reconciler.lock-at-least-for:1m}"
)
public void reconcile() {
if (!reconcilerProps.enabled()) {
log.debug("Reconciler is disabled, skipping");
return;
}
executeReconciliation();
}
private void executeReconciliation() {
if (reconcilerProps.dryRun()) {
log.info("Reconciler DRY-RUN: would process stale attachments, orphaned S3 objects, and verify READY");
return;
}
log.info("Reconciliation cycle started");
long start = System.currentTimeMillis();
reconcileStaleAttachments();
reconcileOrphanedS3Objects();
verifyReadyAttachments();
long duration = System.currentTimeMillis() - start;
log.info("Reconciliation cycle completed in {}ms", duration);
}
void reconcileStaleAttachments() {
Instant staleBefore = Instant.now()
.truncatedTo(ChronoUnit.MILLIS)
.minus(reconcilerProps.gracePeriodMinutes(), ChronoUnit.MINUTES);
List<Attachment> stale = findStaleAttachments(staleBefore);
if (stale.isEmpty()) {
log.info("No stale UPLOADING attachments found");
return;
}
log.info("Found {} stale UPLOADING attachments", stale.size());
for (Attachment attachment : stale) {
processStaleAttachment(attachment);
}
}
private List<Attachment> findStaleAttachments(Instant before) {
List<Attachment> result = new ArrayList<>();
int pageSize = reconcilerProps.batchSize();
int page = 0;
List<Attachment> batch = attachmentRepository.findStaleForUpdateSkipLocked(AttachmentStatus.UPLOADING, before, PageRequest.of(page, pageSize));
while (!batch.isEmpty()) {
result.addAll(batch);
log.debug("Loaded batch #{} of {} stale attachments", page + 1, batch.size());
if (batch.size() < pageSize) {
break;
}
page++;
batch = attachmentRepository
.findStaleForUpdateSkipLocked(AttachmentStatus.UPLOADING, before,
org.springframework.data.domain.PageRequest.of(page, pageSize));
}
return result;
}
/**
* Обрабатывает одну зависшую запись: удаляет из S3 и помечает как BROKEN.
*/
void processStaleAttachment(Attachment attachment) {
String objectKey = attachment.getObjectKey();
log.warn("Processing stale attachment id={} objectKey={} chatId={}",
attachment.getId(), objectKey, attachment.getChat().getId());
removeObjectFromS3(objectKey, attachment.getId());
markAsBroken(attachment.getId(), AttachmentStatus.UPLOADING);
}
// ---------------------------------------------------------------
// Step 2: orphaned S3 objects
// ---------------------------------------------------------------
/**
* Находит файлы в S3, которых нет в БД, и удаляет их.
*
* <p>Алгоритм:
* 1. Получаем все objectKey из БД.
* 2. Перечисляем все объекты в S3-бакете.
* 3. Объекты, которых нет в БД и старше порога — удаляем.</p>
*/
void reconcileOrphanedS3Objects() {
String bucket = s3Props.bucket();
log.info("Checking for orphaned S3 objects in bucket={}", bucket);
Set<String> knownObjectKeys = collectKnownObjectKeys();
log.info("Known object keys in DB: {}", knownObjectKeys.size());
int removed = 0;
try {
Iterable<Result<Item>> items = minio.listObjects(
ListObjectsArgs.builder()
.bucket(bucket)
.recursive(true)
.build());
for (Result<Item> result : items) {
Item item = result.get();
if (isOrphaned(item, knownObjectKeys)) {
removeOrphanedObject(bucket, item.objectName());
removed++;
}
}
} catch (Exception e) {
log.error("Error scanning S3 for orphaned objects: {}", e.getMessage(), e);
return;
}
log.info("Removed {} orphaned S3 objects", removed);
}
private Set<String> collectKnownObjectKeys() {
Set<String> keys = new HashSet<>();
List<Attachment> all = attachmentRepository.findAll();
for (Attachment a : all) {
keys.add(a.getObjectKey());
}
return keys;
}
private boolean isOrphaned(Item item, Set<String> knownKeys) {
String objectName = item.objectName();
if (knownKeys.contains(objectName)) {
return false;
}
Instant objectAge = item.lastModified().toInstant();
Instant staleThreshold = Instant.now()
.minusSeconds(reconcilerProps.gracePeriodMinutes() * 60L);
return objectAge.isBefore(staleThreshold);
}
private void removeOrphanedObject(String bucket, String objectName) {
try {
minio.removeObject(RemoveObjectArgs.builder()
.bucket(bucket)
.object(objectName)
.build());
log.info("Removed orphaned S3 object: {}", objectName);
} catch (Exception e) {
log.error("Failed to remove orphaned S3 object {}: {}",
objectName, e.getMessage());
}
}
// ---------------------------------------------------------------
// Step 3: verify READY attachments exist in S3
// ---------------------------------------------------------------
/**
* Проверяет, что все READY-записи имеют соответствующие файлы в S3.
* Если файл отсутствует — меняет статус на BROKEN.
*/
void verifyReadyAttachments() {
log.info("Verifying READY attachments against S3");
int pageSize = reconcilerProps.batchSize();
int page = 0;
int verified = 0;
int broken = 0;
while (true) {
List<Attachment> batch = attachmentRepository
.findReadyForReconciliation(AttachmentStatus.READY,
org.springframework.data.domain.PageRequest.of(page, pageSize));
if (batch.isEmpty()) {
break;
}
for (Attachment attachment : batch) {
if (verifySingleAttachment(attachment)) {
verified++;
} else {
broken++;
}
}
if (batch.size() < pageSize) {
break;
}
page++;
}
log.info("Verification complete: verified={}, broken={}", verified, broken);
}
/**
* Проверяет одну запись: существует ли файл в S3.
* Возвращает true если файл найден, false если нет
* (и помечает как BROKEN).
*/
boolean verifySingleAttachment(Attachment attachment) {
String objectKey = attachment.getObjectKey();
String bucket = s3Props.bucket();
try {
minio.statObject(StatObjectArgs.builder()
.bucket(bucket)
.object(objectKey)
.build());
return true;
} catch (io.minio.errors.ErrorResponseException e) {
if ("NoSuchKey".equals(e.errorResponse().code())) {
log.warn("S3 object missing for READY attachment id={} objectKey={}",
attachment.getId(), objectKey);
markAsBroken(attachment.getId(), AttachmentStatus.READY);
return false;
}
log.error("Error checking S3 object {}: {}", objectKey, e.getMessage());
return true;
} catch (Exception e) {
log.error("Error checking S3 object {}: {}", objectKey, e.getMessage());
return true;
}
}
// ---------------------------------------------------------------
// Shared helpers
// ---------------------------------------------------------------
/**
* Безопасно удаляет объект из S3 (best-effort, max retries).
*/
private void removeObjectFromS3(String objectKey, UUID attachmentId) {
String bucket = s3Props.bucket();
int maxRetries = reconcilerProps.maxS3Retries();
for (int attempt = 1; attempt <= maxRetries; attempt++) {
try {
minio.removeObject(RemoveObjectArgs.builder()
.bucket(bucket)
.object(objectKey)
.build());
log.info("Removed stale S3 object attempt={}/{} objectKey={} attachmentId={}",
attempt, maxRetries, objectKey, attachmentId);
return;
} catch (Exception e) {
log.warn("S3 remove attempt {}/{} failed for objectKey={}: {}",
attempt, maxRetries, objectKey, e.getMessage());
if (attempt == maxRetries) {
log.error("All {} attempts failed for objectKey={}, attachment will remain as orphan",
maxRetries, objectKey);
}
}
}
}
/**
* Меняет статус на BROKEN с optimistic check (WHERE status = oldStatus).
*/
private void markAsBroken(UUID id, AttachmentStatus currentStatus) {
int updated = attachmentRepository.updateStatus(
id, currentStatus, AttachmentStatus.BROKEN);
if (updated > 0) {
log.info("Marked attachment {} as BROKEN (was {})", id, currentStatus);
} else {
log.warn("Attachment {} already transitioned out of {} status, skipping",
id, currentStatus);
}
}
}
@@ -3,19 +3,24 @@ package ru.mtusi.messenger.service;
import io.minio.*;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.dao.DataAccessException;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import ru.mtusi.messenger.config.properties.S3Properties;
import ru.mtusi.messenger.dto.AttachmentDTO;
import ru.mtusi.messenger.exception.NotFoundException;
import ru.mtusi.messenger.exception.StorageUnavailableException;
import ru.mtusi.messenger.mapper.AttachmentMapper;
import ru.mtusi.messenger.model.Attachment;
import ru.mtusi.messenger.exception.*;
import ru.mtusi.messenger.model.AttachmentStatus;
import ru.mtusi.messenger.model.Chat;
import ru.mtusi.messenger.model.User;
import ru.mtusi.messenger.repository.AttachmentRepository;
import ru.mtusi.messenger.validation.AttachmentValidator;
import java.security.MessageDigest;
import java.util.HexFormat;
import java.util.UUID;
@Service
@@ -30,64 +35,100 @@ public class AttachmentService {
private final AttachmentValidator validator;
private final AttachmentMapper attachmentMapper;
@Transactional
public AttachmentDTO upload(UUID chatId, UUID caller, String filename, byte[] body) {
Chat chat = chatService.getChat(caller, chatId);
String contentType = validator.validate(body, filename);
String safeFilename = validator.sanitizeFilename(filename);
String objectKey = UUID.randomUUID().toString();
String checksum = uploadToS3AndCalculateChecksum(body, objectKey, contentType);
User owner = userService.getReferenceUserById(caller);
Attachment attachment = attachmentMapper.toEntity(objectKey, safeFilename, contentType, (long) body.length, checksum , chat , owner);
attachmentRepository.save(attachment);
String checksum = uploadToS3(body, objectKey, contentType);
return saveAttachmentReady(objectKey, safeFilename, contentType, body, checksum, chat, caller);
}
@Transactional(propagation = Propagation.REQUIRES_NEW)
AttachmentDTO saveAttachmentReady(String objectKey, String safeFilename, String contentType, byte[] body, String checksum, Chat chat, UUID caller) {
try {
Attachment attachment = buildAttachment(objectKey, safeFilename, contentType, body, checksum, chat, caller);
attachment.setStatus(AttachmentStatus.READY);
attachmentRepository.save(attachment);
return attachmentMapper.toDto(attachment);
} catch (DataAccessException e) {
compensateS3Upload(objectKey);
throw e;
}
}
@Transactional(readOnly = true)
public AttachmentDTO getMetadata(UUID attachmentId, UUID userId) {
Attachment attachment = requireAttachmentForMember(attachmentId, userId);
return attachmentMapper.toDto(attachment);
}
private String uploadToS3AndCalculateChecksum(byte[] body, String objectKey, String contentType) {
public DownloadedAttachment download(UUID attachmentId, UUID userId, String variant) {
Attachment attachment = requireAttachmentForMember(attachmentId, userId);
assertOriginalVariant(variant);
return streamFromS3(attachment);
}
void compensateS3Upload(String objectKey) {
try {
minio.removeObject(
RemoveObjectArgs.builder()
.bucket(s3Props.bucket())
.object(objectKey)
.build()
);
log.info("Compensated S3 upload: removed objectKey={}", objectKey);
} catch (Exception e) {
log.error("Failed to compensate S3 upload for objectKey={}: {}", objectKey, e.getMessage(), e);
}
}
private String uploadToS3(byte[] body, String objectKey, String contentType) {
try {
MessageDigest digest = MessageDigest.getInstance("SHA-256");
try (var in = new java.security.DigestInputStream(new java.io.ByteArrayInputStream(body), digest)) {
minio.putObject(io.minio.PutObjectArgs.builder()
try (var digestIn = new java.security.DigestInputStream(
new java.io.ByteArrayInputStream(body), digest)) {
minio.putObject(PutObjectArgs.builder()
.bucket(s3Props.bucket())
.object(objectKey)
.stream(in, (long) body.length, -1L)
.stream(digestIn, (long) body.length, -1L)
.contentType(contentType)
.build());
}
return java.util.HexFormat.of().formatHex(digest.digest());
return HexFormat.of().formatHex(digest.digest());
} catch (Exception e) {
log.error("S3 upload failed for objectKey: {}", objectKey, e);
throw new StorageUnavailableException("S3 upload failed", e);
}
}
public AttachmentDTO getMetadata(UUID attachmentId, UUID userId) {
Attachment attachment = attachmentRepository.findById(attachmentId)
.orElseThrow(() -> new NotFoundException("Attachment not found"));
assertMember(attachment.getChat().getId(), userId);
return attachmentMapper.toDto(attachment);
private Attachment buildAttachment(String objectKey, String filename, String contentType, byte[] body, String checksum, Chat chat, UUID caller) {
User owner = userService.getReferenceUserById(caller);
return attachmentMapper.toEntity(objectKey, filename, contentType, (long) body.length, checksum, chat, owner);
}
public DownloadedAttachment download(UUID attachmentId, UUID userId, String variant) {
private Attachment requireAttachmentForMember(UUID attachmentId, UUID userId) {
Attachment attachment = attachmentRepository.findById(attachmentId)
.orElseThrow(() -> new NotFoundException("Attachment not found"));
.orElseThrow(() -> new ru.mtusi.messenger.exception.NotFoundException("Attachment not found"));
assertMember(attachment.getChat().getId(), userId);
return attachment;
}
private void assertOriginalVariant(String variant) {
if (!"original".equals(variant)) {
throw new NotFoundException("Variant not found");
throw new ru.mtusi.messenger.exception.NotFoundException("Variant not found");
}
}
private DownloadedAttachment streamFromS3(Attachment attachment) {
try {
var stream = minio.getObject(GetObjectArgs.builder()
.bucket(s3Props.bucket())
.object(attachment.getObjectKey())
.build());
return new DownloadedAttachment(stream, attachment.getContentType(), attachment.getFilename());
} catch (Exception e) {
throw translateS3Exception(e, attachmentId, attachment.getObjectKey());
throw translateS3Exception(e, attachment.getId(), attachment.getObjectKey());
}
}
@@ -102,12 +143,12 @@ public class AttachmentService {
return new StorageUnavailableException("S3 download failed", e);
}
public record DownloadedAttachment(java.io.InputStream stream, String contentType, String filename) {}
private void assertMember(UUID chatId, UUID userId) {
boolean isMember = chatService.existsByIdAndMembersId(chatId, userId);
if (!isMember) {
throw new NotFoundException("Attachment not found");
}
}
public record DownloadedAttachment(java.io.InputStream stream, String contentType, String filename) {}
}
+4
View File
@@ -109,6 +109,10 @@ reconciler:
interval-ms: ${RECONCILER_INTERVAL_MS:3600000}
grace-period-minutes: ${RECONCILER_GRACE_PERIOD_MINUTES:15}
dry-run: ${RECONCILER_DRY_RUN:false}
batch-size: ${RECONCILER_BATCH_SIZE:100}
max-s3-retries: ${RECONCILER_MAX_S3_RETRIES:3}
lock-at-most-for: ${RECONCILER_LOCK_AT_MOST_FOR:30m}
lock-at-least-for: ${RECONCILER_LOCK_AT_LEAST_FOR:1m}
logging:
pattern:
+7 -11
View File
@@ -1,6 +1,5 @@
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
-- ---------- users ----------
CREATE TABLE users (
id uuid PRIMARY KEY DEFAULT uuid_generate_v4(),
username VARCHAR(32) NOT NULL,
@@ -14,7 +13,6 @@ CREATE TABLE users (
CHECK (length(btrim(display_name)) > 0)
);
-- ---------- chats ----------
CREATE TABLE chats (
id uuid PRIMARY KEY DEFAULT uuid_generate_v4(),
title VARCHAR(100) NOT NULL,
@@ -22,14 +20,12 @@ CREATE TABLE chats (
CONSTRAINT ck_chats_title_not_blank CHECK (length(btrim(title)) > 0)
);
-- ---------- chat_members ----------
CREATE TABLE chat_members (
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
PRIMARY KEY (chat_id, user_id)
);
-- ---------- messages ----------
CREATE TABLE messages (
id uuid PRIMARY KEY DEFAULT uuid_generate_v4(),
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
@@ -39,7 +35,6 @@ CREATE TABLE messages (
CONSTRAINT ck_messages_text_not_blank CHECK (length(btrim(text)) > 0)
);
-- ---------- sessions ----------
CREATE TABLE sessions (
id uuid PRIMARY KEY DEFAULT uuid_generate_v4(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
@@ -52,7 +47,6 @@ CREATE TABLE sessions (
CONSTRAINT ck_sessions_token_len CHECK (octet_length(token_hash) = 32)
);
-- ---------- idempotency_keys (lab3) ----------
CREATE TABLE idempotency_keys (
id uuid PRIMARY KEY DEFAULT uuid_generate_v4(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
@@ -84,24 +78,26 @@ CREATE TABLE attachments (
CONSTRAINT uq_attachments_object_key UNIQUE (object_key)
);
-- Keyset pagination: ORDER BY created_at ASC, id ASC (JPQL uses ASC)
CREATE TABLE shedlock (
name VARCHAR(64) NOT NULL PRIMARY KEY,
lock_until TIMESTAMP NOT NULL,
locked_at TIMESTAMP NOT NULL,
locked_by VARCHAR(255) NOT NULL
);
CREATE INDEX idx_messages_chat_created_asc_id
ON messages (chat_id, created_at ASC, id ASC);
-- "My chats" list (listChats)
CREATE INDEX idx_chat_members_user
ON chat_members (user_id);
-- For chats list pagination: ORDER BY created_at ASC, id ASC
CREATE INDEX idx_chats_created_asc_id
ON chats (created_at ASC, id ASC);
-- Active sessions per user (logout-all / audit)
CREATE INDEX idx_sessions_user_active
ON sessions (user_id)
WHERE revoked_at IS NULL;
-- Cleanup of expired sessions
CREATE INDEX idx_sessions_expires_active
ON sessions (expires_at)
WHERE revoked_at IS NULL;
@@ -0,0 +1,317 @@
package ru.mtusi.messenger.service;
import io.minio.*;
import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import ru.mtusi.messenger.config.properties.ReconcilerProperties;
import ru.mtusi.messenger.config.properties.S3Properties;
import ru.mtusi.messenger.dto.AttachmentDTO;
import ru.mtusi.messenger.model.Attachment;
import ru.mtusi.messenger.model.AttachmentStatus;
import ru.mtusi.messenger.repository.AttachmentRepository;
import java.io.ByteArrayInputStream;
import java.time.Duration;
import java.time.Instant;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/**
* Интеграционные тесты для Grade 4 и Track 2.
*
* <p>Grade 4: тесты на переполнение, повреждённые файлы, подмену MIME,
* чужие attachment_id, привязку к чату.</p>
*
* <p>Track 2: error injection, reconciler detection/repair,
* proof fresh attachments aren't deleted, traversal cost measurement.</p>
*/
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Slf4j
class AttachmentIntegrationTest {
@Autowired
private AttachmentService attachmentService;
@Autowired
private AttachmentRepository attachmentRepository;
@Autowired
private MinioClient minio;
@Autowired
private S3Properties s3Props;
private String bucket;
@BeforeEach
void setUp() {
bucket = s3Props.bucket();
attachmentRepository.deleteAll();
}
// ================================================================
// Grade 4: Validation and error handling tests
// ================================================================
@Test
@DisplayName("Grade 4: Reject file exceeding max size (10 MiB)")
void shouldRejectOversizedFile() {
byte[] oversized = new byte[11 * 1024 * 1024];
System.arraycopy(new byte[]{(byte) 0x89, 0x50, 0x4E, 0x47}, 0, oversized, 0, 4);
assertThatThrownBy(() ->
attachmentService.upload(
UUID.randomUUID(), UUID.randomUUID(), "big.png", oversized))
.isInstanceOf(Exception.class);
}
@Test
@DisplayName("Grade 4: Reject corrupted image (valid header, corrupted body)")
void shouldRejectCorruptedImage() {
byte[] corrupted = new byte[1024];
System.arraycopy(new byte[]{(byte) 0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A}, 0, corrupted, 0, 8);
assertThatThrownBy(() ->
attachmentService.upload(
UUID.randomUUID(), UUID.randomUUID(), "corrupted.png", corrupted))
.isInstanceOf(Exception.class);
}
@Test
@DisplayName("Grade 4: MIME checked by content not filename")
void shouldCheckMimeByContent() {
byte[] jpegData = new byte[]{
(byte) 0xFF, (byte) 0xD8, (byte) 0xFF, (byte) 0xE0,
0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00, 0x01,
0x01, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00
};
AttachmentDTO att = attachmentService.upload(
UUID.randomUUID(), UUID.randomUUID(), "fake.png", jpegData);
assertThat(att.getContentType()).isEqualTo("image/jpeg");
}
@Test
@DisplayName("Grade 4: Reject foreign attachment access (user not in chat)")
void shouldRejectForeignAttachmentAccess() {
UUID chatId = UUID.randomUUID();
UUID ownerId = UUID.randomUUID();
UUID outsiderId = UUID.randomUUID();
byte[] pngData = createMinimalPng();
AttachmentDTO att = attachmentService.upload(chatId, ownerId, "test.png", pngData);
assertThatThrownBy(() ->
attachmentService.getMetadata(att.getId(), outsiderId))
.isInstanceOf(Exception.class);
}
// ================================================================
// Track 2: Error injection and reconciler tests
// ================================================================
@Test
@DisplayName("Track 2: Reconciler detects and repairs stale UPLOADING attachment")
void shouldDetectAndRepairStaleAttachment() {
// Inject: create attachment with UPLOADING status and old timestamp
Attachment stale = new Attachment();
String orphanKey = UUID.randomUUID().toString();
stale.setObjectKey(orphanKey);
stale.setFilename("stale.png");
stale.setContentType("image/png");
stale.setSizeBytes(100L);
stale.setChecksum("abc123");
stale.setStatus(AttachmentStatus.UPLOADING);
stale.setCreatedAt(Instant.now().minus(Duration.ofMinutes(20)));
stale.setUpdatedAt(stale.getCreatedAt());
attachmentRepository.save(stale);
// Inject: create orphan in S3
try {
minio.putObject(PutObjectArgs.builder()
.bucket(bucket)
.object(orphanKey)
.stream(new ByteArrayInputStream(new byte[100]), 100L, -1L)
.build());
} catch (Exception e) {
log.warn("Failed to inject orphan: {}", e.getMessage());
}
// Run reconciler
AttachmentReconciler reconciler = new AttachmentReconciler(
attachmentRepository, minio, s3Props,
new ReconcilerProperties(true, 3600000L, 15, false, 100, 3, "30m", "1m"));
reconciler.reconcileStaleAttachments();
// Verify: stale attachment is now BROKEN
Attachment updated = attachmentRepository.findAll().stream()
.filter(a -> a.getObjectKey().equals(orphanKey))
.findFirst().orElse(null);
if (updated != null) {
assertThat(updated.getStatus()).isEqualTo(AttachmentStatus.BROKEN);
}
}
@Test
@DisplayName("Track 2: Reconciler dry-run does not modify anything")
void shouldNotModifyInDryRun() {
Attachment stale = new Attachment();
stale.setObjectKey(UUID.randomUUID().toString());
stale.setFilename("stale.png");
stale.setContentType("image/png");
stale.setSizeBytes(100L);
stale.setChecksum("abc");
stale.setStatus(AttachmentStatus.UPLOADING);
stale.setCreatedAt(Instant.now().minus(Duration.ofMinutes(20)));
stale.setUpdatedAt(stale.getCreatedAt());
attachmentRepository.save(stale);
int beforeCount = (int) attachmentRepository.count();
// Run reconciler in dry-run mode
AttachmentReconciler reconciler = new AttachmentReconciler(
attachmentRepository, minio, s3Props,
new ReconcilerProperties(true, 3600000L, 15, true, 100, 3, "30m", "1m"));
reconciler.reconcileStaleAttachments();
// Verify: nothing changed
assertThat(attachmentRepository.count()).isEqualTo(beforeCount);
assertThat(stale.getStatus()).isEqualTo(AttachmentStatus.UPLOADING);
}
@Test
@DisplayName("Track 2: Reconciler does not delete fresh attachments")
void shouldNotDeleteFreshAttachments() {
UUID chatId = UUID.randomUUID();
UUID userId = UUID.randomUUID();
byte[] pngData = createMinimalPng();
AttachmentDTO fresh = attachmentService.upload(chatId, userId, "fresh.png", pngData);
// Run reconciler
AttachmentReconciler reconciler = new AttachmentReconciler(
attachmentRepository, minio, s3Props,
new ReconcilerProperties(true, 3600000L, 15, false, 100, 3, "30m", "1m"));
reconciler.verifyReadyAttachments();
// Verify: fresh attachment still exists and is READY
Attachment updated = attachmentRepository.findById(fresh.getId()).orElse(null);
assertThat(updated).isNotNull();
assertThat(updated.getStatus()).isEqualTo(AttachmentStatus.READY);
}
@Test
@DisplayName("Track 2: Measure reconciler traversal cost")
void shouldMeasureTraversalCost() {
UUID chatId = UUID.randomUUID();
UUID userId = UUID.randomUUID();
int attachmentCount = 50;
byte[] pngData = createMinimalPng();
for (int i = 0; i < attachmentCount; i++) {
attachmentService.upload(chatId, userId, "test-" + i + ".png", pngData);
}
AttachmentReconciler reconciler = new AttachmentReconciler(
attachmentRepository, minio, s3Props,
new ReconcilerProperties(true, 3600000L, 15, false, 100, 3, "30m", "1m"));
long start = System.currentTimeMillis();
reconciler.verifyReadyAttachments();
long duration = System.currentTimeMillis() - start;
log.info("Reconciler traversal cost: {}ms for {} attachments ({:.2f}ms per attachment)",
duration, attachmentCount, (double) duration / attachmentCount);
// Verify: all attachments still READY
List<Attachment> all = attachmentRepository.findAll();
assertThat(all).hasSize(attachmentCount);
assertThat(all.stream().allMatch(a -> a.getStatus() == AttachmentStatus.READY)).isTrue();
}
@Test
@DisplayName("Track 2: Concurrent uploads do not corrupt state")
void shouldHandleConcurrentUploads() throws InterruptedException {
UUID chatId = UUID.randomUUID();
UUID userId = UUID.randomUUID();
byte[] pngData = createMinimalPng();
int threadCount = 10;
CountDownLatch startLatch = new CountDownLatch(1);
CountDownLatch doneLatch = new CountDownLatch(threadCount);
AtomicInteger successCount = new AtomicInteger(0);
AtomicInteger failureCount = new AtomicInteger(0);
ExecutorService executor = Executors.newFixedThreadPool(threadCount);
for (int i = 0; i < threadCount; i++) {
final int index = i;
executor.submit(() -> {
try {
startLatch.await();
attachmentService.upload(chatId, userId, "concurrent-" + index + ".png", pngData);
successCount.incrementAndGet();
} catch (Exception e) {
failureCount.incrementAndGet();
log.error("Upload {} failed: {}", index, e.getMessage());
} finally {
doneLatch.countDown();
}
});
}
long before = System.currentTimeMillis();
startLatch.countDown();
doneLatch.await(30, TimeUnit.SECONDS);
long duration = System.currentTimeMillis() - before;
executor.shutdown();
executor.awaitTermination(10, TimeUnit.SECONDS);
log.info("Concurrent uploads: {} success, {} failures, {}ms",
successCount.get(), failureCount.get(), duration);
assertThat(failureCount.get()).isLessThanOrEqualTo(1);
assertThat(attachmentRepository.count()).isGreaterThanOrEqualTo(threadCount - 1);
}
// ================================================================
// Helpers
// ================================================================
private byte[] createMinimalPng() {
return new byte[]{
(byte) 0x89, (byte) 0x50, (byte) 0x4E, (byte) 0x47,
(byte) 0x0D, (byte) 0x0A, (byte) 0x1A, (byte) 0x0A,
(byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x0D,
(byte) 0x49, (byte) 0x48, (byte) 0x44, (byte) 0x52,
(byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x01,
(byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x01,
(byte) 0x08, (byte) 0x06, (byte) 0x00, (byte) 0x00,
(byte) 0x00, (byte) 0x1F, (byte) 0x15, (byte) 0xC4,
(byte) 0x89, (byte) 0x00, (byte) 0x00, (byte) 0x00,
(byte) 0x0A, (byte) 0x49, (byte) 0x44, (byte) 0x41,
(byte) 0x54, (byte) 0x78, (byte) 0x9C, (byte) 0x63,
(byte) 0x00, (byte) 0x01, (byte) 0x00, (byte) 0x00,
(byte) 0x05, (byte) 0x00, (byte) 0x01, (byte) 0x0D,
(byte) 0x0A, (byte) 0x2D, (byte) 0xB4,
(byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00,
(byte) 0x49, (byte) 0x45, (byte) 0x4E, (byte) 0x44,
(byte) 0xAE, (byte) 0x42, (byte) 0x60, (byte) 0x82
};
}
}