ЛР 4 — БВТ2303 — Кучерявенко — на 4 #66

Closed
Mraksimus wants to merge 4 commits from Mraksimus/mtusi2026lab4-fork:solution/bvt2303-mraksimus into main
211 changed files with 9697 additions and 21 deletions
+20
View File
@@ -18,3 +18,23 @@ __pycache__
node_modules
evidence
.DS_Store
.gradle
.kotlin
build
.idea
*.iml
tests
scripts
examples
run
COURSE.md
CONTRIBUTING.md
MAINTAINERS.md
REPORT.md
README.md
requirements-dev.txt
Dockerfile.example
compose.example.yaml
Makefile
archive
+2 -2
View File
@@ -14,7 +14,7 @@ IDEMPOTENCY_TTL_SECONDS=86400
S3_ENDPOINT=http://s3:9000
S3_REGION=us-east-1
S3_BUCKET=messenger-attachments
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_api_access_key_id
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_api_secret_access_key
MAX_UPLOAD_BYTES=10485760
MAX_IMAGE_PIXELS=40000000
+4
View File
@@ -28,5 +28,9 @@ coverage/
.DS_Store
.idea/
.vscode/
.gradle/
.kotlin/
!gradle/wrapper/
/run/development/
# Keep small, sanitized evidence summaries in evidence/ under version control.
# Inspect git diff --cached: ignore rules are not a secret scanner.
+23
View File
@@ -0,0 +1,23 @@
FROM gradle:8.14-jdk21 AS build
WORKDIR /src
COPY settings.gradle.kts build.gradle.kts gradle.properties ./
COPY gradle ./gradle
COPY gradlew ./
COPY src ./src
COPY contracts ./contracts
RUN --mount=type=cache,target=/home/gradle/.gradle gradle --no-daemon shadowJar
FROM eclipse-temurin:21-jre-alpine
# Distinguishes builds of the same code, e.g. the image rolled out by scripts/rolling-update.sh.
ARG BUILD_ID=dev
LABEL org.opencontainers.image.revision=$BUILD_ID
RUN apk add --no-cache wget && addgroup -S app && adduser -S -G app app
WORKDIR /app
COPY --from=build /src/build/libs/messenger.jar /app/messenger.jar
USER app
ENV APP_HOST=0.0.0.0 APP_PORT=8080
EXPOSE 8080
# Docker's own view of the replica (compose depends_on, rolling update). The balancer runs its own checks every 1 s.
HEALTHCHECK --interval=10s --timeout=3s --start-period=30s --start-interval=1s --retries=3 \
CMD wget -qO- http://127.0.0.1:8080/health/ready || exit 1
ENTRYPOINT ["java","-XX:MaxRAMPercentage=75.0","-jar","/app/messenger.jar"]
+116 -19
View File
@@ -1,37 +1,134 @@
# Отчёт — лабораторная 4
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: Кучерявенко Максим, БВТ2303
- Целевая оценка: **4**
- Commit/PR: после публикации.
- Основа: собственный lab3; структура конфигураций, controllers → services → repositories. Для S3 используется AWS SDK for Kotlin 1.6.41; изображения проверяет JDK ImageIO. ИИ использован как парный разработчик.
- Источники: [AWS SDK for Kotlin](https://github.com/awslabs/aws-sdk-kotlin), [релиз MinIO](https://github.com/minio/minio/releases/tag/RELEASE.2025-09-07T16-13-09Z), [ImageReader](https://docs.oracle.com/en/java/javase/21/docs/api/java.desktop/javax/imageio/ImageReader.html), [S3 uploads](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html).
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Нужны Docker с Compose v2, Python 3.9+, openssl. Для JVM-тестов нужны JDK 21+ и Docker (Testcontainers самостоятельно поднимает PostgreSQL, Redis и MinIO).
```sh
./scripts/init-secrets.sh
# Если 8080 занят lab3, как при проверке этой работы:
export LAB_HTTP_PORT=8084
export BASE_URL=http://localhost:8084
# Для свободного стандартного порта можно использовать 8080 в обеих переменных.
docker compose config --quiet
docker compose up -d --build --wait
make check
make test
./gradlew test
# Остановка именно реплики, принявшей upload; скрипт сам возвращает её в работу.
python3 scripts/attachment-failover.py
# Проверка lab3: запись сообщений во время отказа, идемпотентность, чтение файла.
make test-failover ACTION_SCRIPT=scripts/stop-one.sh
docker compose start api
# Перезапуск LB, обеих API-реплик, PostgreSQL, Redis и MinIO без удаления volumes.
make test-persistence ACTION_SCRIPT=scripts/restart.sh
# Проверки уровня 4:
scripts/check-s3-permissions.sh
python3 scripts/attachment-interrupt.py
python3 scripts/attachment-load.py > evidence/attachment-load.json
# Безопасная остановка, данные сохраняются:
docker compose down
# Только для намеренного удаления всех учебных данных:
# docker compose down -v
```
Compose собирает общий образ `mtusi-messenger:lab4` через `api`; одноразовый `migrate` использует тот же образ и применяет Flyway V1–V5 до старта реплик. `s3-init` ждёт MinIO, создаёт bucket через `mc mb --ignore-existing` и закрывает анонимный доступ через `mc anonymous set none`. Повторный запуск сохраняет bucket и объекты. Секреты создаются локально, существующие файлы не перезаписываются; `.env`, `secrets/` и `run/development/` не входят в Git.
MinIO RELEASE.2025-09-07T16-13-09Z и mc RELEASE.2025-08-13T08-35-41Z закреплены digest в Compose. PostgreSQL 17.6, Redis 8.4.0, HAProxy 3.2.4 — из lab3. API использует Kotlin 2.3.20, Ktor 3.3.0, JDK 21. Проверено на macOS arm64 с Docker 28.4.0. MinIO S3 и console не публикуются на host; наружу доступен только LB на loopback.
### Настройки
`.env` необязателен. В Compose есть значения по умолчанию; `docker compose --env-file .env.example config --quiet` также проходит. Переменные из skeleton сопоставлены так:
- `API_REPLICAS`, `SESSION_TTL_SECONDS`, `IDEMPOTENCY_TTL_SECONDS`, `S3_BUCKET` используются в Compose. Дополнительно `LAB_HTTP_PORT` задаёт опубликованный порт LB.
- `S3_ENDPOINT=http://s3:9000`, `S3_ACCESS_KEY_ID_FILE`, `S3_SECRET_ACCESS_KEY_FILE` явно передаются API в Compose; в локальном JVM-процессе читаются из environment или HOCON `storage`.
- `S3_REGION=us-east-1` закреплён в SDK. `MAX_UPLOAD_BYTES=10485760` и `MAX_IMAGE_PIXELS=40000000` закреплены в валидаторе по обязательному контракту; переменными окружения не увеличиваются.
- Вместо предложенных skeleton `DATABASE_URL_FILE` и `CACHE_URL_FILE` используются `DB_HOST/PORT/NAME/USER/PASSWORD_FILE` и `REDIS_HOST/PORT/PASSWORD_FILE`. PostgreSQL всегда источник истины для сессий; `SESSION_STORE` не переключает реализацию. `APP_ENV` не используется.
- `APP_HOST/PORT` задаются внутри Compose, порт приложения — 8080. `LAB_HTTP_PORT` меняет только внешний порт.
Для отдельного `./gradlew run` задача создаёт недостающие файлы `run/development/config` из `_template`, сохраняя существующие настройки. Нужно настроить доступные локальному JVM PostgreSQL, Redis и S3; изолированные адреса Compose с host не доступны. Полностью воспроизводимый путь проверки выше использует Compose.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
Клиент → HAProxy → две Ktor API-реплики → PostgreSQL (метаданные, связи, сессии) и MinIO (байты). Redis остаётся кэшем сессий с fallback в PostgreSQL. API-реплики взаимозаменяемы, sticky sessions нет.
`attachments` содержит ID, chat_id, owner_id, отображаемое имя, MIME, размер, SHA-256 и время создания. `message_attachments` хранит порядок вложений сообщения, уникальность и внешние ключи. S3 key — `originals/<server-generated UUID>`; filename никогда не используется как путь. В API нет ключей S3 и credentials. В lab4 все опубликованные метаданные уже ready, updated_at равен created_at, variants=[], error_code=null.
Upload: Bearer-аутентификация → проверка членства → filename/MIME/размер → ограниченное чтение тела → проверка изображения → COMMIT записи журнала → S3 PUT → INSERT метаданных и удаление записи журнала в одной транзакции → 201. Нет долгой DB-транзакции во время HTTP upload или S3 PUT. Оригинальные bytes не перекодируются. Download: аутентификация → доступ через chat_id в PostgreSQL → S3 GET → bytes с истинным MIME через API без redirect. Посторонний и неизвестный ID получают одинаковый 404.
Сообщение допускает пустой текст с 1–10 уникальными вложениями этого чата. Член чата может прикрепить загруженный другим участником файл. Создание сообщения, связей и результата Idempotency-Key выполняется в одной транзакции. Отсутствующий attachment_ids и [] равнозначны; повтор возвращает тот же Message. Hash дополнен вложениями, при replay дополнительно сравниваются исходный текст и список ID. Старые текстовые ключи lab3 остаются пригодны для replay. Сохраняются keyset-пагинация, отзыв сессий между репликами и graceful shutdown. Вложения всей страницы выбираются одним дополнительным DSL-запросом и группируются по message_id; тест страницы из 100 сообщений проверяет ровно два SQL-запроса и сохранение порядка вложений.
### Ограничения входных данных
- Тело читается из канала порциями до 8192 bytes, с проверкой лимита до записи во временный файл. Превышение Content-Length отсекается сразу; при отсутствии длины применяется тот же счётчик bytes. Максимум — 10 MiB, превышение — 413; приём тела ограничен 15 секундами.
- ImageIO определяет реальный формат, допускаются PNG/JPEG и совпадение MIME. Размеры считываются из заголовка до `reader.read`; произведение считается в Long, более 40 млн пикселей — 413.
- Затем выполняется полное декодирование, чтобы одного корректного заголовка было недостаточно. Ошибки/предупреждения декодера — 415. Для PNG дополнительно проверяются границы chunks, CRC и IEND, для JPEG — завершающий EOI. ImageIO использует `ignoreMetadata=true`: сжатые zTXt/iTXt не накапливаются. Регрессионный тест покрывает 5000 текстовых chunks.
- Filename: 1–255 Unicode code points, не только пробелы; `/`, `\`, NUL и управляющие символы — 400.
- Приём HTTP и S3 PUT потоковые: тело поступает в уникальный временный файл, AWS SDK читает его через `asByteStream()`. `withTemporaryUpload` создаёт файл и выполняет весь жизненный цикл внутри `Dispatchers.IO`; `finally` удаляет файл при успехе, ошибке и отмене, без потери ресурса при переключении контекста. В Compose `/tmp` — tmpfs 32 MiB, очищаемый при restart; это не постоянное хранилище оригиналов. Семафор допускает одну загрузку на реплику, остальные ждут без собственного буфера файла. Валидация всё ещё использует ограниченный массив ≤10 MiB и raster ≤40 млн пикселей; это честная граница потоковой обработки, а не обещание zero-copy. API ограничен 1 GiB и 1 CPU. У download также есть предел 10 MiB.
- S3-вызов ограничен coroutine timeout в 8 с, включая получение тела ответа. HAProxy server timeout увеличен до 30 с для синхронной загрузки. Автоматическое повторение POST после передачи запроса не включается.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
Проверки выполнены 05.11.2026, через `http://localhost:8084`. Ниже актуальные результаты lab4; материалы в `archive/lab3` исторические.
| Критерий | Команда/сценарий | Наблюдение | Доказательство |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
| Контракт, 201 ready, checksum, сообщение с вложением, ACL | `make test` | PASS публичного сценария lab4 | [smoke.txt](evidence/smoke.txt) |
| Возможности lab3 и новые проверки | `./gradlew test` | 80 тестов, 0 failures/errors/skipped | [gradle-tests.txt](evidence/gradle-tests.txt) |
| PNG/JPEG, MIME, опасное имя, лимиты bytes/pixels до декодирования | AttachmentControllerTest, ImageValidatorTest | Оба формата скачиваются побайтно; 400/413/415 на неверный ввод | [gradle-tests.txt](evidence/gradle-tests.txt) |
| Приватное S3, чужой чат, отказ именно загрузившей реплики | `python3 scripts/attachment-failover.py` | 403 для anonymous S3 GET известного объекта; 404 постороннему; иной X-Instance-Id и те же bytes после отказа | [attachment-failover.txt](evidence/attachment-failover.txt) |
| Сохранность после restart всех компонентов | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | PASS, прежняя сессия, метаданные и оригинал доступны; StartedAt всех процессов изменился | [persistence.txt](evidence/persistence.txt) |
| Запись сообщений во время отказа API | `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` | Проверка восстановления записи, отсутствия дублей и сохранности данных | [failover.txt](evidence/failover.txt) |
| Ошибочный Content-Type и пробельное имя | AttachmentControllerTest | 400 вместо 500/201 | [gradle-tests.txt](evidence/gradle-tests.txt) |
| Недоступность S3 | AttachmentControllerTest: pause MinIO | Upload/content/ready → 503, live → 200; после unpause исходные bytes доступны | [gradle-tests.txt](evidence/gradle-tests.txt) |
| Негативные запросы уровня 4 | AttachmentControllerTest | 413 для большого файла, 415 для повреждения/MIME, 404 для чужого ID и другого чата | [gradle-tests.txt](evidence/gradle-tests.txt) |
| Сбой между S3 и БД, grace и повторная очистка | AttachmentCleanupTest | Свежие/готовые объекты сохранены, orphan и поздний PUT удалены, публикация abandoned запрещена | [gradle-tests.txt](evidence/gradle-tests.txt) |
| Обрыв клиента | `python3 scripts/attachment-interrupt.py` | Нет опубликованных метаданных и временных файлов после закрытия соединения | [attachment-interrupt.txt](evidence/attachment-interrupt.txt) |
| 10 одновременных загрузок, память | `python3 scripts/attachment-load.py` | 10 успешных upload/download, sampled peak ниже лимита реплик | [attachment-load.json](evidence/attachment-load.json) |
| Отдельные права и повторяемый bootstrap | `scripts/check-s3-permissions.sh` | Запрещённые операции отклонены, console недоступна из общей сети | [s3-permissions.txt](evidence/s3-permissions.txt) |
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
## Согласованность и очистка уровня 4
## Эксперименты и отказы
Миграция V5 добавляет `attachment_uploads`. Перед S3 PUT в PostgreSQL фиксируется ID будущего объекта и срок действия (время БД + 1 час). После PUT публикация берёт строку `FOR UPDATE`, проверяет срок и отсутствие признака `abandoned`, затем атомарно создаёт Attachment и удаляет запись журнала. Поэтому подтверждённый объект не попадает в очистку.
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Каждая реплика раз в 30 секунд выбирает до 100 просроченных записей через `FOR UPDATE SKIP LOCKED`, помечает их abandoned и переносит следующую попытку на минуту вперёд. После COMMIT, без долгой DB-транзакции, удаляет соответствующие S3-объекты. Ошибка S3 оставляет запись для повтора. Публикация и очистка сериализуются блокировкой одной строки: либо метаданные опубликованы, либо загрузка окончательно abandoned.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
Записи abandoned сохраняются и периодически обрабатываются повторно даже после успешного DELETE. Это намеренно: процесс, замороженный дольше grace period, может поздно закончить PUT. Опубликовать метаданные он уже не сможет, а повторная очистка удалит поздний объект. Без сохранения такой записи первая уборка могла бы пропустить позднюю запись навсегда. Плата — рост журнала неудачных загрузок и повторные DELETE отсутствующих объектов; политика удаления tombstone требует отдельного ограничения максимального времени жизни загрузчика. Успешные загрузки строки журнала не накапливают.
## Ограничения и следующие шаги
Тест `AttachmentCleanupTest` воспроизводит сбой после фактического PUT до записи метаданных, сохраняет свежий объект внутри grace, сдвигает срок записи для проверки уборки без часового ожидания, моделирует отказ DELETE и повтор, поздний PUT и отказ публикации. Подтверждённый объект остаётся доступен. Глобального обхода всего bucket и восстановления вручную удалённых объектов нет — это отдельный исследовательский трек 5.
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
## S3-права
`secrets/s3_access_key_id` и `s3_secret_access_key` используются только MinIO и bootstrap. API получает отдельные `s3_api_access_key_id` и `s3_api_secret_access_key`, административные файлы в него не монтируются. Прикладная policy разрешает `ListBucket` для рабочего bucket (нужен HeadBucket/readiness) и только Get/Put/DeleteObject для `originals/*`. API не может управлять пользователями, политиками, создавать buckets или записывать другие prefixes. Console слушает только `127.0.0.1:9001` внутри контейнера MinIO, поэтому недоступна даже из общей сети Compose.
Bootstrap повторно создаёт/обновляет пользователя и policy с теми же локальными credentials и не трогает объекты. `check-s3-permissions.sh` запускает bootstrap дважды и проверяет реальные отказы запрещённых операций. Анонимный GET известного существующего объекта отдельно получает 403 в `attachment-failover.py`.
## Эксперимент с параллельными загрузками
`attachment-load.py` синхронно выпускает 10 клиентов через Barrier. Каждый отправляет реальный PNG 1600×1600, 7 684 003 bytes (псевдослучайные RGB-пиксели, seed 42), затем скачивает оригинал вторым участником и проверяет SHA-256. Профиль — один burst без повторов, две API-реплики, по одной активной обработке на реплику; очередь включена во время HTTP-ответа. Docker 28.4.0, macOS arm64, лимиты каждой API: 1 CPU / 1 GiB. Это функциональная проверка ограничения памяти, не определение предельного throughput.
Память измеряется через `memory.current` cgroup v2 каждой API, с паузой 0,2 с между обходами (плюс время docker exec). Включает JVM/native и tmpfs, не только heap. Замер дискретный, поэтому краткие пики между отсчётами могут быть пропущены. Raw-отсчёты, исходные значения и результаты всех запросов: [attachment-load.json](evidence/attachment-load.json).
Все 10 upload вернули 201, все 10 download совпали по SHA-256. Наблюдаемые пики по репликам: 335579ab19d6: 282.4 MiB, a8ee56f6ec22: 259.0 MiB. Время upload с ожиданием очереди — от 1.030 до 2.973 с; ошибок не было.
## Ограничения
201 подтверждается только после успешных S3 PUT и COMMIT PostgreSQL. Между этими хранилищами нет общей транзакции. При падении после PUT, до COMMIT, возможен объект без метаданных. Такой объект учтён в журнале и удаляется после grace period описанным выше механизмом. При потере ответа upload его повтор может создать ещё одно вложение, как разрешено контрактом.
Вход и S3 PUT ограничены по bytes и времени; валидация допускает до 40 млн пикселей и только один декодер на реплику. Проведённый burst не доказывает работу при любом количестве соединений, любой комбинации размеров и намеренном истощении ресурсов. Очередь не обещает фиксированное время ответа при перегрузке; HAProxy сохраняет свои таймауты. Случайные orphan-объекты, созданные вручную или прежней версией до журнала V5, автоматическая очистка журнала не обнаруживает.
Readiness требует PostgreSQL и S3, Redis допускает fallback. При отказе S3 HAProxy выведет обе реплики из ротации; LB может вернуть собственный 503 вместо JSON API. Прямые обращения к API при отказе S3 проверены тестом. Одноузловые PostgreSQL, MinIO и LB остаются точками отказа: volumes переживают restart, но не заменяют репликацию и backup. Сохраняются ограничения конкурентной keyset-пагинации и часов из [отчёта lab3](archive/lab3/REPORT.md).
+125
View File
@@ -0,0 +1,125 @@
plugins {
alias(libs.plugins.kotlin)
alias(libs.plugins.kotlin.serialization)
alias(libs.plugins.ksp)
alias(libs.plugins.shadow)
id("application")
}
group = "com.mtusi.messenger"
version = "2.0.0"
repositories {
mavenCentral()
}
dependencies {
implementation(libs.kotlinx.serialization.json)
implementation(libs.kotlinx.serialization.hocon)
implementation(libs.logback.classic)
implementation(ktorLibs.server.core)
implementation(ktorLibs.server.netty)
implementation(ktorLibs.server.auth)
implementation(ktorLibs.server.contentNegotiation)
implementation(ktorLibs.server.statusPages)
implementation(ktorLibs.server.callLogging)
implementation(ktorLibs.server.defaultHeaders)
implementation(ktorLibs.server.cors)
implementation(ktorLibs.server.swagger)
implementation(ktorLibs.serialization.kotlinx.json)
implementation(libs.koin.core)
implementation(libs.koin.ktor)
implementation(libs.koin.annotations)
ksp(libs.koin.compiler)
implementation(libs.exposed.core)
implementation(libs.exposed.jdbc)
implementation(libs.exposed.kotlin.datetime)
implementation(libs.exposed.migration.core)
implementation(libs.exposed.migration.jdbc)
implementation(libs.postgres)
implementation(libs.hikaricp)
implementation(libs.flyway)
implementation(libs.flyway.postgresql)
implementation(libs.lettuce)
implementation(awssdk.services.s3)
implementation(libs.bouncycastle)
testImplementation(kotlin("test"))
testImplementation(ktorLibs.server.testHost)
testImplementation(ktorLibs.client.contentNegotiation)
testImplementation(libs.kotlinx.coroutines.test)
testImplementation(libs.koin.test)
testImplementation(libs.testContainers)
testImplementation(libs.testContainers.postgresql)
}
kotlin {
jvmToolchain(21)
}
ksp {
arg("KOIN_CONFIG_CHECK", "true")
arg("KOIN_DEFAULT_MODULE", "false")
}
application {
mainClass = "com.mtusi.messenger.main.AppKt"
applicationDefaultJvmArgs = listOf("-Dio.ktor.development=false")
}
sourceSets {
named("main") {
resources {
srcDir(layout.projectDirectory.dir("contracts"))
}
}
}
tasks {
test {
useJUnitPlatform()
}
shadowJar {
archiveBaseName = "messenger"
archiveClassifier = ""
archiveVersion = ""
mergeServiceFiles()
}
getByName<JavaExec>("run") {
workingDir = file("run/development")
doFirst {
val configDir = file("run/development/config")
configDir.mkdirs()
file("run/_template/config").listFiles()
?.filter { it.isFile && it.extension == "conf" }
?.forEach { template ->
val target = configDir.resolve(template.name)
if (!target.exists()) template.copyTo(target)
}
args = configDir.listFiles()
?.filter { it.extension == "conf" }
?.sortedBy { it.name }
?.map { "-config=config/${it.name}" }
?: emptyList()
}
}
register<JavaExec>("generateMigration") {
group = "database"
description = "Generates a new migration"
classpath = sourceSets["main"].runtimeClasspath
mainClass.set("com.mtusi.messenger.main.plugins.DatabaseKt")
}
}
+216
View File
@@ -0,0 +1,216 @@
name: mtusi-messenger-lab4
services:
postgres:
image: postgres:17.6-alpine3.22
command: ["postgres", "-c", "shared_preload_libraries=pg_stat_statements", "-c", "pg_stat_statements.track=all"]
environment:
POSTGRES_DB: messenger
POSTGRES_USER: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
secrets:
- db_password
volumes:
- postgres-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 5s
timeout: 3s
retries: 10
start_period: 10s
deploy:
resources:
limits:
cpus: "1.0"
memory: 1g
restart: unless-stopped
redis:
image: redis:8.4.0-alpine
# Session cache only, PostgreSQL is the source of truth: no RDB/AOF, a restart simply empties the cache.
# Every key carries a TTL, so under memory pressure volatile-lru evicts the least used sessions.
command:
- sh
- -c
- >-
exec docker-entrypoint.sh redis-server
--requirepass "$$(cat /run/secrets/redis_password)"
--save ""
--appendonly no
--maxmemory 64mb
--maxmemory-policy volatile-lru
secrets:
- redis_password
healthcheck:
test: ["CMD-SHELL", "REDISCLI_AUTH=\"$$(cat /run/secrets/redis_password)\" redis-cli ping | grep -q PONG"]
interval: 5s
timeout: 3s
retries: 10
start_period: 5s
deploy:
resources:
limits:
cpus: "0.5"
memory: 128m
restart: unless-stopped
s3:
# MinIO RELEASE.2025-09-07T16-13-09Z, pinned multi-platform image.
image: quay.io/minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
command: ["server", "/data", "--console-address", "127.0.0.1:9001"]
environment:
MINIO_ROOT_USER_FILE: /run/secrets/s3_access_key_id
MINIO_ROOT_PASSWORD_FILE: /run/secrets/s3_secret_access_key
secrets:
- s3_access_key_id
- s3_secret_access_key
volumes:
- s3-data:/data
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 5s
timeout: 3s
retries: 10
deploy:
resources:
limits:
cpus: "1.0"
memory: 512m
restart: unless-stopped
s3-init:
# mc RELEASE.2025-08-13T08-35-41Z
image: quay.io/minio/mc@sha256:a7fe349ef4bd8521fb8497f55c6042871b2ae640607cf99d9bede5e9bdf11727
depends_on:
s3:
condition: service_healthy
entrypoint: ["/bin/sh", "/bootstrap.sh"]
environment:
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
secrets:
- s3_access_key_id
- s3_secret_access_key
- s3_api_access_key_id
- s3_api_secret_access_key
volumes:
- ./infra/s3/bootstrap.sh:/bootstrap.sh:ro
restart: "no"
# One-shot: applies Flyway migrations once and exits. Replicas start only after it succeeded,
# so they never race each other on the schema.
migrate:
image: mtusi-messenger:lab4
command: ["migrate"]
healthcheck:
test: ["NONE"]
depends_on:
postgres:
condition: service_healthy
environment:
DB_HOST: postgres
DB_PORT: "5432"
DB_NAME: messenger
DB_USER: messenger
DB_PASSWORD_FILE: /run/secrets/db_password
secrets:
- db_password
deploy:
resources:
limits:
cpus: "0.5"
memory: 384m
restart: "no"
api:
build: .
image: mtusi-messenger:lab4
# Interchangeable replicas: no container_name and no published port, traffic only through `lb`.
deploy:
replicas: ${API_REPLICAS:-2}
resources:
limits:
cpus: "1.0"
memory: 1g
depends_on:
migrate:
condition: service_completed_successfully
postgres:
condition: service_healthy
redis:
condition: service_healthy
s3-init:
condition: service_completed_successfully
environment:
APP_HOST: "0.0.0.0"
APP_PORT: "8080"
DB_HOST: postgres
DB_PORT: "5432"
DB_NAME: messenger
DB_USER: messenger
DB_PASSWORD_FILE: /run/secrets/db_password
DB_MIGRATE_ON_START: "false"
REDIS_HOST: redis
REDIS_PORT: "6379"
REDIS_PASSWORD_FILE: /run/secrets/redis_password
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
SESSION_CACHE_SINGLE_FLIGHT: ${SESSION_CACHE_SINGLE_FLIGHT:-true}
IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400}
SHUTDOWN_DRAIN_DELAY_MS: "3000"
S3_ENDPOINT: http://s3:9000
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_api_access_key_id
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_api_secret_access_key
secrets:
- db_password
- redis_password
- s3_api_access_key_id
- s3_api_secret_access_key
# SIGTERM: readiness 503 and 3 s drain (HAProxy needs <= 2 s to notice), then Ktor closes the socket and
# finishes requests in flight within 5 s. 3 + 5 = 8 s < 10 s before Docker sends SIGKILL.
tmpfs:
- /tmp:size=32m,mode=1777
stop_grace_period: 10s
restart: unless-stopped
lb:
image: haproxy:3.2.4-alpine
volumes:
- ./infra/haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
depends_on:
api:
condition: service_healthy
ports:
- "127.0.0.1:${LAB_HTTP_PORT:-8080}:8080"
# Port 8404 (health and stats) stays inside the compose network.
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8404/healthz"]
interval: 5s
timeout: 2s
retries: 3
deploy:
resources:
limits:
cpus: "0.5"
memory: 128m
restart: unless-stopped
secrets:
db_password:
file: ./secrets/db_password
redis_password:
file: ./secrets/redis_password
s3_access_key_id:
file: ./secrets/s3_access_key_id
s3_secret_access_key:
file: ./secrets/s3_secret_access_key
s3_api_access_key_id:
file: ./secrets/s3_api_access_key_id
s3_api_secret_access_key:
file: ./secrets/s3_api_secret_access_key
volumes:
postgres-data:
s3-data:
+6
View File
@@ -0,0 +1,6 @@
Upload: 201 ready; instance=335579ab19d6
Anonymous S3 GET of known object: 403
Outsider metadata/content: 404
Download: 200; instance=a8ee56f6ec22
SHA-256: 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
PASS: uploading replica stopped; original served by another replica; stopped replica restored
+1
View File
@@ -0,0 +1 @@
PASS: connection closed mid-body; temporary files removed on both replicas; metadata absent
+131
View File
@@ -0,0 +1,131 @@
{
"concurrency": 10,
"file_bytes": 7684003,
"pixels": 2560000,
"sha256": "de56c853868dc1ee1d4b0046023ed484ecd1e4d2ce2d403fbb661d81a414995b",
"baseline_bytes": {
"335579ab19d6": 193679360,
"a8ee56f6ec22": 204886016
},
"sampled_peak_bytes": {
"335579ab19d6": 296071168,
"a8ee56f6ec22": 271552512
},
"results": [
{
"upload": 0,
"seconds": 2.591,
"instance": "a8ee56f6ec22"
},
{
"upload": 1,
"seconds": 2.834,
"instance": "335579ab19d6"
},
{
"upload": 2,
"seconds": 2.52,
"instance": "335579ab19d6"
},
{
"upload": 3,
"seconds": 1.634,
"instance": "a8ee56f6ec22"
},
{
"upload": 4,
"seconds": 1.03,
"instance": "335579ab19d6"
},
{
"upload": 5,
"seconds": 2.973,
"instance": "a8ee56f6ec22"
},
{
"upload": 6,
"seconds": 2.087,
"instance": "335579ab19d6"
},
{
"upload": 7,
"seconds": 1.056,
"instance": "a8ee56f6ec22"
},
{
"upload": 8,
"seconds": 1.593,
"instance": "335579ab19d6"
},
{
"upload": 9,
"seconds": 2.096,
"instance": "a8ee56f6ec22"
}
],
"samples": [
{
"seconds": 0.0,
"bytes": {
"335579ab19d6": 196050944,
"a8ee56f6ec22": 210128896
}
},
{
"seconds": 0.404,
"bytes": {
"335579ab19d6": 212848640,
"a8ee56f6ec22": 223256576
}
},
{
"seconds": 0.749,
"bytes": {
"335579ab19d6": 241385472,
"a8ee56f6ec22": 242810880
}
},
{
"seconds": 1.075,
"bytes": {
"335579ab19d6": 248864768,
"a8ee56f6ec22": 259260416
}
},
{
"seconds": 1.489,
"bytes": {
"335579ab19d6": 266833920,
"a8ee56f6ec22": 249815040
}
},
{
"seconds": 1.838,
"bytes": {
"335579ab19d6": 290893824,
"a8ee56f6ec22": 257802240
}
},
{
"seconds": 2.244,
"bytes": {
"335579ab19d6": 293588992,
"a8ee56f6ec22": 271036416
}
},
{
"seconds": 2.607,
"bytes": {
"335579ab19d6": 296071168,
"a8ee56f6ec22": 271552512
}
},
{
"seconds": 3.007,
"bytes": {
"335579ab19d6": 284782592,
"a8ee56f6ec22": 263704576
}
}
]
}
+4
View File
@@ -0,0 +1,4 @@
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://localhost:8084"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 123, "failed_attempts": 0, "first_recovery_seconds": 5.537}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
+35
View File
@@ -0,0 +1,35 @@
Command: ./gradlew test
BUILD SUCCESSFUL
Tests: 80; failures: 0; errors: 0; skipped: 0
com.mtusi.messenger.main.infrastructure.controllers.attachment.AttachmentControllerTest: tests=3 failures=0
upload originals ACL message links and idempotency()
invalid images filenames and byte limit return client errors()
S3 outage fails upload download and readiness without losing the original()
com.mtusi.messenger.main.infrastructure.controllers.authentication.AuthenticationControllerTest: tests=9 failures=0
com.mtusi.messenger.main.infrastructure.controllers.chat.ChatControllerTest: tests=6 failures=0
com.mtusi.messenger.main.infrastructure.controllers.health.HealthControllerTest: tests=2 failures=0
com.mtusi.messenger.main.infrastructure.controllers.message.MessageConcurrencyTest: tests=1 failures=0
com.mtusi.messenger.main.infrastructure.controllers.message.MessageControllerTest: tests=7 failures=0
com.mtusi.messenger.main.infrastructure.controllers.message.MessageIdempotencyTest: tests=7 failures=0
com.mtusi.messenger.main.infrastructure.controllers.user.UserControllerTest: tests=11 failures=0
com.mtusi.messenger.main.infrastructure.repositories.authentication.session.cache.RedisSessionCacheRepositoryTest: tests=3 failures=0
com.mtusi.messenger.main.infrastructure.repositories.chat.ExposedChatRepositoryTest: tests=2 failures=0
com.mtusi.messenger.main.infrastructure.repositories.message.MessageAttachmentsQueryTest: tests=1 failures=0
history loads ordered attachment links in one query for the entire page()
com.mtusi.messenger.main.infrastructure.services.attachment.AttachmentCleanupTest: tests=1 failures=0
orphan cleanup respects grace and fences late publishers()
com.mtusi.messenger.main.infrastructure.services.attachment.ImageValidatorTest: tests=3 failures=0
compressed text metadata does not consume the decoded image memory budget()
corrupted CRC and missing end chunk are rejected()
pixel limit is enforced on header before decoding()
com.mtusi.messenger.main.infrastructure.services.authentication.CrossReplicaRevocationTest: tests=2 failures=0
com.mtusi.messenger.main.infrastructure.services.authentication.SessionLookupSingleFlightTest: tests=4 failures=0
com.mtusi.messenger.main.infrastructure.services.hashing.DefaultHashingServiceTest: tests=4 failures=0
com.mtusi.messenger.main.plugins.CorsTest: tests=2 failures=0
com.mtusi.messenger.main.plugins.DatabaseMigrationTest: tests=1 failures=0
com.mtusi.messenger.main.plugins.DependencyFailureTest: tests=5 failures=0
com.mtusi.messenger.main.plugins.InstanceIdTest: tests=1 failures=0
com.mtusi.messenger.main.plugins.SessionCacheTest: tests=3 failures=0
com.mtusi.messenger.main.utils.TemporaryUploadTest: tests=2 failures=0
temporary file is removed when upload is cancelled()
temporary file is removed on success and failure()
+17
View File
@@ -0,0 +1,17 @@
Before restart:
/mtusi-messenger-lab4-api-1 started=2026-11-05T11:21:33.808216129Z
/mtusi-messenger-lab4-api-2 started=2026-11-05T11:19:40.744427424Z
/mtusi-messenger-lab4-lb-1 started=2026-11-04T23:04:59.215310882Z
/mtusi-messenger-lab4-postgres-1 started=2026-11-04T23:04:47.265519668Z
/mtusi-messenger-lab4-redis-1 started=2026-11-04T23:04:47.259546585Z
/mtusi-messenger-lab4-s3-1 started=2026-11-05T11:16:50.746732095Z
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://localhost:8084"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
After restart:
/mtusi-messenger-lab4-api-1 started=2026-11-05T11:22:10.858998716Z
/mtusi-messenger-lab4-api-2 started=2026-11-05T11:22:10.784659257Z
/mtusi-messenger-lab4-lb-1 started=2026-11-05T11:22:13.944573717Z
/mtusi-messenger-lab4-postgres-1 started=2026-11-05T11:22:01.053742252Z
/mtusi-messenger-lab4-redis-1 started=2026-11-05T11:22:01.049022586Z
/mtusi-messenger-lab4-s3-1 started=2026-11-05T11:22:01.056079919Z
+10
View File
@@ -0,0 +1,10 @@
Container mtusi-messenger-lab4-s3-1 Running
Bucket created successfully `local/messenger-attachments`.
Access permission for `local/messenger-attachments` is set to `private`
Container mtusi-messenger-lab4-s3-1 Running
Bucket created successfully `local/messenger-attachments`.
Access permission for `local/messenger-attachments` is set to `private`
Container mtusi-messenger-lab4-s3-1 Running
PASS: app cannot administer MinIO, create buckets, change policy or write outside originals/
wget: can't connect to remote host (172.25.0.2): Connection refused
PASS: console unavailable from shared network
+2
View File
@@ -0,0 +1,2 @@
python3 tests/smoke.py --base-url "http://localhost:8084"
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
+1
View File
@@ -0,0 +1 @@
kotlin.code.style=official
+54
View File
@@ -0,0 +1,54 @@
[versions]
kotlin = "2.3.20"
ksp = "2.3.0"
kotlinx-serialization = "1.10.0"
kotlinx-coroutines = "1.9.0"
logback = "1.5.13"
shadow = "8.3.5"
koin = "4.1.0-RC1"
koin-ktor = "4.1.0-Beta8"
koin-compiler = "2.0.0"
exposed = "1.0.0"
postgres = "42.7.3"
hikaricp = "6.3.0"
flyway = "11.9.0"
lettuce = "6.7.1.RELEASE"
bouncycastle = "1.81"
test-containers = "1.21.0"
[plugins]
kotlin = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" }
kotlin-serialization = { id = "org.jetbrains.kotlin.plugin.serialization", version.ref = "kotlin" }
ksp = { id = "com.google.devtools.ksp", version.ref = "ksp" }
shadow = { id = "com.gradleup.shadow", version.ref = "shadow" }
[libraries]
kotlinx-serialization-json = { module = "org.jetbrains.kotlinx:kotlinx-serialization-json", version.ref = "kotlinx-serialization" }
kotlinx-serialization-hocon = { module = "org.jetbrains.kotlinx:kotlinx-serialization-hocon", version.ref = "kotlinx-serialization" }
kotlinx-coroutines-test = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-test", version.ref = "kotlinx-coroutines" }
logback-classic = { module = "ch.qos.logback:logback-classic", version.ref = "logback" }
koin-core = { module = "io.insert-koin:koin-core", version.ref = "koin" }
koin-ktor = { module = "io.insert-koin:koin-ktor3", version.ref = "koin-ktor" }
koin-annotations = { module = "io.insert-koin:koin-annotations", version.ref = "koin-compiler" }
koin-compiler = { module = "io.insert-koin:koin-ksp-compiler", version.ref = "koin-compiler" }
koin-test = { module = "io.insert-koin:koin-test", version.ref = "koin" }
exposed-core = { module = "org.jetbrains.exposed:exposed-core", version.ref = "exposed" }
exposed-jdbc = { module = "org.jetbrains.exposed:exposed-jdbc", version.ref = "exposed" }
exposed-kotlin-datetime = { module = "org.jetbrains.exposed:exposed-kotlin-datetime", version.ref = "exposed" }
exposed-migration-core = { module = "org.jetbrains.exposed:exposed-migration-core", version.ref = "exposed" }
exposed-migration-jdbc = { module = "org.jetbrains.exposed:exposed-migration-jdbc", version.ref = "exposed" }
postgres = { module = "org.postgresql:postgresql", version.ref = "postgres" }
hikaricp = { module = "com.zaxxer:HikariCP", version.ref = "hikaricp" }
flyway = { module = "org.flywaydb:flyway-core", version.ref = "flyway" }
flyway-postgresql = { module = "org.flywaydb:flyway-database-postgresql", version.ref = "flyway" }
lettuce = { module = "io.lettuce:lettuce-core", version.ref = "lettuce" }
bouncycastle = { module = "org.bouncycastle:bcprov-jdk18on", version.ref = "bouncycastle" }
testContainers = { module = "org.testcontainers:testcontainers", version.ref = "test-containers" }
testContainers-postgresql = { module = "org.testcontainers:postgresql", version.ref = "test-containers" }
Binary file not shown.
+7
View File
@@ -0,0 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.14-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Vendored Executable
+251
View File
@@ -0,0 +1,251 @@
#!/bin/sh
#
# Copyright © 2015-2021 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
#
##############################################################################
#
# Gradle start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh Gradle
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
# This is normally unused
# shellcheck disable=SC2034
APP_BASE_NAME=${0##*/}
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD=maximum
warn () {
echo "$*"
} >&2
die () {
echo
echo "$*"
echo
exit 1
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
CLASSPATH="\\\"\\\""
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD=java
if ! command -v java >/dev/null 2>&1
then
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
fi
# Increase the maximum file descriptors if we can.
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Collect all arguments for the java command:
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
# and any embedded shellness will be escaped.
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
# treated as '${Hostname}' itself on the command line.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-classpath "$CLASSPATH" \
-jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"
Vendored
+94
View File
@@ -0,0 +1,94 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@rem SPDX-License-Identifier: Apache-2.0
@rem
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if %ERRORLEVEL% equ 0 goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:execute
@rem Setup the command line
set CLASSPATH=
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %*
:end
@rem End local scope for the variables with windows NT shell
if %ERRORLEVEL% equ 0 goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega
+57
View File
@@ -0,0 +1,57 @@
# Single public entry point in front of the API replicas.
# Backends are discovered through Docker DNS (service name `api`) and receive traffic only while /health/ready is 200.
global
log stdout format raw local0 info
# Without it HAProxy sizes buffers from the file descriptor limit and uses ~100 MB.
maxconn 4096
defaults
mode http
log global
option httplog
option dontlognull
option forwardfor
timeout connect 1s
timeout client 30s
timeout server 30s
timeout http-request 10s
timeout http-keep-alive 5s
# Retried only when the connection to a replica fails, i.e. before the request was sent:
# a POST that reached a replica is never repeated by the balancer.
retries 2
option redispatch 1
resolvers docker
nameserver dns 127.0.0.11:53
resolve_retries 3
timeout resolve 1s
timeout retry 1s
hold valid 2s
hold other 2s
hold refused 2s
hold nx 2s
hold timeout 2s
hold obsolete 2s
accepted_payload_size 8192
frontend http
bind :8080
default_backend api
# Not published: container healthcheck and the backend table (wget -qO- "http://127.0.0.1:8404/stats;csv").
frontend internal
bind :8404
monitor-uri /healthz
stats enable
stats uri /stats
stats refresh 5s
backend api
balance roundrobin
option httpchk GET /health/ready
http-check expect status 200
# A replica leaves rotation after 2 failed checks 1 s apart and returns after 2 successful ones.
default-server check inter 1s fall 2 rise 2 resolvers docker init-addr none
# Up to 4 replicas; slots without a resolved address stay in maintenance.
server-template api 4 api:8080
+20
View File
@@ -0,0 +1,20 @@
#!/bin/sh
set -eu
# No tracing: credentials must not appear in logs.
mc alias set local http://s3:9000 "$(cat /run/secrets/s3_access_key_id)" "$(cat /run/secrets/s3_secret_access_key)" >/dev/null
mc mb --ignore-existing "local/$S3_BUCKET"
mc anonymous set none "local/$S3_BUCKET"
api_user=$(cat /run/secrets/s3_api_access_key_id)
mc admin user add local "$api_user" "$(cat /run/secrets/s3_api_secret_access_key)" >/dev/null
cat > /tmp/attachment-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{"Effect":"Allow","Action":["s3:ListBucket"],"Resource":["arn:aws:s3:::$S3_BUCKET"]},
{"Effect":"Allow","Action":["s3:GetObject","s3:PutObject","s3:DeleteObject"],"Resource":["arn:aws:s3:::$S3_BUCKET/originals/*"]}
]
}
EOF
mc admin policy create local attachments-api /tmp/attachment-policy.json >/dev/null
mc admin policy attach local attachments-api --user "$api_user" >/dev/null
+12
View File
@@ -0,0 +1,12 @@
ktor {
application {
modules = [com.mtusi.messenger.main.AppKt.module]
}
deployment {
host = "127.0.0.1"
port = 8080
}
}
+9
View File
@@ -0,0 +1,9 @@
authentication {
realm = "messenger"
sessionTtlSeconds = 3600
sessionCacheTtlSeconds = 60
# Collapse concurrent cache misses for one token into a single database lookup
sessionCacheSingleFlight = true
# A logout answers 204 only after its tombstone is in Redis; retries up to this long, then 503
sessionRevokeTimeoutMs = 8000
}
+23
View File
@@ -0,0 +1,23 @@
cors {
# Defaults to false
#allowAnyHost = true
# Swagger UI sends "Try it out" to the contract server http://localhost:8080; pages opened elsewhere need these.
allowedHosts = [
{ host = "localhost:8080", schemes = ["http"] }
{ host = "localhost:8081", schemes = ["http"] }
{ host = "127.0.0.1:8080", schemes = ["http"] }
{ host = "127.0.0.1:8081", schemes = ["http"] }
]
# Defaults to true
#allowAnyMethod = true
#allowedMethods = [GET, POST, PATCH, PUT, DELETE, OPTIONS]
allowAnyHeader = true
#allowedHeaders = []
exposedHeaders = [X-Request-Id, X-Instance-Id, WWW-Authenticate]
# Browsers send Origin on same-origin POST too, e.g. Swagger UI served by the API itself.
allowSameOrigin = true
}
+12
View File
@@ -0,0 +1,12 @@
database {
host = "localhost"
port = 5432
name = "messenger"
username = "username"
password = "password"
maxPoolSize = 10
connectionTimeoutMs = 2000
statementTimeoutMs = 5000
# A single local process migrates itself; replicas in compose leave it to the `migrate` service
migrateOnStart = true
}
+4
View File
@@ -0,0 +1,4 @@
instance {
# X-Instance-Id of this process. Defaults to HOSTNAME (the short container id in Docker), else a random alias
#id = "dev"
}
+4
View File
@@ -0,0 +1,4 @@
messages {
# How long a stored Idempotency-Key answers a retry with the original message (contract: at least 24 h)
idempotencyTtlSeconds = 86400
}
+8
View File
@@ -0,0 +1,8 @@
redis {
host = "localhost"
port = 6379
# Only if the local Redis requires AUTH
#password = ""
connectTimeoutMs = 500
commandTimeoutMs = 500
}
+6
View File
@@ -0,0 +1,6 @@
storage {
endpoint = "http://localhost:9000"
bucket = "messenger-attachments"
accessKeyFile = "../../secrets/s3_api_access_key_id"
secretKeyFile = "../../secrets/s3_api_secret_access_key"
}
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""Stop the exact API instance which accepted the upload, then verify the original.
Creates synthetic data; restores the stopped replica in finally. No credentials in output.
"""
import hashlib
import os
from pathlib import Path
import subprocess
import sys
import time
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / 'tests'))
from client import Client, Failure, check
def main():
c = Client(timeout=10)
alice, bob, eve, chat = c.bootstrap()
content = (ROOT / 'tests/fixtures/sample.png').read_bytes()
result = c.call('POST', '/api/v1/chats/' + chat['id'] + '/attachments?filename=sample.png',
201, actor=alice, raw=content, headers={'Content-Type': 'image/png'}, schema='Attachment')
attachment = result['json']
target = result['headers']['X-Instance-Id']
c.message(alice, chat, '', attachments=[attachment['id']])
print('Upload: 201 ready; instance=' + target, flush=True)
# Probe the real known object anonymously from the internal network, not an unknown key.
probe = subprocess.run(['docker', 'compose', 'exec', '-T', 'lb', 'wget', '-S', '-O', '/dev/null',
'http://s3:9000/' + os.environ.get('S3_BUCKET', 'messenger-attachments') + '/originals/' + attachment['id']],
cwd=ROOT, capture_output=True, text=True)
check(probe.returncode != 0 and '403' in probe.stderr, 'private bucket did not reject anonymous GET')
print('Anonymous S3 GET of known object: 403', flush=True)
path = '/api/v1/attachments/' + attachment['id']
c.call('GET', path, 404, actor=eve)
c.call('GET', path + '/content', 404, actor=eve)
print('Outsider metadata/content: 404', flush=True)
try:
subprocess.run([str(ROOT / 'scripts/stop-one.sh')], cwd=ROOT,
env=dict(os.environ, TARGET_INSTANCE_ID=target), check=True, capture_output=True)
deadline = time.monotonic() + 15
while True:
try:
download = c.call('GET', path + '/content', actor=bob)
check(download['headers']['X-Instance-Id'] != target, 'stopped instance returned content')
check(download['bytes'] == content, 'download differs from original')
print('Download: 200; instance=' + download['headers']['X-Instance-Id'], flush=True)
print('SHA-256: ' + hashlib.sha256(content).hexdigest(), flush=True)
break
except Failure:
if time.monotonic() >= deadline:
raise
time.sleep(0.2)
finally:
subprocess.run(['docker', 'compose', 'start', 'api'], cwd=ROOT, check=True, capture_output=True)
print('PASS: uploading replica stopped; original served by another replica; stopped replica restored')
if __name__ == '__main__':
main()
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
"""Abort a binary request mid-body and verify temporary files and metadata are absent."""
import os
from pathlib import Path
import socket
import subprocess
import sys
import time
import urllib.parse
import uuid
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / 'tests'))
from client import Client, check
c = Client()
alice, bob, eve, chat = c.bootstrap()
name = 'interrupted-' + uuid.uuid4().hex + '.png'
base = urllib.parse.urlsplit(c.base)
body = (ROOT / 'tests/fixtures/sample.png').read_bytes()
headers = ('POST /api/v1/chats/' + chat['id'] + '/attachments?filename=' + name + ' HTTP/1.1\r\n'
'Host: localhost\r\nAuthorization: Bearer ' + alice['token'] + '\r\n'
'Content-Type: image/png\r\nContent-Length: 1048576\r\n\r\n')
with socket.create_connection((base.hostname, base.port or 80)) as connection:
connection.sendall(headers.encode() + body[:len(body)//2])
time.sleep(0.5)
# Let the server observe EOF, or the bounded receive timeout.
ids = subprocess.check_output(['docker', 'compose', 'ps', '-q', 'api'], cwd=ROOT, text=True).split()
deadline = time.monotonic() + 20
while True:
files = [subprocess.check_output(['docker', 'exec', cid, 'find', '/tmp', '-name', 'messenger-upload-*'], text=True).strip() for cid in ids]
if not any(files):
break
check(time.monotonic() < deadline, 'temporary upload file survived interruption/timeout')
time.sleep(0.2)
count = subprocess.check_output(['docker', 'compose', 'exec', '-T', 'postgres', 'psql', '-U', 'messenger', '-d', 'messenger', '-Atc',
"SELECT count(*) FROM attachments WHERE filename = '" + name + "'"], cwd=ROOT, text=True).strip()
check(count == '0', 'interrupted upload published metadata')
print('PASS: connection closed mid-body; temporary files removed on both replicas; metadata absent')
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""Ten simultaneous binary uploads and sampled API cgroup memory, without tokens in output."""
import concurrent.futures
import hashlib
import json
import os
from pathlib import Path
import random
import struct
import subprocess
import sys
import threading
import time
import zlib
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / 'tests'))
from client import Client, check
def chunk(kind, data):
return struct.pack('!I', len(data)) + kind + data + struct.pack('!I', zlib.crc32(kind + data))
def fixture():
width = height = 1600
rng = random.Random(42)
pixels = b''.join(b'\0' + rng.randbytes(width * 3) for _ in range(height))
return (b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('!IIBBBBB', width, height, 8, 2, 0, 0, 0))
+ chunk(b'IDAT', zlib.compress(pixels, 1)) + chunk(b'IEND', b''))
def memory(ids):
result = {}
for container in ids:
value = subprocess.check_output(['docker', 'exec', container, 'cat', '/sys/fs/cgroup/memory.current'], text=True)
result[container[:12]] = int(value)
return result
def main():
c = Client(timeout=60)
alice, bob, eve, chat = c.bootstrap()
data = fixture()
expected = hashlib.sha256(data).hexdigest()
ids = subprocess.check_output(['docker', 'compose', 'ps', '-q', 'api'], cwd=ROOT, text=True).split()
baseline = memory(ids)
samples = []
stop = threading.Event()
start = time.monotonic()
def sample():
while not stop.is_set():
samples.append({'seconds': round(time.monotonic() - start, 3), 'bytes': memory(ids)})
stop.wait(0.2)
barrier = threading.Barrier(10)
def upload(index):
client = Client(timeout=60)
barrier.wait(timeout=30)
result = client.call('POST', '/api/v1/chats/' + chat['id'] + '/attachments?filename=load.png',
201, actor=alice, raw=data, headers={'Content-Type': 'image/png'}, schema='Attachment')
check(result['json']['sha256'] == expected, 'uploaded checksum mismatch')
downloaded = client.call('GET', '/api/v1/attachments/' + result['json']['id'] + '/content', actor=bob)
check(hashlib.sha256(downloaded['bytes']).hexdigest() == expected, 'download checksum mismatch')
return {'upload': index, 'seconds': round(result['elapsed'], 3), 'instance': result['headers']['X-Instance-Id']}
sampler = threading.Thread(target=sample)
sampler.start()
try:
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as pool:
results = list(pool.map(upload, range(10)))
finally:
stop.set()
sampler.join()
peak = {key: max([baseline[key]] + [s['bytes'][key] for s in samples]) for key in baseline}
print(json.dumps({'concurrency': 10, 'file_bytes': len(data), 'pixels': 1600 * 1600,
'sha256': expected, 'baseline_bytes': baseline, 'sampled_peak_bytes': peak,
'results': results, 'samples': samples}, indent=2))
if __name__ == '__main__':
main()
+21
View File
@@ -0,0 +1,21 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
# Bootstrap is deliberately repeated; existing objects must survive both runs.
docker compose run --rm s3-init
docker compose run --rm s3-init
docker compose run --rm --entrypoint /bin/sh s3-init -ec '
mc alias set app http://s3:9000 "$(cat /run/secrets/s3_api_access_key_id)" "$(cat /run/secrets/s3_api_secret_access_key)" >/dev/null
if mc admin info app >/dev/null 2>&1; then exit 1; fi
if mc mb app/forbidden-bucket >/dev/null 2>&1; then exit 1; fi
if mc anonymous set download "app/$S3_BUCKET" >/dev/null 2>&1; then exit 1; fi
printf test > /tmp/test-object
if mc cp /tmp/test-object "app/$S3_BUCKET/forbidden-prefix/object" >/dev/null 2>&1; then exit 1; fi
mc stat "app/$S3_BUCKET" >/dev/null
printf "PASS: app cannot administer MinIO, create buckets, change policy or write outside originals/\n"
'
if docker compose exec -T lb wget -q -T 2 -O /dev/null http://s3:9001; then
echo 'FAIL: console reachable from shared network'
exit 1
fi
echo 'PASS: console unavailable from shared network'
+10
View File
@@ -0,0 +1,10 @@
#!/bin/sh
# Runs EXPLAIN (ANALYZE, BUFFERS) of the chat history query against the Compose PostgreSQL on 70 000 synthetic
# messages. The data is inserted and rolled back in one transaction; the plans are saved to evidence/.
set -eu
cd "$(dirname "$0")/.."
mkdir -p evidence
docker compose exec -T postgres psql -U messenger -d messenger -X -q < scripts/sql/explain-history.sql \
| tee evidence/explain-history.txt
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""Sends one message with the SAME Idempotency-Key many times in parallel through the balancer.
Expects every response to be 201 with the same message, at least two replicas answering (X-Instance-Id)
and exactly one message in the chat history. Synthetic users only; the token stays in memory.
"""
import argparse
import base64
import json
import threading
import urllib.error
import urllib.request
import uuid
from collections import Counter
parser = argparse.ArgumentParser()
parser.add_argument('--base-url', default='http://localhost:8080')
parser.add_argument('--parallel', type=int, default=32)
parser.add_argument('--rounds', type=int, default=20)
args = parser.parse_args()
def call(method, path, body=None, headers=None):
data = json.dumps(body).encode() if body is not None else None
request = urllib.request.Request(args.base_url + path, data=data, method=method, headers={
'Content-Type': 'application/json', **(headers or {})})
try:
with urllib.request.urlopen(request, timeout=10) as response:
return response.status, response.headers, json.loads(response.read() or b'null')
except urllib.error.HTTPError as error:
return error.code, error.headers, json.loads(error.read() or b'null')
username = 'race_' + uuid.uuid4().hex[:12]
password = 'race-password-' + uuid.uuid4().hex
status, _, user = call('POST', '/api/v1/users', {'username': username, 'display_name': 'race', 'password': password})
assert status == 201, status
basic = base64.b64encode(f'{username}:{password}'.encode()).decode()
status, _, session = call('POST', '/api/v1/auth/sessions', headers={'Authorization': 'Basic ' + basic})
assert status == 201, status
auth = {'Authorization': 'Bearer ' + session['token']}
status, _, chat = call('POST', '/api/v1/chats', {'title': 'race', 'member_ids': [user['id']]}, auth)
assert status == 201, status
path = f"/api/v1/chats/{chat['id']}/messages"
statuses, instances, violations = Counter(), Counter(), 0
for round_number in range(args.rounds):
key = uuid.uuid4().hex
results = []
barrier = threading.Barrier(args.parallel)
def send():
barrier.wait()
results.append(call('POST', path, {'text': f'race {round_number}'}, {**auth, 'Idempotency-Key': key}))
threads = [threading.Thread(target=send) for _ in range(args.parallel)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
ids = {body['id'] for status, _, body in results if status == 201}
for status, headers, _ in results:
statuses[status] += 1
instances[headers.get('X-Instance-Id', '-')] += 1
if len(ids) != 1:
violations += 1
status, _, page = call('GET', path + '?limit=100', headers=auth)
texts = [m['text'] for m in page['items']]
print(json.dumps({
'rounds': args.rounds,
'parallel_requests_per_round': args.parallel,
'statuses': dict(statuses),
'requests_per_instance': dict(instances),
'rounds_with_more_than_one_message_id': violations,
'messages_in_history': len(texts),
'distinct_texts_in_history': len(set(texts)),
}, indent=2))
ok = violations == 0 and len(texts) == args.rounds == len(set(texts)) and len(instances) >= 2 and set(statuses) == {201}
print('PASS' if ok else 'FAIL')
raise SystemExit(0 if ok else 1)
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
# Creates local secrets for the Compose stack. Never overwrites an existing secret:
# the PostgreSQL volume is initialised with the first password and keeps it.
set -eu
cd "$(dirname "$0")/.."
mkdir -p secrets
# Readable by the non-root users inside the containers (bind-mounted as is).
umask 022
for name in db_password redis_password s3_access_key_id s3_secret_access_key s3_api_access_key_id s3_api_secret_access_key; do
if [ ! -s "secrets/$name" ]; then
openssl rand -hex 24 > "secrets/$name"
echo "created secrets/$name"
else
echo "secrets/$name already exists, keeping it"
fi
done
+55
View File
@@ -0,0 +1,55 @@
#!/bin/sh
# One replica fault under constant load through the balancer, then its restoration.
# MODE=stop docker stop (SIGTERM: drain, graceful shutdown) MODE=kill docker kill (SIGKILL: crash)
# TARGET=api-1|api-2 RATE=150 req/s DURATION=45 s fault at 10 s, restore at 25 s
# Output: evidence/load/failover-<mode>-<target>/{k6.txt,events.json,haproxy.log,per-second.csv,summary.json}
set -eu
cd "$(dirname "$0")/../.."
MODE=${MODE:-stop}
TARGET=${TARGET:-api-1}
RATE=${RATE:-150}
DURATION=${DURATION:-45}
FAULT_AT=${FAULT_AT:-10}
RESTORE_AT=${RESTORE_AT:-25}
out=evidence/load/failover-$MODE-$TARGET
rm -rf "$out" && mkdir -p "$out"
now() { python3 -c 'import time; print(int(time.time() * 1000))'; }
# Sleeps until <seconds> after the load started, so the schedule does not drift with slow docker commands.
sleep_until() { python3 -c "import time; time.sleep(max(0, $start / 1000 + $1 - time.time()))"; }
container=mtusi-messenger-lab4-$TARGET
target_instance=$(docker inspect -f '{{.Config.Hostname}}' "$container")
since=$(date -u +%Y-%m-%dT%H:%M:%SZ)
K6_CSV_TIME_FORMAT=unix_milli k6 run --quiet --out csv="$out/requests.csv" \
-e RATE="$RATE" -e DURATION="${DURATION}s" scripts/load/mixed.js > "$out/k6.txt" 2>&1 &
k6_pid=$!
# Load starts after k6 setup (users, sessions, chat): wait for the first request of the scenario.
until grep -q ',load,' "$out/requests.csv" 2>/dev/null; do sleep 0.2; done
start=$(now)
sleep_until "$FAULT_AT"
fault=$(now)
if [ "$MODE" = kill ]; then docker kill "$container" >/dev/null; else docker stop --time 9 "$container" >/dev/null; fi
fault_done=$(now)
sleep_until "$RESTORE_AT"
restore_begin=$(now)
docker start "$container" >/dev/null
until [ "$(docker inspect -f '{{.State.Health.Status}}' "$container")" = healthy ]; do sleep 0.2; done
restore_healthy=$(now)
restored_instance=$(docker inspect -f '{{.Config.Hostname}}' "$container")
wait "$k6_pid" || true
docker compose logs --timestamps --no-log-prefix --since "$since" lb 2>/dev/null | grep -E 'Server api/|SH--|5[0-9][0-9] ' > "$out/haproxy.log" || true
cat > "$out/events.json" <<JSON
{"mode": "$MODE", "target": "$TARGET", "target_instance": "$target_instance", "restored_instance": "$restored_instance",
"load_start": $start, "fault": $fault, "fault_done": $fault_done, "restore_begin": $restore_begin, "restore_healthy": $restore_healthy}
JSON
python3 scripts/load/summarize.py "$out" > /dev/null
# Raw per-request data stays out of Git (evidence keeps aggregates only); KEEP_RAW=1 keeps it for analysis.
[ "${KEEP_RAW:-0}" = 1 ] || rm -f "$out/requests.csv"
cat "$out/summary.json"
+67
View File
@@ -0,0 +1,67 @@
// Open-model load through the balancer: RATE requests per second regardless of response time.
// 70 % GET /users/me (session lookup), 30 % POST message with a unique Idempotency-Key.
// Every response adds a `served_by` sample tagged with X-Instance-Id and status, so the summary can show
// which replica answered each second. Tokens live only in this process.
import http from 'k6/http';
import encoding from 'k6/encoding';
import { Counter } from 'k6/metrics';
const BASE = __ENV.BASE_URL || 'http://localhost:8080';
const RATE = Number(__ENV.RATE || 150);
const DURATION = __ENV.DURATION || '40s';
const USERS = 10;
const PASSWORD = 'load-test-password';
const servedBy = new Counter('served_by');
export const options = {
scenarios: {
load: {
executor: 'constant-arrival-rate',
rate: RATE,
timeUnit: '1s',
duration: DURATION,
preAllocatedVUs: 50,
maxVUs: 300,
},
},
summaryTrendStats: ['avg', 'med', 'p(95)', 'p(99)', 'max'],
};
export function setup() {
const json = { headers: { 'Content-Type': 'application/json' } };
const users = [];
for (let i = 0; i < USERS; i++) {
const username = `load_${Date.now().toString(36)}_${i}`;
const user = http.post(`${BASE}/api/v1/users`, JSON.stringify({ username, display_name: 'load', password: PASSWORD }), json).json();
const basic = encoding.b64encode(`${username}:${PASSWORD}`);
const session = http.post(`${BASE}/api/v1/auth/sessions`, null, { headers: { Authorization: `Basic ${basic}` } }).json();
users.push({ id: user.id, token: session.token });
}
const chat = http.post(`${BASE}/api/v1/chats`, JSON.stringify({ title: 'load', member_ids: users.map((u) => u.id) }), {
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${users[0].token}` },
}).json();
return { users, chatId: chat.id };
}
export default function (data) {
const user = data.users[Math.floor(Math.random() * data.users.length)];
const auth = { Authorization: `Bearer ${user.token}` };
let response;
if (Math.random() < 0.7) {
response = http.get(`${BASE}/api/v1/users/me`, { headers: auth, tags: { name: 'me' }, timeout: '5s' });
} else {
response = http.post(
`${BASE}/api/v1/chats/${data.chatId}/messages`,
JSON.stringify({ text: `load ${__VU}-${__ITER}` }),
{
headers: { ...auth, 'Content-Type': 'application/json', 'Idempotency-Key': `load-${__VU}-${__ITER}-${Date.now()}` },
tags: { name: 'send' },
timeout: '5s',
},
);
}
servedBy.add(1, { instance: response.headers['X-Instance-Id'] || 'none', code: String(response.status) });
}
+34
View File
@@ -0,0 +1,34 @@
#!/bin/sh
# scripts/rolling-update.sh while k6 keeps RATE req/s through the balancer.
# Output: evidence/load/rolling-update/{k6.txt,rolling.log,events.json,per-second.csv,summary.json}
set -eu
cd "$(dirname "$0")/../.."
RATE=${RATE:-150}
DURATION=${DURATION:-70}
out=evidence/load/rolling-update
rm -rf "$out" && mkdir -p "$out"
now() { python3 -c 'import time; print(int(time.time() * 1000))'; }
# Build outside the measured window: compiling would compete with the API for CPU.
docker compose build --quiet api
{ echo "# before"; docker compose ps api --format '{{.Name}} {{.ID}} {{.Image}}'; } > "$out/rolling.log"
K6_CSV_TIME_FORMAT=unix_milli k6 run --quiet --out csv="$out/requests.csv" \
-e RATE="$RATE" -e DURATION="${DURATION}s" scripts/load/mixed.js > "$out/k6.txt" 2>&1 &
k6_pid=$!
until grep -q ',load,' "$out/requests.csv" 2>/dev/null; do sleep 0.2; done
start=$(now)
sleep 10
begin=$(now)
scripts/rolling-update.sh >> "$out/rolling.log" 2>&1
end=$(now)
wait "$k6_pid" || true
{ echo "# after"; docker compose ps api --format '{{.Name}} {{.ID}} {{.Image}}'; } >> "$out/rolling.log"
echo "{\"load_start\": $start, \"rolling_begin\": $begin, \"rolling_end\": $end}" > "$out/events.json"
python3 scripts/load/summarize.py "$out" > /dev/null
# Raw per-request data stays out of Git (evidence keeps aggregates only); KEEP_RAW=1 keeps it for analysis.
[ "${KEEP_RAW:-0}" = 1 ] || rm -f "$out/requests.csv"
cat "$out/rolling.log" "$out/summary.json"
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env python3
"""Per-second view of one k6 run (requests.csv) plus the events of the experiment (events.json).
Writes per-second.csv and summary.json next to the input. An error is any response that is not 2xx
(including k6 status 0: connection reset or timeout). Times are seconds from the start of the load.
"""
import csv
import json
import math
import sys
from collections import Counter, defaultdict
from pathlib import Path
from urllib.parse import parse_qsl
run = Path(sys.argv[1])
events = json.loads((run / 'events.json').read_text()) if (run / 'events.json').exists() else {}
durations = [] # (t_ms, duration_ms, status)
served = [] # (t_ms, instance, status)
with open(run / 'requests.csv') as f:
for row in csv.DictReader(f):
if row['metric_name'] == 'http_req_duration' and row['scenario'] == 'load':
durations.append((int(row['timestamp']), float(row['metric_value']), int(row['status'] or 0)))
elif row['metric_name'] == 'served_by':
tags = dict(parse_qsl(row['extra_tags']))
served.append((int(row['timestamp']), tags.get('instance', 'none'), int(tags.get('code', 0))))
t0 = min(t for t, _, _ in durations)
rel = lambda ms: (ms - t0) / 1000
def percentile(values, p):
if not values:
return None
values = sorted(values)
return values[min(len(values) - 1, math.ceil(p / 100 * len(values)) - 1)]
per_second = defaultdict(lambda: {'ok': 0, 'errors': 0, 'durations': [], 'instances': Counter()})
for t, d, status in durations:
second = per_second[math.floor(rel(t))]
second['durations'].append(d)
if 200 <= status < 300:
second['ok'] += 1
else:
second['errors'] += 1
for t, instance, status in served:
if 200 <= status < 300:
per_second[math.floor(rel(t))]['instances'][instance] += 1
instances = sorted({i for s in per_second.values() for i in s['instances']})
with open(run / 'per-second.csv', 'w', newline='') as f:
writer = csv.writer(f)
writer.writerow(['second', 'ok', 'errors', 'p99_ms', 'max_ms'] + instances)
for second in sorted(per_second):
s = per_second[second]
writer.writerow([second, s['ok'], s['errors'], round(percentile(s['durations'], 99), 1), round(max(s['durations']), 1)]
+ [s['instances'].get(i, 0) for i in instances])
errors = sorted(rel(t) for t, _, status in durations if not 200 <= status < 300)
seconds = max(per_second) + 1
summary = {
'requests': len(durations),
'ok': len(durations) - len(errors),
'errors': len(errors),
'error_codes': dict(Counter(str(status) for _, _, status in durations if not 200 <= status < 300)),
'ok_rps': round((len(durations) - len(errors)) / seconds, 1),
'p50_ms': round(percentile([d for _, d, _ in durations], 50), 1),
'p95_ms': round(percentile([d for _, d, _ in durations], 95), 1),
'p99_ms': round(percentile([d for _, d, _ in durations], 99), 1),
'max_ms': round(max(d for _, d, _ in durations), 1),
'requests_per_instance': dict(Counter(i for _, i, status in served if 200 <= status < 300)),
}
for name, value in events.items():
if isinstance(value, (int, float)):
summary[f'event_{name}_s'] = round(rel(value), 2)
if 'fault' in events:
fault = rel(events['fault'])
after = [e - fault for e in errors if e >= fault - 0.5]
summary['errors_after_fault'] = len(after)
summary['last_error_after_fault_s'] = round(max(after), 2) if after else None
target = events.get('target_instance')
end = rel(events['restore_begin']) if 'restore_begin' in events else float('inf')
last = [rel(t) - fault for t, i, status in served if i == target and fault <= rel(t) < end]
summary['target_last_response_after_fault_s'] = round(max(last), 2) if last else None
if 'restore_begin' in events and events.get('restored_instance'):
begin = rel(events['restore_begin'])
first = [rel(t) - begin for t, i, status in served
if i == events['restored_instance'] and 200 <= status < 300 and rel(t) >= begin]
summary['restored_replica_first_response_after_start_s'] = round(min(first), 2) if first else None
# HAProxy state changes (docker log timestamps): when the balancer took the target out and put it back.
log = run / 'haproxy.log'
if log.exists() and 'fault' in events:
from datetime import datetime
def at(line):
stamp = line.split()[0].rstrip('Z')
whole, _, fraction = stamp.partition('.')
return datetime.fromisoformat(whole + '+00:00').timestamp() * 1000 + float('0.' + (fraction or '0')) * 1000
changes = [(at(line), line) for line in log.read_text().splitlines() if 'Server api/' in line and '[WARNING]' not in line]
down = [t for t, line in changes if ('is DOWN' in line or 'going DOWN' in line) and t >= events['fault']]
summary['lb_marked_down_after_fault_s'] = round((min(down) - events['fault']) / 1000, 2) if down else None
if 'restore_begin' in events:
up = [t for t, line in changes if 'is UP, reason' in line and t >= events['restore_begin']]
summary['lb_marked_up_after_start_s'] = round((min(up) - events['restore_begin']) / 1000, 2) if up else None
(run / 'summary.json').write_text(json.dumps(summary, indent=2, ensure_ascii=False))
print(json.dumps(summary, indent=2, ensure_ascii=False))
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
# Restarts the balancer, every API replica, PostgreSQL, Redis and S3, keeping the named volume (no `down -v`, no re-seed).
set -eu
cd "$(dirname "$0")/.."
docker compose stop lb api redis postgres s3
docker compose up -d --wait
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# Reproduces the review scenario on the running stack, everything through the balancer:
# session + message, warm the session on both replicas, CLIENT PAUSE <ms> WRITE on Redis (reads keep working),
# logout, then read and repeat the original message with the old token and key.
# Expected: logout is 204 only once the tombstone is in Redis; afterwards every replica answers 401.
# PAUSE_MS=5000 (default): the logout waits for the pause and answers 204.
# PAUSE_MS=12000: longer than the logout deadline (8 s): the logout answers 503 and every replica still rejects the token.
set -euo pipefail
cd "$(dirname "$0")/.."
PAUSE_MS=${PAUSE_MS:-5000}
B=${BASE_URL:-http://localhost:8080}/api/v1
U=pause_$RANDOM$RANDOM
PW=pause-test-password
KEY=pause-$RANDOM$RANDOM
redis() { docker compose exec -T redis redis-cli --no-auth-warning -a "$(cat secrets/redis_password)" "$@"; }
now() { python3 -c 'import time; print(time.time())'; }
status() { curl -s -o /dev/null -w '%{http_code} %header{x-instance-id}' "$@"; }
curl -s -o /dev/null -X POST "$B/users" -H 'Content-Type: application/json' \
-d "{\"username\":\"$U\",\"display_name\":\"pause\",\"password\":\"$PW\"}"
T=$(curl -s -X POST "$B/auth/sessions" -u "$U:$PW" | python3 -c 'import sys,json;print(json.load(sys.stdin)["token"])')
ME=$(curl -s "$B/users/me" -H "Authorization: Bearer $T" | python3 -c 'import sys,json;print(json.load(sys.stdin)["id"])')
CHAT=$(curl -s -X POST "$B/chats" -H "Authorization: Bearer $T" -H 'Content-Type: application/json' \
-d "{\"title\":\"pause\",\"member_ids\":[\"$ME\"]}" | python3 -c 'import sys,json;print(json.load(sys.stdin)["id"])')
send() { status -X POST "$B/chats/$CHAT/messages" -H "Authorization: Bearer $T" -H 'Content-Type: application/json' \
-H "Idempotency-Key: $KEY" -d '{"text":"original message"}'; }
echo "send message with Idempotency-Key: $(send)"
echo "warm the session on both replicas:"
for i in 1 2 3 4; do echo " GET /users/me -> $(status "$B/users/me" -H "Authorization: Bearer $T")"; done
echo "redis: CLIENT PAUSE $PAUSE_MS WRITE -> $(redis CLIENT PAUSE "$PAUSE_MS" WRITE)"
start=$(now)
echo "DELETE /auth/sessions/current -> $(status -X DELETE "$B/auth/sessions/current" -H "Authorization: Bearer $T") after $(python3 -c "print(round($(now) - $start, 2))") s"
sleep 1.5 # replicas check pending revocations once a second; only matters for the unconfirmed (503) case
echo "old token after logout (redis writes still paused: $(python3 -c "print('yes' if $(now) - $start < $PAUSE_MS / 1000 else 'no')")):"
for i in 1 2 3 4; do echo " GET /users/me -> $(status "$B/users/me" -H "Authorization: Bearer $T")"; done
for i in 1 2; do echo " POST message, same key -> $(send)"; done
python3 -c "import time; time.sleep(max(0, $start + $PAUSE_MS / 1000 + 2 - time.time()))"
echo "after the pause: session key in redis = $(redis GET "session:$(printf %s "$T" | shasum -a 256 | cut -d' ' -f1)")"
for i in 1 2; do echo " GET /users/me -> $(status "$B/users/me" -H "Authorization: Bearer $T")"; done
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Replaces the API replicas one by one with a freshly built image while the balancer keeps serving.
# 1. build the new image, 2. apply migrations once (they must be compatible with the running version),
# 3. for every old replica: start one new replica, wait until HAProxy has it UP, then stop (drain) and remove the old.
# The number of serving replicas never drops below the configured count.
set -euo pipefail
cd "$(dirname "$0")/.."
build_id=${BUILD_ID:-$(git rev-parse --short HEAD)-$(date +%s)}
log() { echo "$(date -u +%H:%M:%S) $*"; }
up_in_lb() {
docker compose exec -T lb wget -qO- 'http://127.0.0.1:8404/stats;csv' | awk -F, '$1 == "api" && $2 != "BACKEND" && $18 == "UP"' | wc -l
}
log "build image (BUILD_ID=$build_id)"
docker compose build --quiet --build-arg BUILD_ID="$build_id" api
log "migrate"
docker compose run --rm --no-deps migrate > /dev/null
old=$(docker compose ps -q api)
count=$(echo "$old" | wc -l | tr -d ' ')
for container in $old; do
before=$(docker compose ps -q api | sort)
docker compose up -d --no-deps --no-recreate --scale api=$((count + 1)) api 2> /dev/null
new=$(comm -13 <(echo "$before") <(docker compose ps -q api | sort))
log "started $(docker inspect -f '{{.Name}}' "$new") ($(echo "$new" | cut -c1-12))"
until [ "$(docker inspect -f '{{.State.Health.Status}}' "$new")" = healthy ]; do sleep 0.5; done
until [ "$(up_in_lb)" -ge $((count + 1)) ]; do sleep 0.5; done
log "new replica is UP in HAProxy"
docker stop --time 9 "$container" > /dev/null
docker rm "$container" > /dev/null
log "stopped and removed $(echo "$container" | cut -c1-12)"
done
log "done: $(docker compose ps -q api | while read -r c; do docker inspect -f '{{.Name}} {{index .Config.Labels "org.opencontainers.image.revision"}}' "$c"; done | tr '\n' ' ')"
+100
View File
@@ -0,0 +1,100 @@
-- EXPLAIN (ANALYZE, BUFFERS) of the chat history query on synthetic data.
-- Everything runs in one transaction that is rolled back: the database is left untouched.
-- The queries are the ones ExposedMessageRepository.findPageByChatId generates (page size 50 + 1 look-ahead row).
\set ON_ERROR_STOP on
\pset pager off
BEGIN;
INSERT INTO users (id, username, display_name, password_hash, created_at)
VALUES ('00000000-0000-0000-0000-000000000001', 'explain_alice', 'Alice', 'not-a-hash', now()),
('00000000-0000-0000-0000-000000000002', 'explain_bob', 'Bob', 'not-a-hash', now());
-- One large chat (20 000 messages) and 50 neighbours with 1 000 messages each: 70 000 rows in total,
-- so the plan has to skip foreign chats instead of reading a table that only holds the target chat.
INSERT INTO chats (id, title, created_at)
SELECT ('00000000-0000-0000-0001-' || lpad(n::text, 12, '0'))::uuid, 'explain chat ' || n, now()
FROM generate_series(0, 50) AS n;
INSERT INTO chat_members (chat_id, user_id, position)
SELECT c.id, u.id, u.position
FROM chats c
CROSS JOIN (VALUES ('00000000-0000-0000-0000-000000000001'::uuid, 0),
('00000000-0000-0000-0000-000000000002'::uuid, 1)) AS u (id, position)
WHERE c.title LIKE 'explain chat %';
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
SELECT gen_random_uuid(),
'00000000-0000-0000-0001-000000000000',
CASE WHEN n % 2 = 0 THEN '00000000-0000-0000-0000-000000000001'::uuid
ELSE '00000000-0000-0000-0000-000000000002'::uuid END,
'message #' || n,
timestamptz '2026-01-01 00:00:00+00' + n * interval '1 second'
FROM generate_series(1, 20000) AS n;
INSERT INTO messages (id, chat_id, sender_id, text, created_at)
SELECT gen_random_uuid(),
('00000000-0000-0000-0001-' || lpad(c::text, 12, '0'))::uuid,
'00000000-0000-0000-0000-000000000001',
'neighbour message #' || n,
timestamptz '2026-01-01 00:00:00+00' + n * interval '1 second'
FROM generate_series(1, 50) AS c, generate_series(1, 1000) AS n;
ANALYZE users, chats, chat_members, messages;
\echo
\echo '== Data volume'
SELECT count(*) AS total_messages,
count(*) FILTER (WHERE chat_id = '00000000-0000-0000-0001-000000000000') AS target_chat_messages
FROM messages;
-- Cursor of the 15 000th message: a deep page, where OFFSET pagination would have to skip 15 000 rows.
SELECT created_at AS cursor_created_at, id AS cursor_id
FROM messages
WHERE chat_id = '00000000-0000-0000-0001-000000000000'
ORDER BY created_at, id
OFFSET 14999 LIMIT 1 \gset
\echo
\echo '== 1. First page: WHERE chat_id = ? ORDER BY created_at, id LIMIT 51'
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
SELECT messages.id, messages.chat_id, messages.sender_id, messages.text, messages.created_at
FROM messages
WHERE messages.chat_id = '00000000-0000-0000-0001-000000000000'
ORDER BY messages.created_at ASC, messages.id ASC
LIMIT 51;
\echo
\echo '== 2. Deep page after the 15 000th message: (created_at, id) > (?, ?)'
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
SELECT messages.id, messages.chat_id, messages.sender_id, messages.text, messages.created_at
FROM messages
WHERE (messages.chat_id = '00000000-0000-0000-0001-000000000000')
AND ((messages.created_at, messages.id) > (:'cursor_created_at', :'cursor_id'))
ORDER BY messages.created_at ASC, messages.id ASC
LIMIT 51;
\echo
\echo '== 3. Same deep page with OFFSET instead of a cursor (for comparison, not used by the API)'
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
SELECT messages.id, messages.chat_id, messages.sender_id, messages.text, messages.created_at
FROM messages
WHERE messages.chat_id = '00000000-0000-0000-0001-000000000000'
ORDER BY messages.created_at ASC, messages.id ASC
LIMIT 51 OFFSET 15000;
\echo
\echo '== 4. Deep page without messages_chat_id_created_at_id_idx (index scans disabled for this query only)'
SET LOCAL enable_indexscan = off;
SET LOCAL enable_bitmapscan = off;
SET LOCAL enable_indexonlyscan = off;
EXPLAIN (ANALYZE, BUFFERS, COSTS OFF)
SELECT messages.id, messages.chat_id, messages.sender_id, messages.text, messages.created_at
FROM messages
WHERE (messages.chat_id = '00000000-0000-0000-0001-000000000000')
AND ((messages.created_at, messages.id) > (:'cursor_created_at', :'cursor_id'))
ORDER BY messages.created_at ASC, messages.id ASC
LIMIT 51;
ROLLBACK;
+22
View File
@@ -0,0 +1,22 @@
#!/bin/sh
# Stops the ONE API replica named by TARGET_INSTANCE_ID and leaves it stopped (no DB/LB/volume changes).
# The replica's X-Instance-Id is its container hostname, i.e. the short container id.
# Restore afterwards with: docker compose start api
set -eu
cd "$(dirname "$0")/.."
: "${TARGET_INSTANCE_ID:?test provides target API instance alias}"
case "$TARGET_INSTANCE_ID" in
*[!0-9a-f]* | "") echo "unexpected instance id" >&2; exit 2 ;;
esac
# Only containers of the api service are candidates, so a wrong id can never stop the database or the balancer.
container=$(docker compose ps -q api | grep "^$TARGET_INSTANCE_ID" || true)
if [ -z "$container" ] || [ "$(printf '%s\n' "$container" | wc -l)" -ne 1 ]; then
echo "no single api replica matches $TARGET_INSTANCE_ID" >&2
exit 1
fi
docker stop --time 9 "$container" >/dev/null
+21
View File
@@ -0,0 +1,21 @@
dependencyResolutionManagement {
repositories {
mavenCentral()
}
versionCatalogs {
create("awssdk") {
from("aws.sdk.kotlin:version-catalog:1.6.41")
}
create("ktorLibs") {
from("io.ktor:ktor-version-catalog:3.3.0")
}
}
}
plugins {
id("org.gradle.toolchains.foojay-resolver-convention") version "1.0.0"
}
rootProject.name = "mtusi-messenger-lab4"
@@ -0,0 +1,50 @@
package com.mtusi.messenger.main
import com.mtusi.messenger.main.plugins.authentication.configureAuthentication
import com.mtusi.messenger.main.plugins.configureAttachmentCleanup
import com.mtusi.messenger.main.plugins.configureCallLogging
import com.mtusi.messenger.main.plugins.configureContentNegotiation
import com.mtusi.messenger.main.plugins.configureCors
import com.mtusi.messenger.main.plugins.configureDatabase
import com.mtusi.messenger.main.plugins.configureDefaultHeaders
import com.mtusi.messenger.main.plugins.configureGracefulShutdown
import com.mtusi.messenger.main.plugins.configureInstanceId
import com.mtusi.messenger.main.plugins.configureKoin
import com.mtusi.messenger.main.plugins.configureRedis
import com.mtusi.messenger.main.plugins.configureRequestId
import com.mtusi.messenger.main.plugins.configureSessionRevocationSync
import com.mtusi.messenger.main.plugins.configureRouting
import com.mtusi.messenger.main.plugins.configureStatusPages
import com.mtusi.messenger.main.plugins.migrateDatabase
import io.ktor.server.application.Application
import io.ktor.server.netty.EngineMain
fun main(args: Array<String>) {
// `migrate` applies database migrations and exits; the API replicas start with migrations already in place.
if (args.firstOrNull() == "migrate") {
return migrateDatabase()
}
EngineMain.main(args)
}
fun Application.module() {
configureKoin()
configureDatabase()
configureRedis()
configureGracefulShutdown()
configureSessionRevocationSync()
configureAttachmentCleanup()
configureContentNegotiation()
configureDefaultHeaders()
configureInstanceId()
configureCors()
configureRequestId()
configureCallLogging()
configureStatusPages()
configureAuthentication()
configureRouting()
}
@@ -0,0 +1,31 @@
package com.mtusi.messenger.main.compat.serializers
import io.ktor.http.HttpMethod
import kotlinx.serialization.KSerializer
import kotlinx.serialization.Serializable
import kotlinx.serialization.descriptors.PrimitiveKind
import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
typealias SerialHttpMethod = @Serializable(HttpMethodSerializer::class) HttpMethod
object HttpMethodSerializer : KSerializer<HttpMethod> {
override val descriptor = PrimitiveSerialDescriptor(
serialName = "HttpMethod",
kind = PrimitiveKind.STRING
)
override fun serialize(
encoder: Encoder,
value: HttpMethod
) {
encoder.encodeString(value.value)
}
override fun deserialize(decoder: Decoder): HttpMethod {
return HttpMethod.parse(decoder.decodeString())
}
}
@@ -0,0 +1,31 @@
package com.mtusi.messenger.main.compat.serializers
import kotlinx.serialization.KSerializer
import kotlinx.serialization.Serializable
import kotlinx.serialization.descriptors.PrimitiveKind
import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import java.util.UUID
typealias SerialUUID = @Serializable(UUIDSerializer::class) UUID
object UUIDSerializer : KSerializer<UUID> {
override val descriptor = PrimitiveSerialDescriptor(
serialName = "UUID",
kind = PrimitiveKind.STRING
)
override fun deserialize(decoder: Decoder): UUID {
return decoder.decodeString().run(UUID::fromString)
}
override fun serialize(
encoder: Encoder,
value: UUID
) {
encoder.encodeString(value.toString())
}
}
@@ -0,0 +1,8 @@
package com.mtusi.messenger.main.config
import org.koin.core.annotation.ComponentScan
import org.koin.core.annotation.Module
@Module(createdAtStart = true)
@ComponentScan
class ConfigModule
@@ -0,0 +1,15 @@
package com.mtusi.messenger.main.config.authentication
interface AuthenticationConfig {
val realm: String
val sessionTtlSeconds: Long
val sessionCacheTtlSeconds: Long
val sessionCacheSingleFlight: Boolean
/**
* How long a logout keeps trying to write its tombstone to Redis before it answers 503 instead of 204.
*/
val sessionRevokeTimeoutMs: Long
}
@@ -0,0 +1,11 @@
package com.mtusi.messenger.main.config.authentication
import io.ktor.server.application.Application
import io.ktor.server.config.getAs
import org.koin.core.annotation.Provided
import org.koin.core.annotation.Single
@Single
class DefaultAuthenticationConfig(
@Provided app: Application
) : AuthenticationConfig by app.environment.config.property("authentication").getAs<SerialAuthenticationConfig>()
@@ -0,0 +1,12 @@
package com.mtusi.messenger.main.config.authentication
import kotlinx.serialization.Serializable
@Serializable
data class SerialAuthenticationConfig(
override val realm: String,
override val sessionTtlSeconds: Long,
override val sessionCacheTtlSeconds: Long,
override val sessionCacheSingleFlight: Boolean = true,
override val sessionRevokeTimeoutMs: Long = 8000
) : AuthenticationConfig
@@ -0,0 +1,100 @@
package com.mtusi.messenger.main.config.cors
import com.mtusi.messenger.main.compat.serializers.SerialHttpMethod
import io.ktor.http.HttpMethod
import kotlinx.serialization.Serializable
import io.ktor.server.plugins.cors.CORSConfig as CorsPluginConfig
interface CorsConfig {
/**
* Whether any host should be allowed
*
* Must default to false
*/
val allowAnyHost: Boolean
val allowedHosts: List<AllowedHost>
/**
* Whether any [HttpMethod][io.ktor.http.HttpMethod] should be allowed
*
* Must default to true
*/
val allowAnyMethod: Boolean
val allowedMethods: List<SerialHttpMethod>
/**
* Whether any header should be allowed
*
* Must default to true
*/
val allowAnyHeader: Boolean
val allowedHeaders: List<String>
/**
* Response headers a browser page may read, e.g. X-Request-Id and X-Instance-Id in Swagger UI
*/
val exposedHeaders: List<String>
/**
* Whether requests from the API's own origin should be allowed
*
* Must default to false
*/
val allowSameOrigin: Boolean
/**
* Whether credentials should be allowed
*
* Must default to true
*/
val allowCredentials: Boolean
/**
* Whether non-simple content types should be allowed
*
* Must default to true
*
* @see [io.ktor.server.plugins.cors.CORSConfig.allowNonSimpleContentTypes]
*/
val allowNonSimpleContentTypes: Boolean
@Serializable
data class AllowedHost(
val host: String,
val schemes: List<String>
)
}
fun CorsPluginConfig.configure(config: CorsConfig) {
if (config.allowAnyHost) {
anyHost()
} else {
config.allowedHosts.forEach { (host, schemes) ->
allowHost(host, schemes)
}
}
if (config.allowAnyMethod) {
HttpMethod.DefaultMethods.forEach(::allowMethod)
} else {
config.allowedMethods.forEach(::allowMethod)
}
if (config.allowAnyHeader) {
allowHeaders { true }
} else {
config.allowedHeaders.forEach(::allowHeader)
}
config.exposedHeaders.forEach(::exposeHeader)
allowSameOrigin = config.allowSameOrigin
allowCredentials = config.allowCredentials
allowNonSimpleContentTypes = config.allowNonSimpleContentTypes
}
@@ -0,0 +1,11 @@
package com.mtusi.messenger.main.config.cors
import io.ktor.server.application.Application
import io.ktor.server.config.getAs
import org.koin.core.annotation.Provided
import org.koin.core.annotation.Single
@Single
class DefaultCorsConfig(
@Provided app: Application
) : CorsConfig by app.environment.config.property("cors").getAs<SerialCorsConfig>()
@@ -0,0 +1,18 @@
package com.mtusi.messenger.main.config.cors
import com.mtusi.messenger.main.compat.serializers.SerialHttpMethod
import kotlinx.serialization.Serializable
@Serializable
data class SerialCorsConfig(
override val allowAnyHost: Boolean = false,
override val allowedHosts: List<CorsConfig.AllowedHost> = emptyList(),
override val allowAnyMethod: Boolean = true,
override val allowedMethods: List<SerialHttpMethod> = emptyList(),
override val allowAnyHeader: Boolean = true,
override val allowedHeaders: List<String> = emptyList(),
override val exposedHeaders: List<String> = emptyList(),
override val allowSameOrigin: Boolean = false,
override val allowCredentials: Boolean = true,
override val allowNonSimpleContentTypes: Boolean = true
) : CorsConfig
@@ -0,0 +1,22 @@
package com.mtusi.messenger.main.config.database
interface DatabaseConfig {
val host: String
val port: Int
val name: String
val username: String
val password: String?
val passwordFile: String?
val maxPoolSize: Int
val connectionTimeoutMs: Long
val statementTimeoutMs: Long
val migrateOnStart: Boolean
val jdbcConnectionUrl: String
get() = "jdbc:postgresql://$host:$port/$name"
}
@@ -0,0 +1,11 @@
package com.mtusi.messenger.main.config.database
import io.ktor.server.application.Application
import io.ktor.server.config.getAs
import org.koin.core.annotation.Provided
import org.koin.core.annotation.Single
@Single
class DefaultDatabaseConfig(
@Provided app: Application
) : DatabaseConfig by app.environment.config.property("database").getAs<SerialDatabaseConfig>()
@@ -0,0 +1,17 @@
package com.mtusi.messenger.main.config.database
import kotlinx.serialization.Serializable
@Serializable
data class SerialDatabaseConfig(
override val host: String,
override val port: Int,
override val name: String,
override val username: String,
override val password: String? = null,
override val passwordFile: String? = null,
override val maxPoolSize: Int,
override val connectionTimeoutMs: Long,
override val statementTimeoutMs: Long,
override val migrateOnStart: Boolean = true
) : DatabaseConfig
@@ -0,0 +1,11 @@
package com.mtusi.messenger.main.config.message
import io.ktor.server.application.Application
import io.ktor.server.config.getAs
import org.koin.core.annotation.Provided
import org.koin.core.annotation.Single
@Single
class DefaultMessageConfig(
@Provided app: Application
) : MessageConfig by app.environment.config.propertyOrNull("messages")?.getAs<SerialMessageConfig>() ?: SerialMessageConfig()
@@ -0,0 +1,5 @@
package com.mtusi.messenger.main.config.message
interface MessageConfig {
val idempotencyTtlSeconds: Long
}
@@ -0,0 +1,8 @@
package com.mtusi.messenger.main.config.message
import kotlinx.serialization.Serializable
@Serializable
data class SerialMessageConfig(
override val idempotencyTtlSeconds: Long = 86400
) : MessageConfig
@@ -0,0 +1,11 @@
package com.mtusi.messenger.main.config.redis
import io.ktor.server.application.Application
import io.ktor.server.config.getAs
import org.koin.core.annotation.Provided
import org.koin.core.annotation.Single
@Single
class DefaultRedisConfig(
@Provided app: Application
) : RedisConfig by app.environment.config.property("redis").getAs<SerialRedisConfig>()
@@ -0,0 +1,15 @@
package com.mtusi.messenger.main.config.redis
interface RedisConfig {
val host: String
val port: Int
val username: String?
val password: String?
val passwordFile: String?
val connectTimeoutMs: Long
val commandTimeoutMs: Long
}
@@ -0,0 +1,14 @@
package com.mtusi.messenger.main.config.redis
import kotlinx.serialization.Serializable
@Serializable
data class SerialRedisConfig(
override val host: String,
override val port: Int,
override val username: String? = null,
override val password: String? = null,
override val passwordFile: String? = null,
override val connectTimeoutMs: Long,
override val commandTimeoutMs: Long
) : RedisConfig
@@ -0,0 +1,12 @@
package com.mtusi.messenger.main.config.storage
import io.ktor.server.application.Application
import io.ktor.server.config.getAs
import org.koin.core.annotation.Provided
import org.koin.core.annotation.Single
@Single
class DefaultStorageConfig(
@Provided app: Application
) : StorageConfig by
(app.environment.config.propertyOrNull("storage")?.getAs<SerialStorageConfig>() ?: SerialStorageConfig())
@@ -0,0 +1,13 @@
package com.mtusi.messenger.main.config.storage
import kotlinx.serialization.Serializable
@Serializable
data class SerialStorageConfig(
override val endpoint: String = "http://localhost:9000",
override val bucket: String = "attachments",
override val accessKey: String? = null,
override val secretKey: String? = null,
override val accessKeyFile: String? = null,
override val secretKeyFile: String? = null,
) : StorageConfig
@@ -0,0 +1,10 @@
package com.mtusi.messenger.main.config.storage
interface StorageConfig {
val endpoint: String
val bucket: String
val accessKey: String?
val secretKey: String?
val accessKeyFile: String?
val secretKeyFile: String?
}
@@ -0,0 +1,37 @@
package com.mtusi.messenger.main.extensions
import com.mtusi.messenger.main.infrastructure.types.AuthChallenge
import com.mtusi.messenger.main.infrastructure.types.ErrorCode
import com.mtusi.messenger.main.plugins.RequestIdKey
import com.mtusi.messenger.main.plugins.authentication.UserPrincipal
import io.ktor.http.HttpHeaders
import io.ktor.server.application.ApplicationCall
import io.ktor.server.auth.principal
import io.ktor.server.response.header
import io.ktor.server.response.respond
import kotlinx.serialization.Serializable
@Serializable
data class ErrorBody(val error: ErrorDetails)
@Serializable
data class ErrorDetails(
val code: String,
val message: String,
val requestId: String,
)
suspend fun ApplicationCall.respondError(
code: ErrorCode,
message: String,
challenge: String = AuthChallenge.BEARER,
) {
val requestId = attributes[RequestIdKey]
if (code == ErrorCode.UNAUTHORIZED) {
response.header(HttpHeaders.WWWAuthenticate, challenge)
}
respond(code.status, ErrorBody(ErrorDetails(code.wire, message, requestId)))
}
val ApplicationCall.currentUser: UserPrincipal
get() = principal<UserPrincipal>()!!
@@ -0,0 +1,31 @@
package com.mtusi.messenger.main.infrastructure.controllers._common.parameters
import com.mtusi.messenger.main.infrastructure.types.ApiException
import com.mtusi.messenger.main.infrastructure.types.ErrorCode
import io.ktor.server.application.ApplicationCall
private const val DEFAULT_LIMIT = 50
private const val MAX_LIMIT = 100
data class PageParameters(
val limit: Int,
val cursor: String?
)
fun ApplicationCall.pageParameters(): PageParameters {
val query = request.queryParameters
val limit = query["limit"]
?.let { it.toIntOrNull() ?: throw ApiException(ErrorCode.INVALID_REQUEST, "invalid limit") }
?: DEFAULT_LIMIT
if (limit !in 1..MAX_LIMIT) {
throw ApiException(ErrorCode.INVALID_REQUEST, "limit out of range")
}
val cursor = query["cursor"]
?.takeIf { it.isNotEmpty() }
return PageParameters(limit, cursor)
}
@@ -0,0 +1,106 @@
package com.mtusi.messenger.main.infrastructure.controllers.attachment
import com.mtusi.messenger.main.extensions.currentUser
import com.mtusi.messenger.main.infrastructure.services.attachment.AttachmentService
import com.mtusi.messenger.main.infrastructure.services.attachment.MAX_ATTACHMENT_BYTES
import com.mtusi.messenger.main.infrastructure.types.ApiException
import com.mtusi.messenger.main.infrastructure.types.ErrorCode
import com.mtusi.messenger.main.plugins.authentication.SESSION_AUTH
import com.mtusi.messenger.main.utils.requireUuid
import io.ktor.http.*
import io.ktor.server.auth.authenticate
import io.ktor.server.request.*
import io.ktor.server.response.*
import io.ktor.server.routing.*
import io.ktor.utils.io.readAvailable
import kotlinx.coroutines.withTimeoutOrNull
import org.koin.ktor.ext.inject
import com.mtusi.messenger.main.utils.withTemporaryUpload
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.coroutines.sync.Semaphore
import kotlinx.coroutines.sync.withPermit
import kotlin.time.Duration.Companion.milliseconds
fun Routing.configureAttachmentRouting() {
val service by inject<AttachmentService>()
val uploads = Semaphore(1)
authenticate(SESSION_AUTH) {
post("/api/v1/chats/{chat_id}/attachments") {
val actor = call.currentUser
val chatId = requireUuid(call.parameters["chat_id"], "chat_id")
service.requireMember(chatId, actor.id)
val filename = call.request.queryParameters["filename"]
?: throw ApiException(ErrorCode.INVALID_REQUEST, "filename is required")
if (filename.isBlank() || filename.codePointCount(0, filename.length) !in 1..255 || filename.any { it == '/' || it == '\\' || it.isISOControl() }) {
throw ApiException(ErrorCode.INVALID_REQUEST, "invalid filename")
}
val type = try {
call.request.contentType().withoutParameters().toString().lowercase()
} catch (_: BadContentTypeFormatException) {
throw ApiException(ErrorCode.INVALID_REQUEST, "invalid Content-Type")
}
if (type !in setOf("image/png", "image/jpeg")) {
throw ApiException(ErrorCode.UNSUPPORTED_MEDIA_TYPE, "expected PNG or JPEG")
}
if ((call.request.contentLength() ?: 0) > MAX_ATTACHMENT_BYTES) {
throw ApiException(ErrorCode.PAYLOAD_TOO_LARGE, "file exceeds 10 MiB")
}
uploads.withPermit {
withTemporaryUpload { file ->
withTimeoutOrNull(15_000.milliseconds) {
withContext(Dispatchers.IO) {
val input = call.receiveChannel()
val buffer = ByteArray(8192)
var size = 0L
file.outputStream().use { output ->
while (true) {
val count = input.readAvailable(buffer)
if (count == -1) break
size += count
if (size > MAX_ATTACHMENT_BYTES) {
throw ApiException(ErrorCode.PAYLOAD_TOO_LARGE, "file exceeds 10 MiB")
}
output.write(buffer, 0, count)
}
}
}
} ?: throw ApiException(ErrorCode.INVALID_REQUEST, "upload timed out")
val attachment = service.upload(chatId, actor.id, filename, type, file)
call.respond(HttpStatusCode.Created, attachment)
}
}
}
get("/api/v1/attachments/{id}") {
call.respond(service.find(requireUuid(call.parameters["id"], "id"), call.currentUser.id))
}
get("/api/v1/attachments/{id}/content") {
val attachment = service.find(requireUuid(call.parameters["id"], "id"), call.currentUser.id)
if ((call.request.queryParameters["variant"] ?: "original") != "original") {
throw ApiException(ErrorCode.NOT_FOUND, "variant not found")
}
call.response.header(HttpHeaders.CacheControl, "private, no-store")
call.response.header("X-Content-Type-Options", "nosniff")
call.respondBytes(service.content(attachment), ContentType.parse(attachment.contentType))
}
}
}
@@ -0,0 +1,53 @@
package com.mtusi.messenger.main.infrastructure.controllers.authentication
import com.mtusi.messenger.main.extensions.currentUser
import com.mtusi.messenger.main.infrastructure.controllers.authentication.responses.SessionResponse
import com.mtusi.messenger.main.infrastructure.services.authentication.AuthenticationService
import com.mtusi.messenger.main.plugins.authentication.CREDENTIALS_AUTH
import com.mtusi.messenger.main.plugins.authentication.CredentialsPrincipal
import com.mtusi.messenger.main.plugins.authentication.SESSION_AUTH
import com.mtusi.messenger.main.plugins.suspendedTransaction
import io.ktor.http.HttpStatusCode
import io.ktor.server.auth.authenticate
import io.ktor.server.auth.principal
import io.ktor.server.response.respond
import io.ktor.server.routing.Routing
import io.ktor.server.routing.delete
import io.ktor.server.routing.post
import io.ktor.server.routing.route
import org.koin.ktor.ext.inject
fun Routing.configureAuthenticationRouting() = route("/api/v1/auth/sessions") {
val authenticationService by inject<AuthenticationService>()
authenticate(CREDENTIALS_AUTH) {
post {
val (user) = call.principal<CredentialsPrincipal>()!!
val issued = suspendedTransaction {
authenticationService.createSession(user.id)
}
call.respond(
HttpStatusCode.Created,
SessionResponse(
token = issued.token,
expiresAt = issued.session.expiresAt,
user = user
)
)
}
}
authenticate(SESSION_AUTH) {
delete("current") {
authenticationService.revokeSession(call.currentUser.session)
call.respond(HttpStatusCode.NoContent)
}
}
}
@@ -0,0 +1,13 @@
package com.mtusi.messenger.main.infrastructure.controllers.authentication.responses
import com.mtusi.messenger.main.infrastructure.dto.User
import kotlinx.serialization.Serializable
import kotlin.time.Instant
@Serializable
data class SessionResponse(
val token: String,
val tokenType: String = "Bearer",
val expiresAt: Instant,
val user: User,
)
@@ -0,0 +1,112 @@
package com.mtusi.messenger.main.infrastructure.controllers.chat
import com.mtusi.messenger.main.extensions.currentUser
import com.mtusi.messenger.main.extensions.respondError
import com.mtusi.messenger.main.infrastructure.controllers._common.parameters.pageParameters
import com.mtusi.messenger.main.infrastructure.controllers.chat.requests.CreateChatRequest
import com.mtusi.messenger.main.infrastructure.models.chat.MAX_CHAT_MEMBERS
import com.mtusi.messenger.main.infrastructure.models.chat.MAX_CHAT_TITLE_LENGTH
import com.mtusi.messenger.main.infrastructure.services.chat.ChatService
import com.mtusi.messenger.main.infrastructure.services.chat.results.ChatCreationResult
import com.mtusi.messenger.main.infrastructure.services.chat.results.ChatPageResult
import com.mtusi.messenger.main.infrastructure.types.ApiException
import com.mtusi.messenger.main.infrastructure.types.ErrorCode
import com.mtusi.messenger.main.plugins.authentication.SESSION_AUTH
import com.mtusi.messenger.main.plugins.suspendedTransaction
import com.mtusi.messenger.main.utils.requireNonBlankBounded
import com.mtusi.messenger.main.utils.requireUuid
import io.ktor.http.HttpStatusCode
import io.ktor.server.auth.authenticate
import io.ktor.server.request.receive
import io.ktor.server.response.respond
import io.ktor.server.routing.Routing
import io.ktor.server.routing.get
import io.ktor.server.routing.post
import io.ktor.server.routing.route
import org.koin.ktor.ext.inject
fun Routing.configureChatRouting() = route("/api/v1/chats") {
val chatService by inject<ChatService>()
authenticate(SESSION_AUTH) {
post {
val actor = call.currentUser
val body = call.receive<CreateChatRequest>()
requireNonBlankBounded(
body.title,
min = 1,
max = MAX_CHAT_TITLE_LENGTH,
field = "title"
)
if (body.memberIds.isEmpty() || body.memberIds.size > MAX_CHAT_MEMBERS) {
throw ApiException(ErrorCode.INVALID_REQUEST, "member_ids size out of range")
}
val memberIds = body.memberIds.map { requireUuid(it, "member_ids") }
val result = suspendedTransaction {
chatService.create(
creatorId = actor.id,
title = body.title,
memberIds = memberIds
)
}
when (result) {
is ChatCreationResult.DuplicateMembers ->
call.respondError(ErrorCode.INVALID_REQUEST, "member_ids must be unique")
is ChatCreationResult.CreatorNotMember ->
call.respondError(ErrorCode.INVALID_REQUEST, "creator must be in member ids")
is ChatCreationResult.UnknownMembers ->
call.respondError(ErrorCode.INVALID_REQUEST, "unknown member")
is ChatCreationResult.Success ->
call.respond(HttpStatusCode.Created, result.chat)
}
}
get {
val actor = call.currentUser
val (limit, cursor) = call.pageParameters()
val result = suspendedTransaction {
chatService.findPageAccessible(
userId = actor.id,
limit = limit,
cursor = cursor
)
}
when (result) {
is ChatPageResult.InvalidCursor ->
call.respondError(ErrorCode.INVALID_REQUEST, "invalid cursor")
is ChatPageResult.Success ->
call.respond(HttpStatusCode.OK, result.page)
}
}
get("{chat_id}") {
val actor = call.currentUser
val chatId = requireUuid(call.parameters["chat_id"], "chat_id")
val chat = suspendedTransaction {
chatService.findAccessible(actor.id, chatId)
}
if (chat == null) {
call.respondError(ErrorCode.NOT_FOUND, "chat not found")
return@get
}
call.respond(HttpStatusCode.OK, chat)
}
}
}
@@ -0,0 +1,9 @@
package com.mtusi.messenger.main.infrastructure.controllers.chat.requests
import kotlinx.serialization.Serializable
@Serializable
data class CreateChatRequest(
val title: String,
val memberIds: List<String>,
)
@@ -0,0 +1,67 @@
package com.mtusi.messenger.main.infrastructure.controllers.health
import com.mtusi.messenger.main.extensions.respondError
import com.mtusi.messenger.main.infrastructure.controllers.health.responses.HealthResponse
import com.mtusi.messenger.main.infrastructure.types.ErrorCode
import com.mtusi.messenger.main.plugins.isDraining
import com.mtusi.messenger.main.plugins.suspendedTransaction
import com.mtusi.messenger.main.infrastructure.repositories.attachment.storage.ObjectStorageRepository
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import org.koin.ktor.ext.inject
import io.ktor.http.HttpStatusCode
import io.ktor.server.application.log
import io.ktor.server.response.respond
import io.ktor.server.routing.Routing
import io.ktor.server.routing.get
import io.ktor.server.routing.route
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.currentCoroutineContext
import kotlinx.coroutines.ensureActive
fun Routing.configureHealthRouting() = route("/health") {
get("live") {
call.respond(HttpStatusCode.OK, HealthResponse())
}
val storage by inject<ObjectStorageRepository>()
// PostgreSQL and S3 are dependencies of the synchronous path; Redis has a DB fallback.
get("ready") {
// SIGTERM received: leave the balancer's rotation while requests in flight are still being finished.
if (call.application.isDraining) {
call.respondError(ErrorCode.UNAVAILABLE, "shutting down")
return@get
}
val isDatabaseReachable = try {
suspendedTransaction {
exec("SELECT 1")
}
true
} catch (cause: Exception) {
if (cause is CancellationException) {
currentCoroutineContext().ensureActive()
}
call.application.log.warn("Readiness check failed: ${cause.message}")
false
}
if (!isDatabaseReachable) {
call.respondError(ErrorCode.UNAVAILABLE, "database unavailable")
return@get
}
if (!withContext(Dispatchers.IO) { storage.ready() }) {
call.respondError(ErrorCode.UNAVAILABLE, "object storage unavailable")
return@get
}
call.respond(HttpStatusCode.OK, HealthResponse())
}
}
@@ -0,0 +1,6 @@
package com.mtusi.messenger.main.infrastructure.controllers.health.responses
import kotlinx.serialization.Serializable
@Serializable
data class HealthResponse(val status: String = "ok")
@@ -0,0 +1,94 @@
package com.mtusi.messenger.main.infrastructure.controllers.message
import com.mtusi.messenger.main.infrastructure.types.ApiException
import com.mtusi.messenger.main.extensions.currentUser
import com.mtusi.messenger.main.extensions.respondError
import com.mtusi.messenger.main.infrastructure.controllers._common.parameters.pageParameters
import com.mtusi.messenger.main.infrastructure.controllers.message.requests.CreateMessageRequest
import com.mtusi.messenger.main.infrastructure.models.message.MAX_MESSAGE_TEXT_LENGTH
import com.mtusi.messenger.main.infrastructure.services.message.MessageService
import com.mtusi.messenger.main.infrastructure.services.message.results.MessageCreationResult
import com.mtusi.messenger.main.infrastructure.services.message.results.MessagePageResult
import com.mtusi.messenger.main.infrastructure.types.ErrorCode
import com.mtusi.messenger.main.plugins.authentication.SESSION_AUTH
import com.mtusi.messenger.main.plugins.suspendedTransaction
import com.mtusi.messenger.main.utils.requireIdempotencyKey
import com.mtusi.messenger.main.utils.requireUuid
import io.ktor.http.HttpStatusCode
import io.ktor.server.auth.authenticate
import io.ktor.server.request.receive
import io.ktor.server.response.respond
import io.ktor.server.routing.Routing
import io.ktor.server.routing.get
import io.ktor.server.routing.post
import io.ktor.server.routing.route
import org.koin.ktor.ext.inject
fun Routing.configureMessageRouting() = route("/api/v1/chats/{chat_id}/messages") {
val messageService by inject<MessageService>()
authenticate(SESSION_AUTH) {
post {
val actor = call.currentUser
val chatId = requireUuid(call.parameters["chat_id"], "chat_id")
val idempotencyKey = requireIdempotencyKey(call.request.headers["Idempotency-Key"])
val body = call.receive<CreateMessageRequest>()
val attachmentIds = body.attachmentIds.map { requireUuid(it, "attachment_ids") }
if (attachmentIds.size > 10 || attachmentIds.distinct().size != attachmentIds.size ||
body.text.codePointCount(0, body.text.length) > MAX_MESSAGE_TEXT_LENGTH ||
(body.text.isBlank() && attachmentIds.isEmpty())) {
throw ApiException(ErrorCode.INVALID_REQUEST, "invalid message")
}
val result = suspendedTransaction {
messageService.send(
chatId = chatId,
senderId = actor.id,
text = body.text,
idempotencyKey = idempotencyKey,
attachmentIds = attachmentIds
)
}
when (result) {
is MessageCreationResult.ChatNotFound ->
call.respondError(ErrorCode.NOT_FOUND, "chat not found")
is MessageCreationResult.IdempotencyKeyReused ->
call.respondError(ErrorCode.CONFLICT, "Idempotency-Key was used with a different message")
is MessageCreationResult.Success ->
call.respond(HttpStatusCode.Created, result.message)
}
}
get {
val actor = call.currentUser
val chatId = requireUuid(call.parameters["chat_id"], "chat_id")
val (limit, cursor) = call.pageParameters()
val result = suspendedTransaction {
messageService.findPageByChatId(
chatId = chatId,
actorId = actor.id,
limit = limit,
cursor = cursor
)
}
when (result) {
is MessagePageResult.ChatNotFound ->
call.respondError(ErrorCode.NOT_FOUND, "chat not found")
is MessagePageResult.InvalidCursor ->
call.respondError(ErrorCode.INVALID_REQUEST, "invalid cursor")
is MessagePageResult.Success ->
call.respond(HttpStatusCode.OK, result.page)
}
}
}
}
@@ -0,0 +1,9 @@
package com.mtusi.messenger.main.infrastructure.controllers.message.requests
import kotlinx.serialization.Serializable
@Serializable
data class CreateMessageRequest(
val text: String,
val attachmentIds: List<String> = emptyList(),
)
@@ -0,0 +1,83 @@
package com.mtusi.messenger.main.infrastructure.controllers.user
import com.mtusi.messenger.main.extensions.currentUser
import com.mtusi.messenger.main.extensions.respondError
import com.mtusi.messenger.main.infrastructure.controllers.user.requests.CreateUserRequest
import com.mtusi.messenger.main.infrastructure.models.user.MAX_DISPLAY_NAME_LENGTH
import com.mtusi.messenger.main.infrastructure.services.user.UserService
import com.mtusi.messenger.main.infrastructure.services.user.results.UserCreationResult
import com.mtusi.messenger.main.infrastructure.types.ErrorCode
import com.mtusi.messenger.main.plugins.authentication.SESSION_AUTH
import com.mtusi.messenger.main.plugins.suspendedTransaction
import com.mtusi.messenger.main.utils.requireNonBlankBounded
import com.mtusi.messenger.main.utils.requireUsername
import io.ktor.http.HttpStatusCode
import io.ktor.server.auth.authenticate
import io.ktor.server.request.receive
import io.ktor.server.response.respond
import io.ktor.server.routing.Routing
import io.ktor.server.routing.get
import io.ktor.server.routing.post
import io.ktor.server.routing.route
import org.koin.ktor.ext.inject
private const val PASSWORD_MIN = 12
private const val PASSWORD_MAX = 128
fun Routing.configureUserRouting() = route("/api/v1/users") {
val userService by inject<UserService>()
post {
val body = call.receive<CreateUserRequest>()
requireUsername(body.username)
requireNonBlankBounded(
body.displayName,
min = 1,
max = MAX_DISPLAY_NAME_LENGTH,
field = "display_name"
)
requireNonBlankBounded(
body.password,
min = PASSWORD_MIN,
max = PASSWORD_MAX,
field = "password"
)
val result = suspendedTransaction {
userService.create(
username = body.username,
displayName = body.displayName,
password = body.password
)
}
when (result) {
is UserCreationResult.UsernameTaken ->
call.respondError(ErrorCode.CONFLICT, "username already taken")
is UserCreationResult.Success ->
call.respond(HttpStatusCode.Created, result.user)
}
}
authenticate(SESSION_AUTH) {
get("me") {
val user = suspendedTransaction {
userService.findById(call.currentUser.id)
}
if (user == null) {
call.respondError(ErrorCode.UNAUTHORIZED, "unknown user")
return@get
}
call.respond(HttpStatusCode.OK, user)
}
}
}
@@ -0,0 +1,10 @@
package com.mtusi.messenger.main.infrastructure.controllers.user.requests
import kotlinx.serialization.Serializable
@Serializable
data class CreateUserRequest(
val username: String,
val displayName: String,
val password: String,
)
@@ -0,0 +1,21 @@
package com.mtusi.messenger.main.infrastructure.dto
import com.mtusi.messenger.main.compat.serializers.SerialUUID
import kotlinx.serialization.Serializable
import kotlin.time.Instant
@Serializable
data class Attachment(
val id: SerialUUID,
val chatId: SerialUUID,
val ownerId: SerialUUID,
val filename: String,
val contentType: String,
val sizeBytes: Long,
val sha256: String,
val createdAt: Instant,
val updatedAt: Instant = createdAt,
val status: String = "ready",
val variants: List<String> = emptyList(),
val errorCode: String? = null,
)
@@ -0,0 +1,13 @@
package com.mtusi.messenger.main.infrastructure.dto
import com.mtusi.messenger.main.compat.serializers.SerialUUID
import kotlinx.serialization.Serializable
import kotlin.time.Instant
@Serializable
data class Chat(
val id: SerialUUID,
val title: String,
val memberIds: List<SerialUUID>,
val createdAt: Instant,
)
@@ -0,0 +1,15 @@
package com.mtusi.messenger.main.infrastructure.dto
import com.mtusi.messenger.main.compat.serializers.SerialUUID
import kotlinx.serialization.Serializable
import kotlin.time.Instant
@Serializable
data class Message(
val id: SerialUUID,
val chatId: SerialUUID,
val senderId: SerialUUID,
val text: String,
val createdAt: Instant,
val attachmentIds: List<SerialUUID> = emptyList(),
)
@@ -0,0 +1,9 @@
package com.mtusi.messenger.main.infrastructure.dto
import kotlinx.serialization.Serializable
@Serializable
data class PageResponse<T : Any>(
val items: List<T>,
val nextCursor: String?,
)
@@ -0,0 +1,13 @@
package com.mtusi.messenger.main.infrastructure.dto
import com.mtusi.messenger.main.compat.serializers.SerialUUID
import kotlinx.serialization.Serializable
import kotlin.time.Instant
@Serializable
data class User(
val id: SerialUUID,
val username: String,
val displayName: String,
val createdAt: Instant,
)
@@ -0,0 +1,6 @@
package com.mtusi.messenger.main.infrastructure.dto.authentication
data class IssuedSession(
val session: Session,
val token: String,
)
@@ -0,0 +1,12 @@
package com.mtusi.messenger.main.infrastructure.dto.authentication
import java.util.UUID
import kotlin.time.Instant
data class Session(
val id: UUID,
val userId: UUID,
val tokenHash: String,
val createdAt: Instant,
val expiresAt: Instant,
)
@@ -0,0 +1,8 @@
package com.mtusi.messenger.main.infrastructure.dto.authentication
import kotlin.time.Instant
data class SessionRevocation(
val tokenHash: String,
val expiresAt: Instant,
)
@@ -0,0 +1,8 @@
package com.mtusi.messenger.main.infrastructure.dto.authentication
import com.mtusi.messenger.main.infrastructure.dto.User
data class UserCredentials(
val user: User,
val passwordHash: String,
)
@@ -0,0 +1,16 @@
package com.mtusi.messenger.main.infrastructure.models.attachment
import com.mtusi.messenger.main.infrastructure.models.chat.ChatModel
import com.mtusi.messenger.main.infrastructure.models.user.UserModel
import org.jetbrains.exposed.v1.core.dao.id.java.UUIDTable
import org.jetbrains.exposed.v1.datetime.timestamp
object AttachmentModel : UUIDTable("attachments") {
val chatId = reference("chat_id", ChatModel)
val ownerId = reference("owner_id", UserModel)
val filename = varchar("filename", 255)
val contentType = varchar("content_type", 32)
val sizeBytes = long("size_bytes")
val sha256 = varchar("sha256", 64)
val createdAt = timestamp("created_at")
}
@@ -0,0 +1,11 @@
package com.mtusi.messenger.main.infrastructure.models.attachment
import com.mtusi.messenger.main.infrastructure.models.message.MessageModel
import org.jetbrains.exposed.v1.core.Table
object MessageAttachmentModel : Table("message_attachments") {
val messageId = reference("message_id", MessageModel)
val attachmentId = reference("attachment_id", AttachmentModel)
val position = integer("position")
override val primaryKey = PrimaryKey(messageId, attachmentId)
}
@@ -0,0 +1,18 @@
package com.mtusi.messenger.main.infrastructure.models.attachment
import com.mtusi.messenger.main.utils.exposed.operations.plus
import org.jetbrains.exposed.v1.core.dao.id.java.UUIDTable
import org.jetbrains.exposed.v1.core.longLiteral
import org.jetbrains.exposed.v1.datetime.CurrentTimestamp
import org.jetbrains.exposed.v1.datetime.timestamp
object UploadJournalModel : UUIDTable("attachment_uploads") {
val expiresAt = timestamp("expires_at").defaultExpression(CurrentTimestamp + longLiteral(3600))
val abandoned = bool("abandoned").default(false)
val nextAttemptAt = timestamp("next_attempt_at").defaultExpression(CurrentTimestamp + longLiteral(3600))
init {
index("attachment_uploads_next_attempt_idx", false, nextAttemptAt)
}
}
@@ -0,0 +1,17 @@
package com.mtusi.messenger.main.infrastructure.models.authentication
import com.mtusi.messenger.main.infrastructure.models.user.UserModel
import org.jetbrains.exposed.v1.core.ReferenceOption
import org.jetbrains.exposed.v1.core.dao.id.java.UUIDTable
import org.jetbrains.exposed.v1.datetime.timestamp
object SessionModel : UUIDTable("sessions") {
val userId = reference("user_id", UserModel, onDelete = ReferenceOption.CASCADE, onUpdate = ReferenceOption.NO_ACTION, fkName = "sessions_user_id_fkey")
val tokenHash = varchar("token_hash", 64).uniqueIndex("sessions_token_hash_key")
val createdAt = timestamp("created_at")
val expiresAt = timestamp("expires_at")
init {
index("sessions_user_id_idx", false, userId)
}
}
@@ -0,0 +1,11 @@
package com.mtusi.messenger.main.infrastructure.models.authentication
import org.jetbrains.exposed.v1.core.Table
import org.jetbrains.exposed.v1.datetime.timestamp
object SessionRevocationModel : Table("session_revocations") {
val tokenHash = varchar("token_hash", 64)
val expiresAt = timestamp("expires_at")
override val primaryKey = PrimaryKey(tokenHash, name = "session_revocations_pkey")
}
@@ -0,0 +1,17 @@
package com.mtusi.messenger.main.infrastructure.models.chat
import com.mtusi.messenger.main.infrastructure.models.user.UserModel
import org.jetbrains.exposed.v1.core.ReferenceOption
import org.jetbrains.exposed.v1.core.Table
object ChatMemberModel : Table("chat_members") {
val chatId = reference("chat_id", ChatModel, onDelete = ReferenceOption.CASCADE, onUpdate = ReferenceOption.NO_ACTION, fkName = "chat_members_chat_id_fkey")
val userId = reference("user_id", UserModel, onDelete = ReferenceOption.CASCADE, onUpdate = ReferenceOption.NO_ACTION, fkName = "chat_members_user_id_fkey")
val position = integer("position")
override val primaryKey = PrimaryKey(chatId, userId)
init {
index("chat_members_user_id_idx", false, userId)
}
}
@@ -0,0 +1,12 @@
package com.mtusi.messenger.main.infrastructure.models.chat
import org.jetbrains.exposed.v1.core.dao.id.java.UUIDTable
import org.jetbrains.exposed.v1.datetime.timestamp
const val MAX_CHAT_TITLE_LENGTH = 100
const val MAX_CHAT_MEMBERS = 100
object ChatModel : UUIDTable("chats") {
val title = varchar("title", MAX_CHAT_TITLE_LENGTH)
val createdAt = timestamp("created_at")
}
@@ -0,0 +1,25 @@
package com.mtusi.messenger.main.infrastructure.models.message
import com.mtusi.messenger.main.infrastructure.models.chat.ChatModel
import com.mtusi.messenger.main.infrastructure.models.user.UserModel
import org.jetbrains.exposed.v1.core.ReferenceOption
import org.jetbrains.exposed.v1.core.Table
import org.jetbrains.exposed.v1.datetime.timestamp
const val MAX_IDEMPOTENCY_KEY_LENGTH = 128
object MessageIdempotencyKeyModel : Table("message_idempotency_keys") {
val userId = reference("user_id", UserModel, onDelete = ReferenceOption.CASCADE, onUpdate = ReferenceOption.NO_ACTION, fkName = "message_idempotency_keys_user_id_fkey")
val chatId = reference("chat_id", ChatModel, onDelete = ReferenceOption.CASCADE, onUpdate = ReferenceOption.NO_ACTION, fkName = "message_idempotency_keys_chat_id_fkey")
val key = varchar("key", MAX_IDEMPOTENCY_KEY_LENGTH)
val requestHash = varchar("request_hash", 64)
val messageId = optReference("message_id", MessageModel, onDelete = ReferenceOption.CASCADE, onUpdate = ReferenceOption.NO_ACTION, fkName = "message_idempotency_keys_message_id_fkey")
val createdAt = timestamp("created_at")
override val primaryKey = PrimaryKey(userId, chatId, key, name = "message_idempotency_keys_pkey")
init {
index("message_idempotency_keys_chat_id_idx", false, chatId)
index("message_idempotency_keys_message_id_idx", false, messageId)
}
}

Some files were not shown because too many files have changed in this diff Show More