ЛР 4 — БПИ2304 — Кирюпина — на 3 #127

Closed
alexandrakirupina wants to merge 1 commits from alexandrakirupina/mtusi2026lab4:solution/bpi2304-kiryupina into main
39 changed files with 3361 additions and 36 deletions
+5 -18
View File
@@ -1,20 +1,7 @@
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
# These are logical interface suggestions; map actual framework names in REPORT.md.
APP_HOST=0.0.0.0
APP_PORT=8080
APP_ENV=lab
# Generate .env and local credentials with scripts/setup.py; do not commit them.
# Use 8084 to keep lab3 on 8080; API replicas listen on internal port 8080.
APP_PORT=8084
SESSION_TTL_SECONDS=3600
# Choose credentials locally. File paths contain no secret values.
DATABASE_URL_FILE=/run/secrets/database_url
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
API_REPLICAS=2
IDEMPOTENCY_TTL_SECONDS=86400
S3_ENDPOINT=http://s3:9000
S3_REGION=us-east-1
S3_BUCKET=messenger-attachments
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
MAX_UPLOAD_BYTES=10485760
MAX_IMAGE_PIXELS=40000000
# Storage address/bucket/secrets paths are fixed in compose.yaml.
# Upload limits: 10485760 bytes and 40000000 pixels.
+3
View File
@@ -0,0 +1,3 @@
*.sh text eol=lf
*.cmd text eol=crlf
*.sql text eol=lf
+9
View File
@@ -0,0 +1,9 @@
FROM node:22.20.0-alpine
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev --ignore-scripts
COPY src/ ./src/
COPY migrations/ ./migrations/
ENV APP_HOST=0.0.0.0 APP_PORT=8080 APP_ENV=lab
EXPOSE 8080
CMD ["node", "src/server.js"]
+43 -18
View File
@@ -1,37 +1,62 @@
# Отчёт — лабораторная 4
- Автор, группа: TODO
- Целевая оценка; выбранный трек 5, если нужен: TODO
- Commit/PR: TODO после публикации
- Использованные источники, библиотеки и помощь ИИ: TODO
- Автор, группа: Кирюпина Александра Радиевна, БПИ2304.
- Целевая оценка: 3. Трек 5 не выбран.
- Ветка: `solution/bpi2304-kiryupina`; PR: `ЛР 4 — БПИ2304 — Кирюпина — на 3` (ссылка после публикации).
- Источники: контракт курса, документация [MinIO](https://github.com/minio/minio/tree/RELEASE.2025-04-22T22-12-26Z), [Sharp](https://sharp.pixelplumbing.com/). Библиотеки: pg 8.16.3, minio 8.0.5, sharp 0.34.3. ИИ использовался при подготовке кода, проверок и отчёта.
## Запуск из чистого clone
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
Нужны Docker Desktop с Linux containers и Python 3.9+. Команды из папки проекта в PowerShell:
```powershell
py -X utf8 scripts/setup.py
docker compose up -d --build --wait
py -X utf8 scripts/check_contract.py
py -X utf8 tests/smoke.py --base-url http://localhost:8084
```
Setup один раз создаёт локальные `.env` и `secrets/`, они не входят в Git. В `.env` задаются внешний `APP_PORT=8084`, TTL сессии 3600 с и ключа идемпотентности 86400 с. Внутренние адреса, bucket и пути secret-файлов заданы в Compose. Готовность: `GET /health/ready` → 200.
Версии: Node 22.20.0-alpine, PostgreSQL 17.6-alpine, HAProxy 3.0.12-alpine. MinIO собирается из исходников релиза `RELEASE.2025-04-22T22-12-26Z` через Go 1.24.3-alpine, runtime Alpine 3.22.1 (`infra/Dockerfile.s3`); готовый образ из реестра не нужен. Первый build требует доступа к GitHub и репозиториям зависимостей. Миграции и создание приватного bucket автоматические.
Повторная сборка/запуск — та же команда `docker compose up -d --build --wait`. Остановка с сохранением данных — `docker compose down`. Намеренное удаление учебных данных — `docker compose down -v`. Локальные secrets сохраняются между запусками. Linux: `python3` вместо `py -X utf8`; эквиваленты проверок — `make check`, `make test BASE_URL=http://localhost:8084`.
## Архитектура и решения
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
Клиент → HAProxy → два API → PostgreSQL + MinIO. Из lab3 перенесены сессии, членство в чатах, пагинация и общая идемпотентность сообщений. Текстовые сообщения продолжают работать; добавлен список `attachment_ids` (до 10 уникальных ID того же чата). Запись сообщения и результата повтора выполняется одной транзакцией PostgreSQL.
Метаданные вложений хранятся в PostgreSQL, оригиналы — в приватном bucket `messenger-attachments`. У API нет volume с файлами. Имя файла используется только для отображения; ключ S3 случайный и скрыт. Проверяются PNG/JPEG, MIME, имя, лимит 10 MiB и 40 млн пикселей до полной распаковки. Повреждённые изображения отклоняются. После декодирования сохраняются исходные байты, без преобразования.
Ответ 201 ready выдаётся после S3 PUT и записи метаданных. Если INSERT не удался, выполняется попытка удалить объект. Download идёт через API с проверкой членства, без redirect. Оба хранилища имеют persistent volumes. Readiness зависит от PostgreSQL и S3; liveness — только от процесса. Пагинация сортирует `(created_at,id)`; при конкурентных вставках snapshot между страницами не гарантируется.
## Доказательства критериев
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
| --- | --- | --- | --- |
| Публичный smoke | make test | TODO | TODO |
| Остальные пункты уровня 3 | TODO | TODO | TODO |
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
Результаты запусков и состояния контейнеров записаны в `evidence/`. Команды persistence/failover ниже используют неизменённые публичные тесты. На Windows выполнены прямые Python-команды, эквивалентные make-целям.
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
| Критерий | Проверка | Наблюдение | Файл в evidence/ |
| --- | --- | --- | --- |
| Контракт и публичный API | check_contract.py, smoke.py | PASS; 17 файлов задания/контракта/тестов не изменены | check.txt, smoke.txt, contract-integrity.txt |
| Приватный S3, PNG/JPEG и ограничения | student-tests/check.js | Точные байты и SHA-256; anonymous S3 GET=403; неверные файлы отклонены | student-tests.txt |
| Остановка принявшей файл реплики → download другой | check.js --lifecycle | Точные байты, 4054 и 2199 мс с учётом hook | student-tests.txt, stop-api*.json |
| Полный restart с сохранением оригинала и метаданных | публичный persistence | PASS: данные и прежняя сессия сохранены | persistence.txt, restart-scope.json |
| Работа при отказе API, общая идемпотентность | публичный failover | PASS: 2 реплики; восстановление 5.156 с; 6 успешных записей, 1 неуспешная попытка во время переключения | failover.txt |
```powershell
py -X utf8 tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url http://localhost:8084
py -X utf8 tests/resilience.py --mode failover --action-script scripts/stop-one.cmd --base-url http://localhost:8084
py -X utf8 scripts/control.py restore
node student-tests/check.js --lifecycle
```
Для последней команды нужен Node.js 22+. На Linux hooks — `scripts/restart.sh` и `scripts/stop-one.sh` с executable bit. Hook restart перезапускает LB, оба API, PostgreSQL и MinIO, не удаляет volumes; stop-one останавливает именно переданную API-реплику.
## Эксперименты и отказы
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
Локальный стенд: Windows, Docker Desktop 29.9.2, 12 CPU, около 7.7 GiB RAM у Docker; отдельных лимитов контейнеров нет (`evidence/environment.txt`). Это функциональные проверки, измерение максимальной производительности не выполнялось.
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
| --- | --- | --- | --- |
| TODO | TODO | TODO | TODO |
Проверены fixture PNG 320×200 (111817 байт), JPEG 20×10, повреждённые файлы, подмена MIME, опасные имена, превышение bytes (включая chunked) и пикселей. 12 параллельных отправок с одним ключом создали одно сообщение. После остановки загрузившей реплики файл прочитан через другую; полный restart сохранил файл, сессию и повтор сообщения. При отключённом S3 получены live=200 и ready=503, после возврата S3 — прежний оригинал. Все дополнительные проверки прошли; вывод в student-tests.txt.
## Ограничения и следующие шаги
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
Общей транзакции PostgreSQL/S3 нет: авария между операциями может оставить лишний объект, автоматической сверки пока нет. API использует общие административные S3 credentials; отдельные ограниченные права — следующий шаг. Оригинал буферизуется до 10 MiB, декодирование ограничено 40 млн пикселей, но общего лимита памяти для множества параллельных загрузок нет. LB, PostgreSQL, S3 и сам компьютер остаются единичными точками отказа. Миниатюры, фоновые задачи и треки 4/5 не реализованы.
+44
View File
@@ -0,0 +1,44 @@
# Лабораторная 4 — запуск в PowerShell
Открой эту папку в VS Code и терминал PowerShell. Нужны запущенный Docker
Desktop (Linux containers), Python 3.9+; для дополнительных тестов — Node.js 22+.
```powershell
py -X utf8 scripts/setup.py
docker compose up -d --build --wait
py -X utf8 scripts/check_contract.py
py -X utf8 tests/smoke.py --base-url http://localhost:8084
```
API доступен на http://localhost:8084. Порт 8080 предыдущей лабы не занят.
Первый запуск скачивает зависимости и собирает MinIO из исходников закреплённого
релиза: это может занять несколько минут. Повторная сборка использует кэш.
Миграции и создание приватного bucket выполняются автоматически.
Проверка полного перезапуска и отказа API (запускай по очереди):
```powershell
py -X utf8 tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url http://localhost:8084
py -X utf8 tests/resilience.py --mode failover --action-script scripts/stop-one.cmd --base-url http://localhost:8084
py -X utf8 scripts/control.py restore
node student-tests/check.js --lifecycle
```
Дополнительный тест сам загружает PNG/JPEG, проверяет ограничения и права доступа,
останавливает принявшую файл реплику, скачивает оригинал через вторую,
перезапускает весь стенд и временно отключает S3. После проверок восстанавливает сервисы.
Остановка с сохранением данных: `docker compose down`.
Повторный запуск и сборка: `docker compose up -d --build --wait`.
Только для намеренного удаления всех учебных данных этой лабы: `docker compose down -v`.
Сохраняй папку secrets между запусками; не добавляй её и .env в Git или архив сдачи.
Если переносишь уже работающий стенд в другую папку на этом же компьютере,
перенеси `.env` и `secrets/` туда отдельно: существующие Docker volumes ожидают
прежние пароли. Для текущего запуска используй уже подготовленную папку проекта.
Linux: вместо `py -X utf8` используй `python3`, вместо hooks `.cmd` — `.sh`.
Перед запуском hooks: `chmod +x scripts/restart.sh scripts/stop-one.sh`.
Эквиваленты публичных проверок: `make check` и `make test BASE_URL=http://localhost:8084`.
Короткий отчёт — REPORT.md; результаты реальных проверок — evidence/.
Заголовок PR: **ЛР 4 — БПИ2304 — Кирюпина — на 3**.
+106
View File
@@ -0,0 +1,106 @@
name: mtusi-lab4
x-s3-env: &s3-env
S3_ENDPOINT: http://s3:9000
S3_REGION: us-east-1
S3_BUCKET: messenger-attachments
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_secret_access_key
x-api: &api
image: mtusi-messenger:lab4
build: .
environment:
<<: *s3-env
APP_HOST: 0.0.0.0
APP_PORT: 8080
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400}
DATABASE_URL_FILE: /run/secrets/database_url
secrets: [database_url, s3_access_key_id, s3_secret_access_key]
depends_on:
migrate:
condition: service_completed_successfully
s3-init:
condition: service_completed_successfully
healthcheck:
test: ["CMD", "node", "-e", "fetch('http://127.0.0.1:8080/health/ready').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"]
interval: 2s
timeout: 4s
retries: 20
stop_grace_period: 10s
services:
db:
image: postgres:17.6-alpine
environment:
POSTGRES_DB: messenger
POSTGRES_USER: messenger
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
secrets: [postgres_password]
volumes: [pgdata:/var/lib/postgresql/data]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
interval: 2s
timeout: 3s
retries: 30
migrate:
image: mtusi-messenger:lab4
build: .
command: ["node", "src/migrate.js"]
environment:
DATABASE_URL_FILE: /run/secrets/database_url
secrets: [database_url]
depends_on:
db:
condition: service_healthy
restart: "no"
s3:
image: mtusi-minio:RELEASE.2025-04-22T22-12-26Z
build:
context: .
dockerfile: infra/Dockerfile.s3
command: ["server", "/data", "--console-address", ":9001"]
environment:
MINIO_ROOT_USER_FILE: /run/secrets/s3_access_key_id
MINIO_ROOT_PASSWORD_FILE: /run/secrets/s3_secret_access_key
secrets: [s3_access_key_id, s3_secret_access_key]
volumes: [s3data:/data]
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 2s
timeout: 3s
retries: 30
s3-init:
image: mtusi-messenger:lab4
build: .
command: ["node", "src/storage.js"]
environment:
<<: *s3-env
secrets: [s3_access_key_id, s3_secret_access_key]
depends_on:
s3:
condition: service_healthy
restart: "no"
api1:
<<: *api
api2:
<<: *api
lb:
image: haproxy:3.0.12-alpine
ports: ["127.0.0.1:${APP_PORT:-8084}:8080"]
volumes: [./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro]
depends_on:
api1:
condition: service_healthy
api2:
condition: service_healthy
volumes:
pgdata:
s3data:
secrets:
postgres_password:
file: ./secrets/postgres_password
database_url:
file: ./secrets/database_url
s3_access_key_id:
file: ./secrets/s3_access_key_id
s3_secret_access_key:
file: ./secrets/s3_secret_access_key
+476
View File
@@ -0,0 +1,476 @@
Image mtusi-messenger:lab4 Building
Image mtusi-minio:RELEASE.2025-04-22T22-12-26Z Building
Image mtusi-messenger:lab4 Building
Image mtusi-messenger:lab4 Building
Image mtusi-messenger:lab4 Building
#1 [internal] load local bake definitions
#1 reading from stdin 2.92kB 0.0s done
#1 DONE 0.0s
#2 [api1 internal] load build definition from Dockerfile
#2 transferring dockerfile: 292B done
#2 DONE 0.0s
#3 [s3 internal] load build definition from Dockerfile.s3
#3 transferring dockerfile: 596B done
#3 DONE 0.0s
#4 [migrate internal] load metadata for docker.io/library/node:22.20.0-alpine
#4 ...
#5 [s3 internal] load metadata for docker.io/library/alpine:3.22.1
#5 ...
#6 [s3 internal] load metadata for docker.io/library/golang:1.24.3-alpine
#6 ...
#5 [s3 internal] load metadata for docker.io/library/alpine:3.22.1
#5 DONE 12.4s
#4 [migrate internal] load metadata for docker.io/library/node:22.20.0-alpine
#4 ...
#6 [s3 internal] load metadata for docker.io/library/golang:1.24.3-alpine
#6 DONE 12.6s
#7 [migrate internal] load .dockerignore
#7 transferring context: 196B done
#7 DONE 0.0s
#4 [api1 internal] load metadata for docker.io/library/node:22.20.0-alpine
#4 DONE 12.6s
#8 [migrate internal] load build context
#8 transferring context: 474B done
#8 DONE 0.0s
#9 [s3 build 1/5] FROM docker.io/library/golang:1.24.3-alpine@sha256:b4f875e650466fa0fe62c6fd3f02517a392123eea85f1d7e69d85f780e4db1c1
#9 resolve docker.io/library/golang:1.24.3-alpine@sha256:b4f875e650466fa0fe62c6fd3f02517a392123eea85f1d7e69d85f780e4db1c1 0.0s done
#9 DONE 0.0s
#10 [s3 build 3/5] RUN git clone --depth 1 --branch RELEASE.2025-04-22T22-12-26Z https://github.com/minio/minio.git /src
#10 CACHED
#11 [s3 build 2/5] RUN apk add --no-cache git || apk add --no-cache git || apk add --no-cache git
#11 CACHED
#12 [s3 build 4/5] WORKDIR /src
#12 CACHED
#13 [s3 stage-1 1/3] FROM docker.io/library/alpine:3.22.1@sha256:4bcff63911fcb4448bd4fdacec207030997caf25e9bea4045fa6c8c44de311d1
#13 resolve docker.io/library/alpine:3.22.1@sha256:4bcff63911fcb4448bd4fdacec207030997caf25e9bea4045fa6c8c44de311d1 0.0s done
#13 DONE 0.1s
#14 [s3 stage-1 2/3] RUN apk add --no-cache ca-certificates curl
#14 CACHED
#15 [migrate 1/6] FROM docker.io/library/node:22.20.0-alpine@sha256:dbcedd8aeab47fbc0f4dd4bffa55b7c3c729a707875968d467aaaea42d6225af
#15 resolve docker.io/library/node:22.20.0-alpine@sha256:dbcedd8aeab47fbc0f4dd4bffa55b7c3c729a707875968d467aaaea42d6225af 0.0s done
#15 DONE 0.0s
#16 [migrate 4/6] RUN npm ci --omit=dev --ignore-scripts
#16 CACHED
#17 [migrate 2/6] WORKDIR /app
#17 CACHED
#18 [migrate 5/6] COPY src/ ./src/
#18 CACHED
#19 [migrate 3/6] COPY package.json package-lock.json ./
#19 CACHED
#20 [migrate 6/6] COPY migrations/ ./migrations/
#20 CACHED
#21 [api1] exporting to image
#21 exporting layers done
#21 exporting manifest sha256:e366d8aa98f16a1bd289889387fdf3401a716a731cdabf9b0a834d3683c8ae39 done
#21 exporting config sha256:0ffd6c566da86551d5fbee607b6d9565c714596a62bc96b7e37a2c03934a0dbd
#21 exporting config sha256:0ffd6c566da86551d5fbee607b6d9565c714596a62bc96b7e37a2c03934a0dbd done
#21 exporting attestation manifest sha256:b3922880990bdf70372adb55bb1bd697b432717f6a5aaace3d3f6669b7283010
#21 exporting attestation manifest sha256:b3922880990bdf70372adb55bb1bd697b432717f6a5aaace3d3f6669b7283010 0.1s done
#21 exporting manifest list sha256:0fd8f69f2afa0bc0d3f70e2142c49e708fc53f70f552ff07fa515d6437db2193 0.1s done
#21 naming to docker.io/library/mtusi-messenger:lab4 0.0s done
#21 unpacking to docker.io/library/mtusi-messenger:lab4
#21 unpacking to docker.io/library/mtusi-messenger:lab4 0.0s done
#21 DONE 0.3s
#22 [s3-init] exporting to image
#22 exporting layers done
#22 exporting manifest sha256:70d2bcedbcf9ca498b7814d5f6a06f8ffd4fdde5d4f3529076a038f240f1d9ab 0.0s done
#22 exporting config sha256:84ad10b06644c20475de71e28b2781a193ec8a190de29d725d7d294752759ba0 0.0s done
#22 exporting attestation manifest sha256:4c05668b90e165538b4451e3785939ecb6c88ef328be01e8b6185d34542081b5 0.1s done
#22 exporting manifest list sha256:caaf8e14d21f60fa57d8c26684b7fff06a4ab1efe9a130c704a409537958a272 0.0s done
#22 naming to docker.io/library/mtusi-messenger:lab4 0.0s done
#22 unpacking to docker.io/library/mtusi-messenger:lab4 0.0s done
#22 DONE 0.3s
#23 [migrate] exporting to image
#23 exporting layers done
#23 exporting manifest sha256:a0571ea49777a38abe14d14c27b759b4cab439713afb494ad8b22dc7e3513201 0.0s done
#23 exporting config sha256:db473997689da60bee5064a412adb972245d3642e4ed03597357c52af6569d09 0.0s done
#23 exporting attestation manifest sha256:ef7595a236175ff992c6293c425c6bc63201ef830e33e60314c95d22af9cbae9 0.1s done
#23 exporting manifest list sha256:ecfcc7984ef7a74654816bab8e4a129a2699c1dd2795860d52151ffe771cae80 0.0s done
#23 naming to docker.io/library/mtusi-messenger:lab4 0.0s done
#23 unpacking to docker.io/library/mtusi-messenger:lab4 0.0s done
#23 DONE 0.3s
#24 [api2] exporting to image
#24 exporting layers done
#24 exporting manifest sha256:75185eadd9eb95e48fbe46d04ab10dd4513f5bce34fd440f3b420e45e018c34a 0.0s done
#24 exporting config sha256:a0bf9ce288f460f565a7c8bf8004bc2e92f86c403e2b07d60679127adaa8694a 0.0s done
#24 exporting attestation manifest sha256:10c05168a828f7ce7095b7de63ee3a47287f5a8670da0507bebe958bf7a33b8b 0.1s done
#24 exporting manifest list sha256:96ceb3e9a5f8bd0269f913a31d95027d1cafbfdbc48022836ed1a4ea58d9578c 0.0s done
#24 naming to docker.io/library/mtusi-messenger:lab4 done
#24 unpacking to docker.io/library/mtusi-messenger:lab4 0.0s done
#24 DONE 0.3s
#25 [s3 build 5/5] RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build for attempt in 1 2 3 4 5; do CGO_ENABLED=0 go build -trimpath -o /minio . && exit 0; done; exit 1
#25 0.526 go: downloading cloud.google.com/go/storage v1.46.0
#25 0.529 go: downloading github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0
#25 0.531 go: downloading github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.2
#25 0.531 go: downloading github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.0
#25 0.531 go: downloading github.com/cheggaaa/pb v1.0.29
#25 0.531 go: downloading github.com/cespare/xxhash/v2 v2.3.0
#25 0.533 go: downloading github.com/coreos/go-oidc/v3 v3.12.0
#25 0.533 go: downloading github.com/coreos/go-systemd/v22 v22.5.0
#25 0.534 go: downloading github.com/cosnicolaou/pbzip2 v1.0.5
#25 0.551 go: downloading github.com/dchest/siphash v1.2.3
#25 ...
#26 [migrate] resolving provenance for metadata file
#26 DONE 0.0s
#27 [api1] resolving provenance for metadata file
#27 DONE 0.1s
#28 [s3-init] resolving provenance for metadata file
#28 DONE 0.1s
#29 [api2] resolving provenance for metadata file
#29 DONE 0.0s
#25 [s3 build 5/5] RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build for attempt in 1 2 3 4 5; do CGO_ENABLED=0 go build -trimpath -o /minio . && exit 0; done; exit 1
#25 2.411 go: downloading github.com/dustin/go-humanize v1.0.1
#25 2.420 go: downloading github.com/fatih/color v1.18.0
#25 2.421 go: downloading github.com/felixge/fgprof v0.9.5
#25 2.453 go: downloading github.com/go-openapi/loads v0.22.0
#25 3.495 go: downloading github.com/golang-jwt/jwt/v4 v4.5.2
#25 3.631 go: downloading github.com/google/uuid v1.6.0
#25 3.631 go: downloading github.com/inconshreveable/mousetrap v1.1.0
#25 3.741 go: downloading github.com/json-iterator/go v1.1.12
#25 3.743 go: downloading github.com/klauspost/compress v1.18.0
#25 3.764 go: downloading cloud.google.com/go v0.118.0
#25 3.852 go: downloading github.com/klauspost/filepathx v1.1.1
#25 3.855 go: downloading github.com/klauspost/pgzip v1.2.6
#25 3.858 go: downloading github.com/klauspost/readahead v1.4.0
#25 3.878 go: downloading github.com/klauspost/reedsolomon v1.12.4
#25 3.878 go: downloading github.com/lithammer/shortuuid/v4 v4.2.0
#25 4.002 go: downloading github.com/miekg/dns v1.1.63
#25 4.183 go: downloading github.com/minio/cli v1.24.2
#25 4.202 go: downloading github.com/minio/console v1.7.6
#25 4.408 go: downloading github.com/minio/dnscache v0.1.1
#25 4.409 go: downloading github.com/minio/dperf v0.6.3
#25 4.438 go: downloading github.com/minio/highwayhash v1.0.3
#25 4.584 go: downloading github.com/minio/kms-go/kes v0.3.1
#25 4.654 go: downloading github.com/minio/madmin-go/v3 v3.0.109
#25 5.132 go: downloading github.com/minio/minio-go/v7 v7.0.90
#25 5.618 go: downloading github.com/minio/mux v1.9.2
#25 5.620 go: downloading github.com/minio/pkg/v3 v3.1.0
#25 6.131 go: downloading github.com/minio/selfupdate v0.6.0
#25 6.131 go: downloading github.com/minio/xxml v0.0.3
#25 6.131 go: downloading github.com/minio/sio v0.4.1
#25 6.345 go: downloading github.com/minio/zipindex v0.4.0
#25 6.354 go: downloading github.com/mitchellh/go-homedir v1.1.0
#25 6.438 go: downloading github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c
#25 7.265 go: downloading github.com/pierrec/lz4/v4 v4.1.22
#25 7.871 go: downloading github.com/pkg/errors v0.9.1
#25 8.014 go: downloading github.com/pkg/sftp v1.13.8
#25 8.103 go: downloading github.com/pkg/xattr v0.4.10
#25 8.840 go: downloading github.com/prometheus/client_golang v1.21.1
#25 9.484 go: downloading github.com/prometheus/client_model v0.6.2
#25 9.484 go: downloading github.com/prometheus/common v0.63.0
#25 9.905 go: downloading github.com/prometheus/procfs v0.16.0
#25 10.07 go: downloading github.com/puzpuzpuz/xsync/v3 v3.5.1
#25 10.55 go: downloading github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475
#25 10.61 go: downloading github.com/rs/cors v1.11.1
#25 10.80 go: downloading github.com/secure-io/sio-go v0.3.1
#25 10.80 go: downloading github.com/shirou/gopsutil/v3 v3.24.5
#25 11.28 go: downloading github.com/tinylib/msgp v1.2.5
#25 11.28 go: downloading github.com/valyala/bytebufferpool v1.0.0
#25 11.59 go: downloading github.com/zeebo/xxh3 v1.0.2
#25 11.94 go: downloading go.etcd.io/etcd/api/v3 v3.5.19
#25 12.04 go: downloading go.etcd.io/etcd/client/v3 v3.5.19
#25 12.37 go: downloading go.uber.org/atomic v1.11.0
#25 12.60 go: downloading goftp.io/server/v2 v2.0.1
#25 12.78 go: downloading golang.org/x/crypto v0.37.0
#25 14.67 go: downloading golang.org/x/oauth2 v0.28.0
#25 14.67 go: downloading golang.org/x/sync v0.13.0
#25 14.89 go: downloading golang.org/x/sys v0.32.0
#25 17.37 go: downloading golang.org/x/term v0.31.0
#25 17.58 go: downloading google.golang.org/api v0.224.0
#25 73.71 go: downloading gopkg.in/yaml.v2 v2.4.0
#25 73.71 go: downloading gopkg.in/yaml.v3 v3.0.1
#25 73.77 go: downloading github.com/mattn/go-runewidth v0.0.16
#25 74.07 go: downloading github.com/go-jose/go-jose/v4 v4.0.5
#25 74.09 go: downloading github.com/mattn/go-colorable v0.1.14
#25 74.61 go: downloading github.com/mattn/go-isatty v0.0.20
#25 74.67 go: downloading github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0
#25 74.98 go: downloading github.com/AzureAD/microsoft-authentication-library-for-go v1.4.1
#25 75.54 go: downloading github.com/google/pprof v0.0.0-20250302191652-9094ed2288e7
#25 75.54 go: downloading github.com/go-openapi/analysis v0.23.0
#25 75.54 go: downloading github.com/go-openapi/spec v0.21.0
#25 78.43 go: downloading github.com/go-openapi/swag v0.23.1
#25 79.26 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
#25 80.95 go: downloading github.com/modern-go/reflect2 v1.0.2
#25 81.17 go: downloading github.com/klauspost/cpuid/v2 v2.2.10
#25 81.17 go: downloading golang.org/x/net v0.39.0
#25 83.94 go: downloading cloud.google.com/go/compute/metadata v0.6.0
#25 83.94 go: downloading cloud.google.com/go/iam v1.3.1
#25 84.52 go: downloading github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.51.0
#25 84.52 go: downloading github.com/googleapis/gax-go/v2 v2.14.1
#25 84.89 go: downloading go.opentelemetry.io/contrib/detectors/gcp v1.35.0
#25 85.08 go: downloading go.opentelemetry.io/otel v1.35.0
#25 85.30 go: downloading go.opentelemetry.io/otel/sdk/metric v1.35.0
#25 87.57 go: downloading go.opentelemetry.io/otel/sdk v1.35.0
#25 107.7 go: downloading google.golang.org/genproto v0.0.0-20250106144421-5f5ef82da422
#25 107.7 go: downloading google.golang.org/grpc v1.71.0
#25 119.9 go: downloading google.golang.org/protobuf v1.36.6
#25 125.0 go: downloading github.com/ncw/directio v1.0.5
#25 125.0 go: downloading aead.dev/mem v0.2.0
#25 125.0 go: downloading github.com/prometheus/prom2json v1.4.2
#25 125.0 go: downloading github.com/safchain/ethtool v0.5.10
#25 125.0 go: downloading github.com/goccy/go-json v0.10.5
#25 125.0 go: downloading github.com/minio/crc64nvme v1.0.1
#25 125.3 go: downloading github.com/go-ini/ini v1.67.0
#25 125.3 go: downloading github.com/minio/md5-simd v1.1.2
#25 125.4 go: downloading golang.org/x/time v0.11.0
#25 125.4 go: downloading github.com/rs/xid v1.6.0
#25 125.4 go: downloading github.com/beevik/ntp v1.4.3
#25 125.4 go: downloading go.uber.org/zap v1.27.0
#25 125.4 go: downloading github.com/go-ldap/ldap/v3 v3.4.10
#25 125.4 go: downloading github.com/IBM/sarama v1.45.1
#25 125.4 go: downloading github.com/eclipse/paho.mqtt.golang v1.5.0
#25 125.4 go: downloading github.com/elastic/go-elasticsearch/v7 v7.17.10
#25 125.7 go: downloading github.com/go-sql-driver/mysql v1.9.0
#25 126.7 go: downloading github.com/gomodule/redigo v1.9.2
#25 126.7 go: downloading github.com/nats-io/nats.go v1.39.1
#25 126.7 go: downloading github.com/lib/pq v1.10.9
#25 126.7 go: downloading github.com/nats-io/stan.go v0.10.4
#25 128.4 go: downloading github.com/nsqio/go-nsq v1.1.0
#25 128.4 go: downloading github.com/rabbitmq/amqp091-go v1.10.0
#25 128.4 go: downloading github.com/xdg/scram v1.0.5
#25 129.6 go: downloading github.com/gobwas/ws v1.4.0
#25 130.0 go: downloading github.com/buger/jsonparser v1.1.1
#25 130.0 go: downloading github.com/minio/kms-go/kms v0.4.0
#25 130.0 go: downloading github.com/minio/simdjson-go v0.4.5
#25 130.0 go: downloading github.com/rjeczalik/notify v0.9.3
#25 130.0 go: downloading github.com/lestrrat-go/jwx/v2 v2.1.4
#25 130.2 go: downloading aead.dev/minisign v0.3.0
#25 130.2 go: downloading github.com/fatih/structs v1.1.0
#25 131.5 go: downloading github.com/kr/fs v0.1.0
#25 131.5 go: downloading github.com/beorn7/perks v1.0.1
#25 131.5 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822
#25 131.5 go: downloading github.com/tklauser/go-sysconf v0.3.15
#25 131.5 go: downloading github.com/gogo/protobuf v1.3.2
#25 131.5 go: downloading github.com/golang/protobuf v1.5.4
#25 133.2 go: downloading go.etcd.io/etcd/client/pkg/v3 v3.5.19
#25 135.7 go: downloading github.com/go-openapi/errors v0.22.0
#25 135.7 go: downloading github.com/go-openapi/runtime v0.28.0
#25 139.2 go: downloading github.com/jessevdk/go-flags v1.6.1
#25 139.2 go: downloading github.com/minio/mc v0.0.0-20250312172924-c1d5d4cbb4ca
#25 140.4 go: downloading github.com/go-openapi/strfmt v0.23.0
#25 140.4 go: downloading github.com/minio/websocket v1.6.0
#25 141.0 go: downloading github.com/unrolled/secure v1.17.0
#25 142.8 go: downloading github.com/rivo/uniseg v0.4.7
#25 142.9 go: downloading github.com/kylelemons/godebug v1.1.0
#25 142.9 go: downloading github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c
#25 142.9 go: downloading github.com/go-openapi/jsonpointer v0.21.1
#25 142.9 go: downloading github.com/mailru/easyjson v0.9.0
#25 142.9 go: downloading github.com/go-openapi/jsonreference v0.21.0
#25 143.3 go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20250311190419-81fb87f6b8bf
#25 144.5 go: downloading go.opentelemetry.io/otel/trace v1.35.0
#25 144.9 go: downloading cloud.google.com/go/monitoring v1.23.0
#25 144.9 go: downloading github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.51.0
#25 144.9 go: downloading github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.27.0
#25 145.0 go: downloading go.opentelemetry.io/otel/metric v1.35.0
#25 145.7 go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20250311190419-81fb87f6b8bf
#25 145.7 go: downloading cloud.google.com/go/auth v0.15.0
#25 145.7 go: downloading cloud.google.com/go/auth/oauth2adapt v0.2.7
#25 145.7 go: downloading go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0
#25 145.7 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4
#25 145.8 go: downloading go.uber.org/multierr v1.11.0
#25 145.8 go: downloading github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358
#25 145.9 go: downloading github.com/go-asn1-ber/asn1-ber v1.5.7
#25 145.9 go: downloading github.com/nats-io/nuid v1.0.1
#25 145.9 go: downloading github.com/gorilla/websocket v1.5.3
#25 146.3 go: downloading filippo.io/edwards25519 v1.1.0
#25 146.7 go: downloading github.com/eapache/go-resiliency v1.7.0
#25 146.7 go: downloading github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3
#25 146.7 go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
#25 146.8 go: downloading github.com/eapache/queue v1.1.0
#25 146.8 go: downloading github.com/jcmturner/gofork v1.7.6
#25 146.8 go: downloading github.com/hashicorp/go-multierror v1.1.1
#25 146.8 go: downloading github.com/jcmturner/gokrb5/v8 v8.4.4
#25 147.4 go: downloading github.com/minio/csvparser v1.0.0
#25 147.4 go: downloading github.com/fraugster/parquet-go v0.12.0
#25 148.1 go: downloading github.com/alecthomas/participle v0.7.1
#25 148.1 go: downloading github.com/golang/snappy v1.0.0
#25 148.1 go: downloading github.com/xdg/stringprep v1.0.3
#25 148.1 go: downloading github.com/nats-io/nkeys v0.4.10
#25 148.1 go: downloading github.com/gobwas/httphead v0.1.0
#25 148.1 go: downloading github.com/coreos/go-semver v0.3.1
#25 148.1 go: downloading github.com/gobwas/pool v0.2.1
#25 148.5 go: downloading github.com/lestrrat-go/option v1.0.1
#25 148.5 go: downloading github.com/lestrrat-go/blackmagic v1.0.2
#25 148.5 go: downloading github.com/lestrrat-go/httprc v1.0.6
#25 148.5 go: downloading github.com/lestrrat-go/iter v1.0.2
#25 148.6 go: downloading github.com/go-openapi/validate v0.24.0
#25 148.6 go: downloading github.com/tklauser/numcpus v0.10.0
#25 148.6 go: downloading github.com/golang-jwt/jwt/v5 v5.2.2
#25 148.7 go: downloading github.com/docker/go-units v0.5.0
#25 149.8 go: downloading github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2
#25 149.8 go: downloading github.com/mitchellh/mapstructure v1.5.0
#25 149.8 go: downloading go.mongodb.org/mongo-driver v1.17.3
#25 149.8 go: downloading github.com/oklog/ulid v1.3.1
#25 149.8 go: downloading github.com/go-logr/logr v1.4.2
#25 149.8 go: downloading go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.59.0
#25 149.8 go: downloading github.com/charmbracelet/bubbles v0.20.0
#25 152.3 go: downloading github.com/charmbracelet/bubbletea v1.3.4
#25 152.3 go: downloading github.com/charmbracelet/lipgloss v1.0.0
#25 155.3 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
#25 155.9 go: downloading github.com/jedib0t/go-pretty/v6 v6.6.7
#25 155.9 go: downloading github.com/mattn/go-ieproxy v0.0.12
#25 155.9 go: downloading github.com/minio/colorjson v1.0.8
#25 155.9 go: downloading github.com/minio/filepath v1.0.0
#25 155.9 go: downloading github.com/muesli/reflow v0.3.0
#25 155.9 go: downloading github.com/muesli/termenv v0.16.0
#25 156.0 go: downloading github.com/olekukonko/tablewriter v0.0.5
#25 157.0 go: downloading github.com/posener/complete v1.2.3
#25 157.0 go: downloading github.com/tidwall/gjson v1.18.0
#25 157.0 go: downloading github.com/vbauerster/mpb/v8 v8.9.3
#25 157.1 go: downloading golang.org/x/text v0.24.0
#25 157.4 go: downloading github.com/josharian/intern v1.0.0
#25 157.6 go: downloading go.opentelemetry.io/auto/sdk v1.1.0
#25 157.6 go: downloading github.com/go-logr/stdr v1.2.2
#25 157.6 go: downloading github.com/envoyproxy/go-control-plane/envoy v1.32.4
#25 159.2 go: downloading github.com/google/s2a-go v0.1.9
#25 161.0 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.3.5
#25 161.1 go: downloading github.com/prometheus/prometheus v0.303.0
#25 162.0 go: downloading github.com/felixge/httpsnoop v1.0.4
#25 164.4 go: downloading github.com/hashicorp/errwrap v1.1.0
#25 167.4 go: downloading github.com/jcmturner/dnsutils/v2 v2.0.0
#25 169.4 go: downloading github.com/hashicorp/go-uuid v1.0.3
#25 171.9 go: downloading github.com/apache/thrift v0.21.0
#25 171.9 go: downloading github.com/lestrrat-go/httpcc v1.0.1
#25 171.9 go: downloading github.com/charmbracelet/x/ansi v0.8.0
#25 172.7 go: downloading github.com/charmbracelet/x/term v0.2.1
#25 172.7 go: downloading github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6
#25 176.9 go: downloading github.com/muesli/cancelreader v0.2.2
#25 176.9 go: downloading github.com/juju/ratelimit v1.0.2
#25 176.9 go: downloading github.com/aymanbagabas/go-osc52/v2 v2.0.1
#25 177.0 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0
#25 180.9 go: downloading github.com/cncf/xds/go v0.0.0-20250121191232-2f005788dc42
#25 181.0 go: downloading github.com/tidwall/match v1.1.1
#25 182.2 go: downloading github.com/tidwall/pretty v1.2.1
#25 182.2 go: downloading github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
#25 182.2 go: downloading github.com/VividCortex/ewma v1.2.0
#25 182.3 go: downloading github.com/jcmturner/rpc/v2 v2.0.3
#25 184.7 go: downloading github.com/envoyproxy/protoc-gen-validate v1.2.1
#25 184.7 go: downloading github.com/jcmturner/aescts/v2 v2.0.0
#25 184.7 go: downloading cel.dev/expr v0.22.0
#25 DONE 256.1s
#30 [s3 stage-1 3/3] COPY --from=build /minio /usr/local/bin/minio
#30 DONE 0.2s
#31 [s3] exporting to image
#31 exporting layers
#31 exporting layers 4.3s done
#31 exporting manifest sha256:d50e9c88097122a227139120be4c032e21725a22317154dd69648b53dc84dc3f 0.0s done
#31 exporting config sha256:c00e48fb21003e068aace42f534d64659fa837adab23e4674cc94bc79e3b1f97 0.0s done
#31 exporting attestation manifest sha256:0481668ae887bdd025097902023c7d1cc45060c1cec382e07b282205c4276fe7 0.0s done
#31 exporting manifest list sha256:b0483eceaa301523e808ae4f7572d4350f0695344334646c23c5f7bef07a718e 0.0s done
#31 naming to docker.io/library/mtusi-minio:RELEASE.2025-04-22T22-12-26Z done
#31 unpacking to docker.io/library/mtusi-minio:RELEASE.2025-04-22T22-12-26Z
#31 unpacking to docker.io/library/mtusi-minio:RELEASE.2025-04-22T22-12-26Z 0.7s done
#31 DONE 5.2s
#32 [s3] resolving provenance for metadata file
#32 DONE 0.0s
Image mtusi-messenger:lab4 Built
Image mtusi-minio:RELEASE.2025-04-22T22-12-26Z Built
Image mtusi-messenger:lab4 Built
Image mtusi-messenger:lab4 Built
Image mtusi-messenger:lab4 Built
Network mtusi-lab4_default Creating
Volume mtusi-lab4_pgdata Creating
Volume mtusi-lab4_s3data Creating
Volume mtusi-lab4_s3data Creating
Volume mtusi-lab4_pgdata Creating
Network mtusi-lab4_default Creating
Volume mtusi-lab4_s3data Created
Volume mtusi-lab4_s3data Created
Volume mtusi-lab4_pgdata Created
Volume mtusi-lab4_pgdata Created
Network mtusi-lab4_default Created
Network mtusi-lab4_default Created
Container mtusi-lab4-db-1 Creating
Container mtusi-lab4-s3-1 Creating
Container mtusi-lab4-db-1 Created
Container mtusi-lab4-migrate-1 Creating
Container mtusi-lab4-s3-1 Created
Container mtusi-lab4-s3-init-1 Creating
Container mtusi-lab4-s3-init-1 Created
Container mtusi-lab4-migrate-1 Created
Container mtusi-lab4-api1-1 Creating
Container mtusi-lab4-api2-1 Creating
Container mtusi-lab4-api2-1 Created
Container mtusi-lab4-api1-1 Created
Container mtusi-lab4-lb-1 Creating
Container mtusi-lab4-lb-1 Created
Container mtusi-lab4-db-1 Starting
Container mtusi-lab4-s3-1 Starting
Container mtusi-lab4-db-1 Started
Container mtusi-lab4-db-1 Waiting
Container mtusi-lab4-s3-1 Started
Container mtusi-lab4-s3-1 Waiting
Container mtusi-lab4-s3-1 Healthy
Container mtusi-lab4-s3-init-1 Starting
Container mtusi-lab4-s3-init-1 Started
Container mtusi-lab4-db-1 Healthy
Container mtusi-lab4-migrate-1 Starting
Container mtusi-lab4-migrate-1 Started
Container mtusi-lab4-migrate-1 Waiting
Container mtusi-lab4-s3-init-1 Waiting
Container mtusi-lab4-migrate-1 Waiting
Container mtusi-lab4-s3-init-1 Waiting
Container mtusi-lab4-migrate-1 Exited
Container mtusi-lab4-s3-init-1 Exited
Container mtusi-lab4-api2-1 Starting
Container mtusi-lab4-s3-init-1 Exited
Container mtusi-lab4-migrate-1 Exited
Container mtusi-lab4-api1-1 Starting
Container mtusi-lab4-api2-1 Started
Container mtusi-lab4-api1-1 Started
Container mtusi-lab4-api1-1 Waiting
Container mtusi-lab4-api2-1 Waiting
Container mtusi-lab4-api2-1 Healthy
Container mtusi-lab4-api1-1 Healthy
Container mtusi-lab4-lb-1 Starting
Container mtusi-lab4-lb-1 Started
Container mtusi-lab4-api2-1 Waiting
Container mtusi-lab4-lb-1 Waiting
Container mtusi-lab4-db-1 Waiting
Container mtusi-lab4-migrate-1 Waiting
Container mtusi-lab4-s3-1 Waiting
Container mtusi-lab4-s3-init-1 Waiting
Container mtusi-lab4-api1-1 Waiting
Container mtusi-lab4-migrate-1 Exited
Container mtusi-lab4-s3-init-1 Exited
Container mtusi-lab4-api1-1 Healthy
Container mtusi-lab4-lb-1 Healthy
Container mtusi-lab4-api2-1 Healthy
Container mtusi-lab4-db-1 Healthy
Container mtusi-lab4-s3-1 Healthy
+1
View File
@@ -0,0 +1 @@
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
+8
View File
@@ -0,0 +1,8 @@
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi-lab4-api1-1 mtusi-messenger:lab4 "docker-entrypoint.s…" api1 5 minutes ago Up 2 seconds (healthy) 8080/tcp
mtusi-lab4-api2-1 mtusi-messenger:lab4 "docker-entrypoint.s…" api2 5 minutes ago Up 2 minutes (healthy) 8080/tcp
mtusi-lab4-db-1 postgres:17.6-alpine "docker-entrypoint.s…" db 5 minutes ago Up 2 minutes (healthy) 5432/tcp
mtusi-lab4-lb-1 haproxy:3.0.12-alpine "docker-entrypoint.s…" lb 5 minutes ago Up 2 minutes 127.0.0.1:8084->8080/tcp
mtusi-lab4-migrate-1 mtusi-messenger:lab4 "docker-entrypoint.s…" migrate 5 minutes ago Exited (0) 5 minutes ago
mtusi-lab4-s3-1 mtusi-minio:RELEASE.2025-04-22T22-12-26Z "minio server /data …" s3 5 minutes ago Up 2 minutes (healthy)
mtusi-lab4-s3-init-1 mtusi-messenger:lab4 "docker-entrypoint.s…" s3-init 5 minutes ago Exited (0) 5 minutes ago
+1
View File
@@ -0,0 +1 @@
PASS: 17 original contract/test/task files byte-identical to supplied archive.
+1
View File
@@ -0,0 +1 @@
Docker=29.9.2; CPUs=12; MemoryBytes=8259383296; OS=Docker Desktop
+3
View File
@@ -0,0 +1,3 @@
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 6, "failed_attempts": 1, "first_recovery_seconds": 5.156}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
+2
View File
@@ -0,0 +1,2 @@
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
+94
View File
@@ -0,0 +1,94 @@
{
"before": [
{
"service": "api1",
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
"running": true,
"started_at": "2026-11-13T09:45:29.802007824Z",
"finished_at": "2026-11-13T09:45:23.458360231Z",
"volumes": []
},
{
"service": "api2",
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
"running": true,
"started_at": "2026-11-13T09:45:29.976703587Z",
"finished_at": "2026-11-13T09:45:23.459942874Z",
"volumes": []
},
{
"service": "db",
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
"running": true,
"started_at": "2026-11-13T09:45:24.574681933Z",
"finished_at": "2026-11-13T09:45:23.879037942Z",
"volumes": [
"mtusi-lab4_pgdata"
]
},
{
"service": "lb",
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
"running": true,
"started_at": "2026-11-13T09:45:33.091747442Z",
"finished_at": "2026-11-13T09:45:23.177520781Z",
"volumes": []
},
{
"service": "s3",
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
"running": true,
"started_at": "2026-11-13T09:45:40.582059708Z",
"finished_at": "2026-11-13T09:45:34.475313953Z",
"volumes": [
"mtusi-lab4_s3data"
]
}
],
"after": [
{
"service": "api1",
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
"running": true,
"started_at": "2026-11-13T09:47:44.323950141Z",
"finished_at": "2026-11-13T09:47:37.999368896Z",
"volumes": []
},
{
"service": "api2",
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
"running": true,
"started_at": "2026-11-13T09:47:44.186050153Z",
"finished_at": "2026-11-13T09:47:37.999033921Z",
"volumes": []
},
{
"service": "db",
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
"running": true,
"started_at": "2026-11-13T09:47:39.127938865Z",
"finished_at": "2026-11-13T09:47:38.306703924Z",
"volumes": [
"mtusi-lab4_pgdata"
]
},
{
"service": "lb",
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
"running": true,
"started_at": "2026-11-13T09:47:47.418593695Z",
"finished_at": "2026-11-13T09:47:37.689985726Z",
"volumes": []
},
{
"service": "s3",
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
"running": true,
"started_at": "2026-11-13T09:47:38.986060426Z",
"finished_at": "2026-11-13T09:47:38.274698815Z",
"volumes": [
"mtusi-lab4_s3data"
]
}
]
}
+1
View File
@@ -0,0 +1 @@
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
+96
View File
@@ -0,0 +1,96 @@
{
"target_instance": "a003f158-ce85-43e5-b7b2-ce885eb0fc8a",
"selected_service": "api1",
"before": [
{
"service": "api1",
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
"running": true,
"started_at": "2026-11-13T09:47:44.323950141Z",
"finished_at": "2026-11-13T09:47:37.999368896Z",
"volumes": []
},
{
"service": "api2",
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
"running": true,
"started_at": "2026-11-13T09:47:44.186050153Z",
"finished_at": "2026-11-13T09:47:37.999033921Z",
"volumes": []
},
{
"service": "db",
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
"running": true,
"started_at": "2026-11-13T09:47:39.127938865Z",
"finished_at": "2026-11-13T09:47:38.306703924Z",
"volumes": [
"mtusi-lab4_pgdata"
]
},
{
"service": "lb",
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
"running": true,
"started_at": "2026-11-13T09:47:47.418593695Z",
"finished_at": "2026-11-13T09:47:37.689985726Z",
"volumes": []
},
{
"service": "s3",
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
"running": true,
"started_at": "2026-11-13T09:47:38.986060426Z",
"finished_at": "2026-11-13T09:47:38.274698815Z",
"volumes": [
"mtusi-lab4_s3data"
]
}
],
"after": [
{
"service": "api1",
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
"running": false,
"started_at": "2026-11-13T09:47:44.323950141Z",
"finished_at": "2026-11-13T09:49:43.202703028Z",
"volumes": []
},
{
"service": "api2",
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
"running": true,
"started_at": "2026-11-13T09:47:44.186050153Z",
"finished_at": "2026-11-13T09:47:37.999033921Z",
"volumes": []
},
{
"service": "db",
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
"running": true,
"started_at": "2026-11-13T09:47:39.127938865Z",
"finished_at": "2026-11-13T09:47:38.306703924Z",
"volumes": [
"mtusi-lab4_pgdata"
]
},
{
"service": "lb",
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
"running": true,
"started_at": "2026-11-13T09:47:47.418593695Z",
"finished_at": "2026-11-13T09:47:37.689985726Z",
"volumes": []
},
{
"service": "s3",
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
"running": true,
"started_at": "2026-11-13T09:47:38.986060426Z",
"finished_at": "2026-11-13T09:47:38.274698815Z",
"volumes": [
"mtusi-lab4_s3data"
]
}
]
}
+96
View File
@@ -0,0 +1,96 @@
{
"target_instance": "b770ec13-25b7-435b-a024-7d805a1c8798",
"selected_service": "api2",
"before": [
{
"service": "api1",
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
"running": true,
"started_at": "2026-11-13T09:44:53.775499857Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": []
},
{
"service": "api2",
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
"running": true,
"started_at": "2026-11-13T09:44:53.604840647Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": []
},
{
"service": "db",
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
"running": true,
"started_at": "2026-11-13T09:44:48.260190707Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": [
"mtusi-lab4_pgdata"
]
},
{
"service": "lb",
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
"running": true,
"started_at": "2026-11-13T09:44:56.441675642Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": []
},
{
"service": "s3",
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
"running": true,
"started_at": "2026-11-13T09:44:48.465999336Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": [
"mtusi-lab4_s3data"
]
}
],
"after": [
{
"service": "api1",
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
"running": true,
"started_at": "2026-11-13T09:44:53.775499857Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": []
},
{
"service": "api2",
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
"running": false,
"started_at": "2026-11-13T09:44:53.604840647Z",
"finished_at": "2026-11-13T09:45:11.765422268Z",
"volumes": []
},
{
"service": "db",
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
"running": true,
"started_at": "2026-11-13T09:44:48.260190707Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": [
"mtusi-lab4_pgdata"
]
},
{
"service": "lb",
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
"running": true,
"started_at": "2026-11-13T09:44:56.441675642Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": []
},
{
"service": "s3",
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
"running": true,
"started_at": "2026-11-13T09:44:48.465999336Z",
"finished_at": "0001-01-01T00:00:00Z",
"volumes": [
"mtusi-lab4_s3data"
]
}
]
}
+6
View File
@@ -0,0 +1,6 @@
PASS: PNG/JPEG exact bytes and SHA-256; private bucket 403; member access; invalid filename/MIME/corrupt file; >10 MiB including chunked and >40M pixels; attachment scope; 12 concurrent sends make 1 message.
{"scenario":"upload-stop-uploader-download","recovery_ms":4054,"same_bytes":true}
{"scenario":"upload-stop-uploader-download","recovery_ms":2199,"same_bytes":true}
PASS: full restart preserves metadata, original bytes, session and idempotent attachment message.
PASS: stopped S3 makes API readiness 503 while liveness remains 200.
PASS: S3 recovery restores download without data loss.
+11
View File
@@ -0,0 +1,11 @@
FROM golang:1.24.3-alpine AS build
RUN apk add --no-cache git || apk add --no-cache git || apk add --no-cache git
RUN git clone --depth 1 --branch RELEASE.2025-04-22T22-12-26Z https://github.com/minio/minio.git /src
WORKDIR /src
RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build \
for attempt in 1 2 3 4 5; do CGO_ENABLED=0 go build -trimpath -o /minio . && exit 0; done; exit 1
FROM alpine:3.22.1
RUN apk add --no-cache ca-certificates curl
COPY --from=build /minio /usr/local/bin/minio
ENTRYPOINT ["minio"]
+36
View File
@@ -0,0 +1,36 @@
global
log stdout format raw local0
defaults
mode http
log global
timeout connect 1s
timeout client 15s
timeout server 15s
timeout check 1s
retries 0
resolvers docker
nameserver dns 127.0.0.11:53
resolve_retries 3
timeout resolve 1s
timeout retry 1s
hold valid 1s
hold nx 1s
hold timeout 1s
hold refused 1s
hold other 1s
hold obsolete 1s
frontend public
bind :8080
default_backend messenger
backend messenger
balance roundrobin
option httpchk
http-check send meth GET uri /health/ready ver HTTP/1.1 hdr Host localhost
http-check expect status 200
default-server check inter 1s fall 2 rise 1 resolvers docker init-addr last,libc,none
server api1 api1:8080
server api2 api2:8080
+27
View File
@@ -0,0 +1,27 @@
# Хранение данных lab4
PostgreSQL хранит пользователей, хеши паролей, хеши сессий, чаты, участников,
сообщения, результаты идемпотентных запросов и метаданные вложений.
Миграции 001–002 перенесены из lab3; миграция 003 добавляет вложения.
`attachments`: UUID, чат, владелец, отображаемое имя, MIME, размер, SHA-256,
случайный object_key и время создания/изменения. Object key наружу не выдаётся.
`message_attachments`: связь сообщения и вложения, позиция в списке.
Байты оригинала находятся только в приватном bucket `messenger-attachments`.
API использует ограниченный буфер в RAM, не пишет файл на диск.
Порядок upload: авторизация → проверка имени/размера/MIME → полное декодирование
с лимитом пикселей → S3 PUT → INSERT метаданных → 201 ready.
Ошибка INSERT вызывает попытку удалить объект. Общей транзакции S3/PostgreSQL нет:
при аварии между операциями возможен лишний объект. Автоматический reconciler
не реализован (целевая оценка 3).
Отправка сообщения и запись результата идемпотентности выполняются одной
транзакцией PostgreSQL, с общей блокировкой по пользователю, чату и ключу.
Все attachment_ids проверяются на принадлежность чату; порядок сохраняется.
Отсутствующий attachment_ids равнозначен пустому списку.
У API нет собственного volume. PostgreSQL использует pgdata, MinIO — s3data.
Оба API читают одни и те же данные. S3 и консоль доступны только внутри Compose.
Bootstrap создаёт bucket и удаляет анонимную policy. Для учебного уровня 3
API использует общие административные S3 credentials из локальных secret-файлов.
+35
View File
@@ -0,0 +1,35 @@
CREATE TABLE users (
id uuid PRIMARY KEY,
username text NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
display_name text NOT NULL CHECK (char_length(display_name) BETWEEN 1 AND 100),
password_hash text NOT NULL,
created_at timestamptz(3) NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chats (
id uuid PRIMARY KEY,
title text NOT NULL CHECK (char_length(title) BETWEEN 1 AND 100),
created_at timestamptz(3) NOT NULL DEFAULT clock_timestamp()
);
CREATE TABLE chat_members (
chat_id uuid NOT NULL REFERENCES chats(id),
user_id uuid NOT NULL REFERENCES users(id),
PRIMARY KEY (chat_id, user_id)
);
CREATE TABLE messages (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL,
sender_id uuid NOT NULL,
text text NOT NULL CHECK (char_length(text) BETWEEN 1 AND 4000),
created_at timestamptz(3) NOT NULL DEFAULT clock_timestamp(),
FOREIGN KEY (chat_id, sender_id) REFERENCES chat_members(chat_id, user_id)
);
CREATE TABLE sessions (
token_hash text PRIMARY KEY,
user_id uuid NOT NULL REFERENCES users(id),
expires_at timestamptz(3) NOT NULL
);
CREATE TABLE app_settings (key text PRIMARY KEY, value text NOT NULL);
CREATE INDEX messages_history ON messages(chat_id, created_at, id);
CREATE INDEX chats_order ON chats(created_at, id);
CREATE INDEX member_chats ON chat_members(user_id, chat_id);
CREATE INDEX sessions_expiry ON sessions(expires_at);
+11
View File
@@ -0,0 +1,11 @@
CREATE TABLE message_requests (
user_id uuid NOT NULL REFERENCES users(id),
chat_id uuid NOT NULL REFERENCES chats(id),
key text NOT NULL,
text text NOT NULL,
message_id uuid NOT NULL REFERENCES messages(id),
response jsonb NOT NULL,
expires_at timestamptz NOT NULL,
PRIMARY KEY (user_id, chat_id, key)
);
CREATE INDEX message_requests_expiry ON message_requests(expires_at);
+25
View File
@@ -0,0 +1,25 @@
CREATE TABLE attachments (
id uuid PRIMARY KEY,
chat_id uuid NOT NULL REFERENCES chats(id),
owner_id uuid NOT NULL REFERENCES users(id),
filename text NOT NULL CHECK (char_length(filename) BETWEEN 1 AND 255),
content_type text NOT NULL CHECK (content_type IN ('image/png','image/jpeg')),
size_bytes integer NOT NULL CHECK (size_bytes BETWEEN 1 AND 10485760),
sha256 text NOT NULL,
object_key text NOT NULL UNIQUE,
created_at timestamptz(3) NOT NULL DEFAULT clock_timestamp(),
updated_at timestamptz(3) NOT NULL DEFAULT clock_timestamp(),
UNIQUE (id,chat_id)
);
CREATE INDEX attachments_chat ON attachments(chat_id);
ALTER TABLE messages DROP CONSTRAINT messages_text_check;
ALTER TABLE messages ADD CONSTRAINT messages_text_check CHECK (char_length(text) BETWEEN 0 AND 4000);
CREATE TABLE message_attachments (
message_id uuid NOT NULL REFERENCES messages(id),
attachment_id uuid NOT NULL REFERENCES attachments(id),
position integer NOT NULL,
PRIMARY KEY (message_id,attachment_id),
UNIQUE (message_id,position)
);
ALTER TABLE message_requests ADD COLUMN attachment_ids uuid[] NOT NULL DEFAULT '{}';
UPDATE message_requests SET response = response || '{"attachment_ids":[]}'::jsonb;
+1449
View File
File diff suppressed because it is too large Load Diff
+14
View File
@@ -0,0 +1,14 @@
{
"name": "mtusi-lab4-messenger",
"version": "1.0.0",
"private": true,
"scripts": {
"start": "node src/server.js",
"migrate": "node src/migrate.js"
},
"dependencies": {
"pg": "8.16.3",
"minio": "8.0.5",
"sharp": "0.34.3"
}
}
+75
View File
@@ -0,0 +1,75 @@
"""Explicit lab4-only lifecycle hooks. Never delete volumes or print credentials."""
import json
import os
from pathlib import Path
import subprocess
import sys
ROOT = Path(__file__).resolve().parents[1]
os.chdir(ROOT)
def docker(*args):
return subprocess.check_output(['docker', *args], text=True).strip()
def compose(*args):
return docker('compose', *args)
def snapshot():
result = []
containers = compose('ps', '-a', '-q').splitlines()
if not containers:
return result
# One bulk inspect keeps the stop hook well below its 10-second budget.
# Only these selected fields are written; Config/Env are never logged.
for container in json.loads(docker('inspect', *containers)):
service = container['Config']['Labels']['com.docker.compose.service']
if service not in ('lb', 'api1', 'api2', 'db', 's3'):
continue
state, mounts = container['State'], container['Mounts']
result.append({'service': service, 'container': container['Id'], 'running': state['Running'],
'started_at': state['StartedAt'], 'finished_at': state['FinishedAt'],
'volumes': [m['Name'] for m in mounts if m['Type'] == 'volume']})
return result
def save(name, value):
(ROOT / 'evidence').mkdir(exist_ok=True)
(ROOT / 'evidence' / name).write_text(json.dumps(value, indent=2) + '\n', encoding='utf-8')
def main():
action = sys.argv[1]
if action == 'restart':
before = snapshot()
compose('stop', 'lb', 'api1', 'api2', 'db', 's3')
compose('up', '-d', '--wait', '--wait-timeout', '30', 'db', 's3')
compose('run', '--rm', '--no-deps', 's3-init')
compose('run', '--rm', '--no-deps', 'migrate')
compose('up', '-d', '--no-deps', '--wait', '--wait-timeout', '30', 'api1', 'api2')
compose('up', '-d', '--no-deps', 'lb')
save('restart-scope.json', {'before': before, 'after': snapshot()})
elif action == 'stop-one':
target = os.environ.get('TARGET_INSTANCE_ID')
if not target:
raise RuntimeError('TARGET_INSTANCE_ID required')
before = snapshot()
selected = None
for service in ('api1', 'api2'):
code = "fetch('http://127.0.0.1:8080/health/live').then(r=>console.log(r.headers.get('x-instance-id'))).catch(()=>process.exit(1))"
instance = compose('exec', '-T', service, 'node', '-e', code)
if instance == target:
selected = service
break
if selected is None:
raise RuntimeError('Target is not a running API in this lab')
compose('stop', '-t', '2', selected)
after = snapshot()
assert not next(s for s in after if s['service'] == selected)['running']
assert all(s['running'] for s in after if s['service'] != selected)
save('stop-' + selected + '.json', {'target_instance': target, 'selected_service': selected, 'before': before, 'after': after})
elif action == 'restore':
compose('up', '-d', '--no-deps', '--wait', '--wait-timeout', '30', 'api1', 'api2')
else:
raise RuntimeError('Unknown action')
print('PASS: ' + action + ' completed for mtusi-lab4')
if __name__ == '__main__':
main()
+7
View File
@@ -0,0 +1,7 @@
@echo off
if defined LAB_PYTHON (
"%LAB_PYTHON%" -X utf8 "%~dp0control.py" restart
) else (
py -X utf8 "%~dp0control.py" restart
)
exit /b %errorlevel%
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
exec python3 scripts/control.py restart
+26
View File
@@ -0,0 +1,26 @@
"""Generate local credentials once; never print or rotate existing secrets."""
from pathlib import Path
import secrets
import shutil
root = Path(__file__).resolve().parents[1]
directory = root / 'secrets'
directory.mkdir(exist_ok=True)
password_file = directory / 'postgres_password'
url_file = directory / 'database_url'
if password_file.exists() != url_file.exists():
raise SystemExit('Incomplete secrets: restore the missing file; do not rotate an existing database password.')
if not password_file.exists():
password = secrets.token_hex(32)
password_file.write_text(password + '\n', encoding='utf-8')
url_file.write_text('postgresql://messenger:' + password + '@db:5432/messenger\n', encoding='utf-8')
password_file.chmod(0o600)
url_file.chmod(0o600)
for filename in ('s3_access_key_id', 's3_secret_access_key'):
destination = directory / filename
if not destination.exists():
destination.write_text(secrets.token_hex(10 if filename == 's3_access_key_id' else 24) + '\n', encoding='utf-8')
destination.chmod(0o600)
if not (root / '.env').exists():
shutil.copyfile(root / '.env.example', root / '.env')
print('Local .env and secrets ready; values are not printed. Existing credentials preserved.')
+7
View File
@@ -0,0 +1,7 @@
@echo off
if defined LAB_PYTHON (
"%LAB_PYTHON%" -X utf8 "%~dp0control.py" stop-one
) else (
py -X utf8 "%~dp0control.py" stop-one
)
exit /b %errorlevel%
+4
View File
@@ -0,0 +1,4 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
exec python3 scripts/control.py stop-one
+97
View File
@@ -0,0 +1,97 @@
'use strict';
const sharp = require('sharp');
const { randomUUID, createHash } = require('node:crypto');
const { pool } = require('./db');
const { client, bucket, unavailable } = require('./storage');
const MAX_BYTES = 10485760, MAX_PIXELS = 40000000;
sharp.cache(false);
function fail(status, code, message) { throw Object.assign(new Error(message), { status, code }); }
function visible(row) {
const { object_key, ...data } = row;
return { ...data, status: 'ready', variants: [], error_code: null };
}
function readLimited(req) {
return new Promise((resolve, reject) => {
let size = 0, failed = false;
const chunks = [];
const tooLarge = () => {
failed = true;
reject(Object.assign(new Error('File exceeds 10 MiB'), { status: 413, code: 'payload_too_large' }));
};
if (Number(req.headers['content-length'] || 0) > MAX_BYTES) { req.resume(); tooLarge(); return; }
req.on('data', chunk => {
if (failed) return;
size += chunk.length;
if (size > MAX_BYTES) { chunks.length = 0; tooLarge(); }
else chunks.push(chunk);
});
req.on('end', () => { if (!failed) resolve(Buffer.concat(chunks)); });
req.on('error', () => reject(Object.assign(new Error('Upload interrupted'), { status: 400, code: 'invalid_request' })));
});
}
async function validateImage(bytes, contentType) {
// Reject other formats before asking the image decoder to process them.
const png = bytes.length >= 8 && bytes.subarray(0, 8).equals(Buffer.from([137,80,78,71,13,10,26,10]));
const jpeg = bytes.length >= 3 && bytes[0] === 255 && bytes[1] === 216 && bytes[2] === 255;
if ((!png && !jpeg) || (png ? 'image/png' : 'image/jpeg') !== contentType)
fail(415, 'unsupported_media_type', 'Expected a matching PNG or JPEG');
try {
const decoder = sharp(bytes, { limitInputPixels: MAX_PIXELS, failOn: 'warning' });
const info = await decoder.metadata();
if (!info.width || !info.height || info.width * info.height > MAX_PIXELS)
fail(413, 'payload_too_large', 'Image exceeds 40 million pixels');
if (!['png', 'jpeg'].includes(info.format)) fail(415, 'unsupported_media_type', 'Unsupported image');
// Full decoding detects truncated/corrupt payloads; the raw result is discarded.
// Pixel limit is enforced before allocation. Original bytes are never transformed.
await decoder.raw().toBuffer();
} catch (error) {
if (error.status) throw error;
if (/pixel limit/i.test(error.message)) fail(413, 'payload_too_large', 'Image exceeds 40 million pixels');
fail(415, 'unsupported_media_type', 'Corrupt image');
}
}
async function upload(req, query, chat, user) {
const filename = query.get('filename');
if (query.getAll('filename').length !== 1 || !filename || !filename.isWellFormed() ||
/^[\p{White_Space}\uFEFF]*$/u.test(filename) || [...filename].length > 255 || /[\/\\\p{Cc}]/u.test(filename))
fail(400, 'invalid_request', 'Invalid filename');
const contentType = req.headers['content-type']?.split(';')[0].trim().toLowerCase();
if (!['image/png', 'image/jpeg'].includes(contentType)) fail(415, 'unsupported_media_type', 'Expected PNG or JPEG');
const bytes = await readLimited(req);
await validateImage(bytes, contentType);
const id = randomUUID(), objectKey = 'originals/' + randomUUID();
const checksum = createHash('sha256').update(bytes).digest('hex');
try { await client.putObject(bucket, objectKey, bytes, bytes.length, { 'Content-Type': contentType }); }
catch { throw unavailable(); }
try {
const result = await pool.query(`INSERT INTO attachments(id,chat_id,owner_id,filename,content_type,size_bytes,sha256,object_key)
VALUES($1,$2,$3,$4,$5,$6,$7,$8) RETURNING *`, [id, chat.id, user.id, filename, contentType, bytes.length, checksum, objectKey]);
return visible(result.rows[0]);
} catch (error) {
// Best-effort compensation. A process crash between stores still needs a reconciler (not grade 3).
await client.removeObject(bucket, objectKey).catch(() => console.error('Object compensation failed'));
throw error;
}
}
async function accessible(id, user) {
const result = await pool.query(`SELECT a.* FROM attachments a WHERE a.id=$1 AND EXISTS
(SELECT 1 FROM chat_members WHERE chat_id=a.chat_id AND user_id=$2)`, [id, user.id]);
if (!result.rowCount) fail(404, 'not_found', 'Object not found');
return result.rows[0];
}
async function download(row) {
try {
const stream = await client.getObject(bucket, row.object_key);
const chunks = [];
let size = 0;
for await (const chunk of stream) {
size += chunk.length;
if (size > MAX_BYTES) { stream.destroy(); throw new Error(); }
chunks.push(chunk);
}
const bytes = Buffer.concat(chunks);
if (bytes.length !== row.size_bytes || createHash('sha256').update(bytes).digest('hex') !== row.sha256) throw new Error();
return bytes;
} catch { throw unavailable(); }
}
module.exports = { upload, accessible, visible, download, validateImage };
+22
View File
@@ -0,0 +1,22 @@
'use strict';
const { readFileSync } = require('node:fs');
const { Pool } = require('pg');
const pool = new Pool({
connectionString: readFileSync(process.env.DATABASE_URL_FILE || '/run/secrets/database_url', 'utf8').trim(),
max: 10, connectionTimeoutMillis: 3000, idleTimeoutMillis: 10000,
statement_timeout: 5000,
});
pool.on('error', () => console.error('Database connection unavailable'));
async function transaction(work) {
const client = await pool.connect();
try {
await client.query('BEGIN');
const result = await work(client);
await client.query('COMMIT');
return result;
} catch (error) {
await client.query('ROLLBACK').catch(() => {});
throw error;
} finally { client.release(); }
}
module.exports = { pool, transaction };
+36
View File
@@ -0,0 +1,36 @@
'use strict';
const { randomUUID } = require('node:crypto');
const { transaction } = require('./db');
const retention = Number(process.env.IDEMPOTENCY_TTL_SECONDS || 86400);
if (!Number.isSafeInteger(retention) || retention < 86400) throw new Error('Idempotency retention must be at least 24 hours');
async function sendMessage(userId, chatId, key, text, attachmentIds = []) {
return transaction(async client => {
const attached = await client.query('SELECT id FROM attachments WHERE chat_id=$1 AND id=ANY($2::uuid[])', [chatId, attachmentIds]);
if (attached.rowCount !== attachmentIds.length)
throw Object.assign(new Error('Object not found'), { status: 404, code: 'not_found' });
// Shared PostgreSQL lock serializes this operation across all API replicas.
// Hash collisions only serialize unrelated keys; the actual lookup uses the full scope.
const scope = JSON.stringify([userId, chatId, key]);
await client.query('SELECT pg_advisory_xact_lock(hashtextextended($1, 0))', [scope]);
await client.query(`DELETE FROM message_requests WHERE user_id=$1 AND chat_id=$2 AND key=$3
AND expires_at <= clock_timestamp()`, [userId, chatId, key]);
const prior = await client.query('SELECT text,attachment_ids,response FROM message_requests WHERE user_id=$1 AND chat_id=$2 AND key=$3', [userId, chatId, key]);
if (prior.rowCount) {
if (prior.rows[0].text !== text || JSON.stringify(prior.rows[0].attachment_ids) !== JSON.stringify(attachmentIds))
throw Object.assign(new Error('Key already used for another message'), { status: 409, code: 'conflict' });
return prior.rows[0].response;
}
const message = (await client.query(`INSERT INTO messages(id,chat_id,sender_id,text)
VALUES($1,$2,$3,$4) RETURNING *`, [randomUUID(), chatId, userId, text])).rows[0];
for (let i = 0; i < attachmentIds.length; i++)
await client.query('INSERT INTO message_attachments VALUES($1,$2,$3)', [message.id, attachmentIds[i], i]);
message.attachment_ids = attachmentIds;
await client.query(`INSERT INTO message_requests(user_id,chat_id,key,text,message_id,response,expires_at,attachment_ids)
VALUES($1,$2,$3,$4,$5,$6::jsonb,clock_timestamp()+$7*interval '1 second',$8::uuid[])`,
[userId, chatId, key, text, message.id, JSON.stringify(message), retention, attachmentIds]);
// The HTTP handler returns 201 only after transaction() completes COMMIT.
return message;
});
}
module.exports = { sendMessage };
+37
View File
@@ -0,0 +1,37 @@
'use strict';
const { readFileSync, readdirSync } = require('node:fs');
const path = require('node:path');
const { createHash, randomBytes } = require('node:crypto');
const { pool } = require('./db');
async function migrate() {
const client = await pool.connect();
try {
await client.query('SELECT pg_advisory_lock(202602)');
await client.query(`CREATE TABLE IF NOT EXISTS schema_migrations
(version text PRIMARY KEY, checksum text NOT NULL)`);
const dir = path.join(__dirname, '..', 'migrations');
for (const version of readdirSync(dir).filter(f => /^\d+.*\.sql$/.test(f)).sort()) {
const sql = readFileSync(path.join(dir, version), 'utf8');
const checksum = createHash('sha256').update(sql).digest('hex');
const existing = await client.query('SELECT checksum FROM schema_migrations WHERE version=$1', [version]);
if (existing.rowCount) {
if (existing.rows[0].checksum !== checksum) throw new Error('Migration checksum mismatch');
continue;
}
await client.query('BEGIN');
try {
await client.query(sql);
await client.query('INSERT INTO schema_migrations VALUES ($1,$2)', [version, checksum]);
await client.query('COMMIT');
} catch (error) { await client.query('ROLLBACK'); throw error; }
console.log('Applied migration ' + version);
}
await client.query(`INSERT INTO app_settings VALUES ('cursor_secret',$1) ON CONFLICT DO NOTHING`, [randomBytes(32).toString('hex')]);
} finally {
await client.query('SELECT pg_advisory_unlock(202602)').catch(() => {});
client.release();
}
}
module.exports = { migrate };
if (require.main === module) migrate().then(() => console.log('Migrations up to date'))
.catch(() => { console.error('Migration failed'); process.exitCode = 1; }).finally(() => pool.end());
+256
View File
@@ -0,0 +1,256 @@
'use strict';
const http = require('node:http');
const { randomUUID, randomBytes, createHash, createHmac, scrypt, timingSafeEqual } = require('node:crypto');
const { promisify } = require('node:util');
const { pool, transaction } = require('./db');
const { sendMessage } = require('./messages');
const attachments = require('./attachments');
const storage = require('./storage');
const instanceId = randomUUID();
let stopping = false;
const derive = promisify(scrypt);
const kdfOptions = { N: 32768, r: 8, p: 1, maxmem: 64 * 1024 * 1024 };
const ttl = Number(process.env.SESSION_TTL_SECONDS || 3600);
if (!Number.isSafeInteger(ttl) || ttl < 1) throw new Error('Invalid SESSION_TTL_SECONDS');
let cursorSecret;
const userColumns = 'id, username, display_name, created_at';
const chatSelect = `SELECT c.id,c.title,c.created_at,
ARRAY(SELECT user_id FROM chat_members WHERE chat_id=c.id ORDER BY user_id) AS member_ids FROM chats c`;
function fail(status, code, message) { throw Object.assign(new Error(message), { status, code }); }
function invalid(message = 'Invalid request') { fail(400, 'invalid_request', message); }
function unauthorized() { fail(401, 'unauthorized', 'Invalid credentials or session'); }
function uuid(value) {
if (typeof value !== 'string' || value.length !== 36 || !/^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$/i.test(value)) invalid('Invalid UUID');
return value.toLowerCase();
}
function fields(body, names) {
if (!body || typeof body !== 'object' || Array.isArray(body) ||
Object.keys(body).length !== names.length || !names.every(k => Object.hasOwn(body, k))) invalid('Invalid fields');
}
function string(value, max, min = 1) {
if (typeof value !== 'string' || !value.isWellFormed() || /^[\p{White_Space}\uFEFF]*$/u.test(value) ||
[...value].length < min || [...value].length > max || value.includes('\0')) invalid('Invalid string');
return value;
}
function username(value) {
if (typeof value !== 'string' || value.length < 3 || value.length > 32 || /[^a-z0-9_]/.test(value)) invalid('Invalid username');
}
function password(value) {
if (typeof value !== 'string' || !value.isWellFormed() || [...value].length < 12 || [...value].length > 128 ||
/^[\p{White_Space}\uFEFF]*$/u.test(value)) invalid('Invalid password');
}
function digest(token) { return createHash('sha256').update(token).digest('hex'); }
async function hashPassword(value) {
const salt = randomBytes(16).toString('hex');
const hash = await derive(value, salt, 64, kdfOptions);
return `scrypt$32768$8$1$${salt}$${hash.toString('hex')}`;
}
const dummySalt = randomBytes(16).toString('hex');
async function verifyPassword(value, encoded) {
const parts = encoded?.split('$');
const salt = parts ? parts[4] : dummySalt;
const expected = parts ? Buffer.from(parts[5], 'hex') : Buffer.alloc(64);
const actual = await derive(value, salt, 64, kdfOptions);
return Boolean(encoded) && expected.length === actual.length && timingSafeEqual(expected, actual);
}
async function rawBody(req) {
const chunks = [];
let size = 0;
for await (const chunk of req) {
size += chunk.length;
if (size > 1024 * 1024) fail(413, 'payload_too_large', 'Request body too large');
chunks.push(chunk);
}
return Buffer.concat(chunks);
}
async function body(req) {
if (req.headers['content-type']?.split(';')[0].trim().toLowerCase() !== 'application/json')
fail(415, 'unsupported_media_type', 'Expected application/json');
const bytes = await rawBody(req);
try { return JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes)); }
catch { invalid('Invalid JSON'); }
}
async function identity(req) {
const match = /^Bearer ([A-Za-z0-9_-]{43})$/i.exec(req.headers.authorization || '');
if (!match) unauthorized();
const tokenHash = digest(match[1]);
const result = await pool.query(`SELECT u.id,u.username,u.display_name,u.created_at
FROM sessions s JOIN users u ON u.id=s.user_id
WHERE token_hash=$1 AND expires_at > clock_timestamp()`, [tokenHash]);
if (!result.rowCount) unauthorized();
return { user: result.rows[0], tokenHash };
}
async function accessible(id, user) {
const result = await pool.query(chatSelect + ` WHERE c.id=$1 AND EXISTS
(SELECT 1 FROM chat_members WHERE chat_id=c.id AND user_id=$2)`, [uuid(id), user.id]);
if (!result.rowCount) fail(404, 'not_found', 'Object not found');
return result.rows[0];
}
function signature(payload) { return createHmac('sha256', cursorSecret).update(payload).digest('base64url'); }
function pagination(query, scope) {
if (query.getAll('limit').length > 1 || query.getAll('cursor').length > 1) invalid();
const raw = query.get('limit') ?? '50';
if (!/^\d+$/.test(raw) || Number(raw) < 1 || Number(raw) > 100) invalid('Invalid limit');
let after = null;
if (query.has('cursor')) {
try {
const token = query.get('cursor');
if (!/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(token)) throw new Error();
const [payload, mac] = token.split('.');
if (signature(payload) !== mac) throw new Error();
after = JSON.parse(Buffer.from(payload, 'base64url').toString('utf8'));
if (after.scope !== scope || typeof after.created_at !== 'string' || !Number.isFinite(Date.parse(after.created_at))) throw new Error();
uuid(after.id);
} catch { invalid('Invalid cursor'); }
}
return { limit: Number(raw), after, scope };
}
function page(rows, paging) {
const items = rows.slice(0, paging.limit);
let next_cursor = null;
if (rows.length > paging.limit) {
const last = items.at(-1);
const payload = Buffer.from(JSON.stringify({ scope: paging.scope, created_at: last.created_at, id: last.id })).toString('base64url');
next_cursor = payload + '.' + signature(payload);
}
return { items, next_cursor };
}
async function route(req) {
const url = new URL(req.url, 'http://localhost');
const path = url.pathname, method = req.method;
if (method === 'GET' && path === '/health/live') return [200, { status: 'ok' }];
if (stopping) fail(503, 'unavailable', 'Server is stopping');
if (method === 'GET' && path === '/health/ready') {
try { await pool.query('SELECT 1'); }
catch { fail(503, 'unavailable', 'Database unavailable'); }
await storage.ready();
return [200, { status: 'ok' }];
}
if (method === 'POST' && path === '/api/v1/users') {
const data = await body(req);
fields(data, ['username', 'display_name', 'password']);
username(data.username); string(data.display_name, 100); password(data.password);
const hash = await hashPassword(data.password);
try {
const result = await pool.query(`INSERT INTO users(id,username,display_name,password_hash)
VALUES($1,$2,$3,$4) RETURNING ${userColumns}`, [randomUUID(), data.username, data.display_name, hash]);
return [201, result.rows[0]];
} catch (error) {
if (error.code === '23505') fail(409, 'conflict', 'Username already exists');
throw error;
}
}
if (method === 'POST' && path === '/api/v1/auth/sessions') {
const match = /^Basic ([A-Za-z0-9+/]+={0,2})$/i.exec(req.headers.authorization || '');
if (!match) unauthorized();
let name, value;
try {
const bytes = Buffer.from(match[1], 'base64');
if (bytes.toString('base64') !== match[1]) throw new Error();
const decoded = new TextDecoder('utf-8', { fatal: true }).decode(bytes);
const colon = decoded.indexOf(':');
if (colon < 0) throw new Error();
name = decoded.slice(0, colon); value = decoded.slice(colon + 1);
username(name); password(value);
} catch { unauthorized(); }
if ((await rawBody(req)).length) invalid('Login must not have a body');
const result = await pool.query(`SELECT ${userColumns},password_hash FROM users WHERE username=$1`, [name]);
const row = result.rows[0];
if (!await verifyPassword(value, row?.password_hash)) unauthorized();
const { password_hash, ...user } = row;
const token = randomBytes(32).toString('base64url');
const session = await pool.query(`INSERT INTO sessions(token_hash,user_id,expires_at)
VALUES($1,$2,clock_timestamp()+$3*interval '1 second') RETURNING expires_at`, [digest(token), user.id, ttl]);
return [201, { token, token_type: 'Bearer', expires_at: session.rows[0].expires_at, user }];
}
const { user, tokenHash } = await identity(req);
if (method === 'DELETE' && path === '/api/v1/auth/sessions/current') {
await pool.query('DELETE FROM sessions WHERE token_hash=$1', [tokenHash]);
return [204, null];
}
if (method === 'GET' && path === '/api/v1/users/me') return [200, user];
if (path === '/api/v1/chats' && method === 'POST') {
const data = await body(req);
fields(data, ['title', 'member_ids']); string(data.title, 100);
if (!Array.isArray(data.member_ids) || data.member_ids.length < 1 || data.member_ids.length > 100) invalid('Invalid members');
const ids = data.member_ids.map(uuid);
if (new Set(ids).size !== ids.length || !ids.includes(user.id)) invalid('Invalid members');
return [201, await transaction(async client => {
const known = await client.query('SELECT id FROM users WHERE id=ANY($1::uuid[])', [ids]);
if (known.rowCount !== ids.length) invalid('Unknown member');
const chat = (await client.query('INSERT INTO chats(id,title) VALUES($1,$2) RETURNING *', [randomUUID(), data.title])).rows[0];
await client.query('INSERT INTO chat_members(chat_id,user_id) SELECT $1,unnest($2::uuid[])', [chat.id, ids]);
return { ...chat, member_ids: ids.sort() };
})];
}
if (path === '/api/v1/chats' && method === 'GET') {
const paging = pagination(url.searchParams, 'chats:' + user.id);
const rows = (await pool.query(chatSelect + ` WHERE EXISTS
(SELECT 1 FROM chat_members WHERE chat_id=c.id AND user_id=$1)
AND ($2::timestamptz IS NULL OR (c.created_at,c.id)>($2::timestamptz,$3::uuid))
ORDER BY c.created_at,c.id LIMIT $4`, [user.id, paging.after?.created_at ?? null, paging.after?.id ?? null, paging.limit + 1])).rows;
return [200, page(rows, paging)];
}
const attachmentPath = path.match(/^\/api\/v1\/attachments\/([^/]+)(\/content)?$/);
if (attachmentPath && method === 'GET') {
const row = await attachments.accessible(uuid(attachmentPath[1]), user);
if (!attachmentPath[2]) return [200, attachments.visible(row)];
if (url.searchParams.getAll('variant').length > 1) invalid('Invalid variant');
if ((url.searchParams.get('variant') ?? 'original') !== 'original') fail(404, 'not_found', 'Object not found');
return [200, await attachments.download(row), { 'Content-Type': row.content_type, 'X-Content-Type-Options': 'nosniff' }];
}
const match = path.match(/^\/api\/v1\/chats\/([^/]+)(\/messages|\/attachments)?$/);
if (match) {
const chat = await accessible(match[1], user);
if (!match[2] && method === 'GET') return [200, chat];
if (match[2] === '/attachments' && method === 'POST') return [201, await attachments.upload(req, url.searchParams, chat, user)];
if (match[2] === '/messages' && method === 'POST') {
const key = req.headers['idempotency-key'];
if (typeof key !== 'string' || key.length < 1 || key.length > 128 || /[^A-Za-z0-9._:-]/.test(key)) invalid('Invalid Idempotency-Key');
const data = await body(req);
if (!data || typeof data !== 'object' || Array.isArray(data) || !Object.hasOwn(data, 'text') ||
Object.keys(data).some(k => !['text', 'attachment_ids'].includes(k))) invalid('Invalid fields');
if (typeof data.text !== 'string' || !data.text.isWellFormed() || [...data.text].length > 4000 || data.text.includes('\0')) invalid('Invalid text');
const rawIds = Object.hasOwn(data, 'attachment_ids') ? data.attachment_ids : [];
if (!Array.isArray(rawIds) || rawIds.length > 10) invalid('Invalid attachments');
const ids = rawIds.map(uuid);
if (new Set(ids).size !== ids.length || (ids.length === 0 && /^[\p{White_Space}\uFEFF]*$/u.test(data.text))) invalid('Empty message or duplicate attachments');
return [201, await sendMessage(user.id, chat.id, key, data.text, ids)];
}
if (match[2] === '/messages' && method === 'GET') {
const paging = pagination(url.searchParams, 'messages:' + chat.id);
const result = await pool.query(`SELECT m.*, ARRAY(SELECT attachment_id FROM message_attachments
WHERE message_id=m.id ORDER BY position) AS attachment_ids FROM messages m WHERE m.chat_id=$1
AND ($2::timestamptz IS NULL OR (m.created_at,m.id)>($2::timestamptz,$3::uuid))
ORDER BY m.created_at,m.id LIMIT $4`, [chat.id, paging.after?.created_at ?? null, paging.after?.id ?? null, paging.limit + 1]);
return [200, page(result.rows, paging)];
}
}
fail(404, 'not_found', 'Object not found');
}
const server = http.createServer(async (req, res) => {
const request_id = randomUUID();
let status, data, responseHeaders;
const headers = { 'Content-Type': 'application/json; charset=utf-8', 'X-Request-Id': request_id, 'X-Instance-Id': instanceId };
try { [status, data, responseHeaders] = await route(req); }
catch (error) {
status = error.status || (/^(08|53|57)/.test(error.code || '') || ['ECONNREFUSED', 'ECONNRESET', 'ETIMEDOUT'].includes(error.code) ? 503 : 500);
data = { error: { code: error.status ? error.code : status === 503 ? 'unavailable' : 'internal_error',
message: error.status ? error.message : status === 503 ? 'Database unavailable' : 'Internal error', request_id } };
if (status === 401) headers['WWW-Authenticate'] = req.url.split('?')[0] === '/api/v1/auth/sessions'
? 'Basic realm="messenger", charset="UTF-8"' : 'Bearer realm="messenger"';
}
if (status === 413) headers.Connection = 'close';
res.writeHead(status, { ...headers, ...responseHeaders });
res.end(status === 204 ? undefined : Buffer.isBuffer(data) ? data : JSON.stringify(data));
});
async function main() {
cursorSecret = (await pool.query("SELECT value FROM app_settings WHERE key='cursor_secret'")).rows[0].value;
server.listen(Number(process.env.APP_PORT || 8080), process.env.APP_HOST || '0.0.0.0', () => console.log('Messenger lab4 ready; instance=' + instanceId));
}
for (const signal of ['SIGTERM', 'SIGINT']) process.on(signal, () => {
stopping = true;
server.close(() => pool.end());
setTimeout(() => process.exit(1), 9000).unref();
});
main().catch(() => { console.error('Startup failed: check database and migrations'); process.exitCode = 1; pool.end(); });
+36
View File
@@ -0,0 +1,36 @@
'use strict';
const { readFileSync } = require('node:fs');
const { Agent } = require('node:http');
const { Client } = require('minio');
const endpoint = new URL(process.env.S3_ENDPOINT || 'http://s3:9000');
const bucket = process.env.S3_BUCKET || 'messenger-attachments';
const client = new Client({
endPoint: endpoint.hostname, port: Number(endpoint.port || 9000), useSSL: false,
region: process.env.S3_REGION || 'us-east-1',
accessKey: readFileSync(process.env.S3_ACCESS_KEY_ID_FILE || '/run/secrets/s3_access_key_id', 'utf8').trim(),
secretKey: readFileSync(process.env.S3_SECRET_ACCESS_KEY_FILE || '/run/secrets/s3_secret_access_key', 'utf8').trim(),
transportAgent: new Agent({ keepAlive: true, timeout: 5000, maxSockets: 16 }),
});
function unavailable() { return Object.assign(new Error('Object storage unavailable'), { status: 503, code: 'unavailable' }); }
async function ready() {
try { if (!await client.bucketExists(bucket)) throw new Error(); }
catch { throw unavailable(); }
}
async function bootstrap() {
for (let attempt = 0; attempt < 30; attempt++) {
try {
if (!await client.bucketExists(bucket)) await client.makeBucket(bucket, process.env.S3_REGION || 'us-east-1');
// Remove any anonymous policy; newly created buckets are private by default.
await client.setBucketPolicy(bucket, '');
console.log('Private bucket ready');
return;
} catch (error) {
if (attempt === 29) throw unavailable();
await new Promise(resolve => setTimeout(resolve, 1000));
}
}
}
module.exports = { client, bucket, ready, unavailable };
if (require.main === module) bootstrap().then(() => process.exit(0)).catch(() => {
console.error('Bucket initialization failed'); process.exit(1);
});
+151
View File
@@ -0,0 +1,151 @@
'use strict';
const assert = require('node:assert/strict');
const { readFileSync } = require('node:fs');
const { randomUUID, createHash } = require('node:crypto');
const { execFileSync } = require('node:child_process');
const http = require('node:http');
const base = process.env.BASE_URL || 'http://localhost:8084';
const png = readFileSync('tests/fixtures/sample.png');
const sha = bytes => createHash('sha256').update(bytes).digest('hex');
const pause = ms => new Promise(resolve => setTimeout(resolve, ms));
async function call(method, path, expected = 200, actor, data, headers = {}) {
const response = await fetch(base + path, { method, headers: {
...(actor ? { Authorization: 'Bearer ' + actor.token } : {}),
...(data === undefined ? {} : { 'Content-Type': Buffer.isBuffer(data) ? 'image/png' : 'application/json' }), ...headers },
body: data === undefined ? undefined : Buffer.isBuffer(data) ? data : JSON.stringify(data), signal: AbortSignal.timeout(12000) });
assert.equal(response.status, expected, method + ' ' + path.split('?')[0]);
assert.ok(response.headers.get('x-request-id'));
assert.ok(response.headers.get('x-instance-id'));
const bytes = Buffer.from(await response.arrayBuffer());
const json = response.headers.get('content-type')?.includes('application/json') ? JSON.parse(bytes) : null;
if (expected >= 400) assert.equal(json.error.request_id, response.headers.get('x-request-id'));
return { data: json, bytes, headers: response.headers, instance: response.headers.get('x-instance-id') };
}
async function user() {
const username = 'u_' + randomUUID().replaceAll('-', '').slice(0, 14), password = 'Secret-' + randomUUID();
const created = (await call('POST', '/api/v1/users', 201, null, { username, password, display_name: 'Студент' })).data;
const login = (await call('POST', '/api/v1/auth/sessions', 201, null, undefined,
{ Authorization: 'Basic ' + Buffer.from(username + ':' + password).toString('base64') })).data;
return { id: created.id, token: login.token };
}
const docker = (...args) => execFileSync('docker', args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();
const internal = script => docker('compose', 'exec', '-T', 'api1', 'node', '-e', script);
function control(action, target) {
const python = process.env.LAB_PYTHON || (process.platform === 'win32' ? 'py' : 'python3');
execFileSync(python, ['scripts/control.py', action], { stdio: ['ignore', 'pipe', 'pipe'],
env: { ...process.env, ...(target ? { TARGET_INSTANCE_ID: target } : {}) } });
}
async function ready(actor) {
for (let i = 0; i < 40; i++) {
try { await call('GET', '/api/v1/users/me', 200, actor); return; } catch { await pause(250); }
}
throw new Error('Load balancer did not recover');
}
function crc32(buffer) {
let value = 0xffffffff;
for (const byte of buffer) {
value ^= byte;
for (let bit = 0; bit < 8; bit++) value = (value >>> 1) ^ ((value & 1) ? 0xedb88320 : 0);
}
return (value ^ 0xffffffff) >>> 0;
}
function chunkedTooLarge(path, actor) {
return new Promise((resolve, reject) => {
const req = http.request(base + path, { method: 'POST', headers: {
Authorization: 'Bearer ' + actor.token, 'Content-Type': 'image/png', 'Transfer-Encoding': 'chunked' } }, res => {
let body = '';
res.on('data', chunk => body += chunk);
res.on('end', () => {
try { assert.equal(res.statusCode, 413); assert.equal(JSON.parse(body).error.code, 'payload_too_large'); resolve(); }
catch (error) { reject(error); }
});
});
req.on('error', error => { if (!req.res) reject(error); });
req.setTimeout(15000, () => req.destroy(new Error('Chunked upload timeout')));
req.write(png);
for (let i = 0; i < 11; i++) req.write(Buffer.alloc(1024 * 1024));
req.end();
});
}
async function main() {
const a = await user(), b = await user(), outsider = await user();
const chat = (await call('POST', '/api/v1/chats', 201, a, { title: 'Изображения', member_ids: [a.id, b.id] })).data;
const uploadPath = '/api/v1/chats/' + chat.id + '/attachments';
const messages = '/api/v1/chats/' + chat.id + '/messages';
const uploaded = await call('POST', uploadPath + '?filename=' + encodeURIComponent('Фото 😀.png'), 201, a, png);
const att = uploaded.data, content = '/api/v1/attachments/' + att.id + '/content';
assert.equal(att.status, 'ready'); assert.equal(att.sha256, sha(png)); assert.equal(att.size_bytes, png.length);
assert.equal(att.object_key, undefined);
const download = await call('GET', content, 200, b);
assert.deepEqual(download.bytes, png); assert.equal(download.headers.get('content-type'), 'image/png');
for (const suffix of ['', '/content']) await call('GET', '/api/v1/attachments/' + att.id + suffix, 404, outsider);
await call('GET', content, 401);
await call('GET', content + '?variant=thumbnail', 404, a);
const key = randomUUID(), body = { text: '', attachment_ids: [att.id] };
const concurrent = await Promise.all(Array.from({ length: 12 }, () => call('POST', messages, 201, a, body, { 'Idempotency-Key': key })));
for (const result of concurrent) assert.deepEqual(result.data, concurrent[0].data);
assert.equal(new Set(concurrent.map(r => r.instance)).size, 2);
assert.deepEqual(concurrent[0].data.attachment_ids, [att.id]);
assert.equal((await call('GET', messages, 200, b)).data.items.length, 1);
await call('POST', messages, 409, a, { text: 'Changed', attachment_ids: [att.id] }, { 'Idempotency-Key': key });
const textKey = randomUUID();
const plain = await call('POST', messages, 201, a, { text: 'Text only' }, { 'Idempotency-Key': textKey });
const explicit = await call('POST', messages, 201, a, { text: 'Text only', attachment_ids: [] }, { 'Idempotency-Key': textKey });
assert.deepEqual(plain.data, explicit.data);
await call('POST', messages, 400, a, { text: '', attachment_ids: [att.id, att.id] }, { 'Idempotency-Key': randomUUID() });
const otherChat = (await call('POST', '/api/v1/chats', 201, a, { title: 'Other', member_ids: [a.id] })).data;
await call('POST', '/api/v1/chats/' + otherChat.id + '/messages', 404, a, body, { 'Idempotency-Key': randomUUID() });
for (const name of ['../bad.png', 'a/b.png', 'a\\b.png', 'a\0.png', '\u0085', 'a\n.png', 'x'.repeat(256)])
await call('POST', uploadPath + '?filename=' + encodeURIComponent(name), 400, a, png);
await call('POST', uploadPath, 400, a, png);
await call('POST', uploadPath + '?filename=x.png', 415, a, Buffer.from('not an image'));
await call('POST', uploadPath + '?filename=x.png', 415, a, png.subarray(0, 60));
await call('POST', uploadPath + '?filename=x.jpg', 415, a, png, { 'Content-Type': 'image/jpeg' });
await call('POST', uploadPath + '?filename=x.png', 415, a, png, { 'Content-Type': 'text/plain' });
await call('POST', uploadPath + '?filename=x.png', 413, a, Buffer.alloc(10485761));
await chunkedTooLarge(uploadPath + '?filename=large.png', a);
const huge = Buffer.from(png);
huge.writeUInt32BE(8001, 16); huge.writeUInt32BE(5000, 20); huge.writeUInt32BE(crc32(huge.subarray(12, 29)), 29);
await call('POST', uploadPath + '?filename=huge.png', 413, a, huge);
const jpeg = Buffer.from(internal("require('sharp')({create:{width:20,height:10,channels:3,background:'red'}}).jpeg().toBuffer().then(b=>console.log(b.toString('base64')))"), 'base64');
const jpg = await call('POST', uploadPath + '?filename=real.jpg', 201, b, jpeg, { 'Content-Type': 'image/jpeg' });
assert.deepEqual((await call('GET', '/api/v1/attachments/' + jpg.data.id + '/content', 200, a)).bytes, jpeg);
// Resolve the private storage key inside the API container; never expose it in public metadata.
const anonymous = JSON.parse(internal(`const {pool}=require('./src/db');pool.query('SELECT object_key FROM attachments WHERE id=$1',['${att.id}']).then(async r=>{const response=await fetch('http://s3:9000/messenger-attachments/'+r.rows[0].object_key);console.log(JSON.stringify({status:response.status}));}).finally(()=>pool.end());`));
assert.equal(anonymous.status, 403);
console.log('PASS: PNG/JPEG exact bytes and SHA-256; private bucket 403; member access; invalid filename/MIME/corrupt file; >10 MiB including chunked and >40M pixels; attachment scope; 12 concurrent sends make 1 message.');
if (process.argv.includes('--lifecycle')) {
let target = uploaded.instance;
for (let round = 0; round < 2; round++) {
// Stop the exact process that accepted this upload, then download only through LB.
const fresh = round === 0 ? uploaded : await call('POST', uploadPath + '?filename=second.png', 201, a, png);
target = fresh.instance;
const started = Date.now();
control('stop-one', target);
try {
let ok = false;
for (let i = 0; i < 30 && Date.now() - started < 10000; i++) {
try {
const bytes = await call('GET', '/api/v1/attachments/' + fresh.data.id + '/content', 200, b);
assert.notEqual(bytes.instance, target); assert.deepEqual(bytes.bytes, png); ok = true; break;
} catch { await pause(100); }
}
assert.ok(ok, 'Download after uploader failure exceeded 10 seconds');
console.log(JSON.stringify({ scenario: 'upload-stop-uploader-download', recovery_ms: Date.now() - started, same_bytes: true }));
} finally { control('restore'); await ready(a); }
}
control('restart'); await ready(a);
assert.deepEqual((await call('GET', content, 200, b)).bytes, png);
assert.deepEqual((await call('POST', messages, 201, a, body, { 'Idempotency-Key': key })).data, concurrent[0].data);
console.log('PASS: full restart preserves metadata, original bytes, session and idempotent attachment message.');
docker('compose', 'stop', 's3');
try {
const checks = JSON.parse(internal(`Promise.all(['/health/live','/health/ready'].map(async p=>{const r=await fetch('http://127.0.0.1:8080'+p);return r.status})).then(s=>console.log(JSON.stringify(s)))`));
assert.deepEqual(checks, [200, 503]);
console.log('PASS: stopped S3 makes API readiness 503 while liveness remains 200.');
} finally { docker('compose', 'up', '-d', '--wait', 's3'); await ready(a); }
assert.deepEqual((await call('GET', content, 200, a)).bytes, png);
console.log('PASS: S3 recovery restores download without data loss.');
}
}
main().catch(error => { console.error('FAIL: ' + error.message); process.exitCode = 1; });