ЛР 4 — БПИ2304 — Кирюпина — на 3 #127
+5
-18
@@ -1,20 +1,7 @@
|
||||
# Copy to .env for your LOCAL synthetic test environment; never commit .env.
|
||||
# These are logical interface suggestions; map actual framework names in REPORT.md.
|
||||
APP_HOST=0.0.0.0
|
||||
APP_PORT=8080
|
||||
APP_ENV=lab
|
||||
# Generate .env and local credentials with scripts/setup.py; do not commit them.
|
||||
# Use 8084 to keep lab3 on 8080; API replicas listen on internal port 8080.
|
||||
APP_PORT=8084
|
||||
SESSION_TTL_SECONDS=3600
|
||||
# Choose credentials locally. File paths contain no secret values.
|
||||
DATABASE_URL_FILE=/run/secrets/database_url
|
||||
# Optional on 3, required use on 4 in lab2; Redis OR Valkey:
|
||||
SESSION_STORE=postgres
|
||||
CACHE_URL_FILE=/run/secrets/cache_url
|
||||
API_REPLICAS=2
|
||||
IDEMPOTENCY_TTL_SECONDS=86400
|
||||
S3_ENDPOINT=http://s3:9000
|
||||
S3_REGION=us-east-1
|
||||
S3_BUCKET=messenger-attachments
|
||||
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
|
||||
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
|
||||
MAX_UPLOAD_BYTES=10485760
|
||||
MAX_IMAGE_PIXELS=40000000
|
||||
# Storage address/bucket/secrets paths are fixed in compose.yaml.
|
||||
# Upload limits: 10485760 bytes and 40000000 pixels.
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
*.sh text eol=lf
|
||||
*.cmd text eol=crlf
|
||||
*.sql text eol=lf
|
||||
@@ -0,0 +1,9 @@
|
||||
FROM node:22.20.0-alpine
|
||||
WORKDIR /app
|
||||
COPY package.json package-lock.json ./
|
||||
RUN npm ci --omit=dev --ignore-scripts
|
||||
COPY src/ ./src/
|
||||
COPY migrations/ ./migrations/
|
||||
ENV APP_HOST=0.0.0.0 APP_PORT=8080 APP_ENV=lab
|
||||
EXPOSE 8080
|
||||
CMD ["node", "src/server.js"]
|
||||
@@ -1,37 +1,62 @@
|
||||
# Отчёт — лабораторная 4
|
||||
|
||||
- Автор, группа: TODO
|
||||
- Целевая оценка; выбранный трек 5, если нужен: TODO
|
||||
- Commit/PR: TODO после публикации
|
||||
- Использованные источники, библиотеки и помощь ИИ: TODO
|
||||
- Автор, группа: Кирюпина Александра Радиевна, БПИ2304.
|
||||
- Целевая оценка: 3. Трек 5 не выбран.
|
||||
- Ветка: `solution/bpi2304-kiryupina`; PR: `ЛР 4 — БПИ2304 — Кирюпина — на 3` (ссылка после публикации).
|
||||
- Источники: контракт курса, документация [MinIO](https://github.com/minio/minio/tree/RELEASE.2025-04-22T22-12-26Z), [Sharp](https://sharp.pixelplumbing.com/). Библиотеки: pg 8.16.3, minio 8.0.5, sharp 0.34.3. ИИ использовался при подготовке кода, проверок и отчёта.
|
||||
|
||||
## Запуск из чистого clone
|
||||
|
||||
TODO: версии runtime/образов, prerequisites, команды создания локальных secrets/.env, build, миграций, запуска, ожидания готовности, тестов и остановки. Указать необходимые действия оператора. Не вставлять секреты. Отдельно указать безопасную остановку и команду намеренного удаления учебных данных.
|
||||
Нужны Docker Desktop с Linux containers и Python 3.9+. Команды из папки проекта в PowerShell:
|
||||
|
||||
```powershell
|
||||
py -X utf8 scripts/setup.py
|
||||
docker compose up -d --build --wait
|
||||
py -X utf8 scripts/check_contract.py
|
||||
py -X utf8 tests/smoke.py --base-url http://localhost:8084
|
||||
```
|
||||
|
||||
Setup один раз создаёт локальные `.env` и `secrets/`, они не входят в Git. В `.env` задаются внешний `APP_PORT=8084`, TTL сессии 3600 с и ключа идемпотентности 86400 с. Внутренние адреса, bucket и пути secret-файлов заданы в Compose. Готовность: `GET /health/ready` → 200.
|
||||
|
||||
Версии: Node 22.20.0-alpine, PostgreSQL 17.6-alpine, HAProxy 3.0.12-alpine. MinIO собирается из исходников релиза `RELEASE.2025-04-22T22-12-26Z` через Go 1.24.3-alpine, runtime Alpine 3.22.1 (`infra/Dockerfile.s3`); готовый образ из реестра не нужен. Первый build требует доступа к GitHub и репозиториям зависимостей. Миграции и создание приватного bucket автоматические.
|
||||
|
||||
Повторная сборка/запуск — та же команда `docker compose up -d --build --wait`. Остановка с сохранением данных — `docker compose down`. Намеренное удаление учебных данных — `docker compose down -v`. Локальные secrets сохраняются между запусками. Linux: `python3` вместо `py -X utf8`; эквиваленты проверок — `make check`, `make test BASE_URL=http://localhost:8084`.
|
||||
|
||||
## Архитектура и решения
|
||||
|
||||
TODO: схема процессов/хранилищ, источник истины, транзакционные границы, ограничения. Что перенесено из lab3, что изменено в этой работе. Соответствие переменных из .env.example реальной конфигурации.
|
||||
Клиент → HAProxy → два API → PostgreSQL + MinIO. Из lab3 перенесены сессии, членство в чатах, пагинация и общая идемпотентность сообщений. Текстовые сообщения продолжают работать; добавлен список `attachment_ids` (до 10 уникальных ID того же чата). Запись сообщения и результата повтора выполняется одной транзакцией PostgreSQL.
|
||||
|
||||
Метаданные вложений хранятся в PostgreSQL, оригиналы — в приватном bucket `messenger-attachments`. У API нет volume с файлами. Имя файла используется только для отображения; ключ S3 случайный и скрыт. Проверяются PNG/JPEG, MIME, имя, лимит 10 MiB и 40 млн пикселей до полной распаковки. Повреждённые изображения отклоняются. После декодирования сохраняются исходные байты, без преобразования.
|
||||
|
||||
Ответ 201 ready выдаётся после S3 PUT и записи метаданных. Если INSERT не удался, выполняется попытка удалить объект. Download идёт через API с проверкой членства, без redirect. Оба хранилища имеют persistent volumes. Readiness зависит от PostgreSQL и S3; liveness — только от процесса. Пагинация сортирует `(created_at,id)`; при конкурентных вставках snapshot между страницами не гарантируется.
|
||||
|
||||
## Доказательства критериев
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO | TODO |
|
||||
| Остальные пункты уровня 3 | TODO | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
Результаты запусков и состояния контейнеров записаны в `evidence/`. Команды persistence/failover ниже используют неизменённые публичные тесты. На Windows выполнены прямые Python-команды, эквивалентные make-целям.
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
| Критерий | Проверка | Наблюдение | Файл в evidence/ |
|
||||
| --- | --- | --- | --- |
|
||||
| Контракт и публичный API | check_contract.py, smoke.py | PASS; 17 файлов задания/контракта/тестов не изменены | check.txt, smoke.txt, contract-integrity.txt |
|
||||
| Приватный S3, PNG/JPEG и ограничения | student-tests/check.js | Точные байты и SHA-256; anonymous S3 GET=403; неверные файлы отклонены | student-tests.txt |
|
||||
| Остановка принявшей файл реплики → download другой | check.js --lifecycle | Точные байты, 4054 и 2199 мс с учётом hook | student-tests.txt, stop-api*.json |
|
||||
| Полный restart с сохранением оригинала и метаданных | публичный persistence | PASS: данные и прежняя сессия сохранены | persistence.txt, restart-scope.json |
|
||||
| Работа при отказе API, общая идемпотентность | публичный failover | PASS: 2 реплики; восстановление 5.156 с; 6 успешных записей, 1 неуспешная попытка во время переключения | failover.txt |
|
||||
|
||||
```powershell
|
||||
py -X utf8 tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url http://localhost:8084
|
||||
py -X utf8 tests/resilience.py --mode failover --action-script scripts/stop-one.cmd --base-url http://localhost:8084
|
||||
py -X utf8 scripts/control.py restore
|
||||
node student-tests/check.js --lifecycle
|
||||
```
|
||||
|
||||
Для последней команды нужен Node.js 22+. На Linux hooks — `scripts/restart.sh` и `scripts/stop-one.sh` с executable bit. Hook restart перезапускает LB, оба API, PostgreSQL и MinIO, не удаляет volumes; stop-one останавливает именно переданную API-реплику.
|
||||
|
||||
## Эксперименты и отказы
|
||||
|
||||
TODO: гипотеза, hardware/лимиты, версии, объём данных, профиль нагрузки, concurrency/arrival rate, длительность/прогрев, повторы, успешные RPS, ошибки, p50/p95/p99, CPU/RAM. Сохранить raw summary без tokens и объяснить ограничения измерения.
|
||||
Локальный стенд: Windows, Docker Desktop 29.9.2, 12 CPU, около 7.7 GiB RAM у Docker; отдельных лимитов контейнеров нет (`evidence/environment.txt`). Это функциональные проверки, измерение максимальной производительности не выполнялось.
|
||||
|
||||
| Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных |
|
||||
| --- | --- | --- | --- |
|
||||
| TODO | TODO | TODO | TODO |
|
||||
Проверены fixture PNG 320×200 (111817 байт), JPEG 20×10, повреждённые файлы, подмена MIME, опасные имена, превышение bytes (включая chunked) и пикселей. 12 параллельных отправок с одним ключом создали одно сообщение. После остановки загрузившей реплики файл прочитан через другую; полный restart сохранил файл, сессию и повтор сообщения. При отключённом S3 получены live=200 и ready=503, после возврата S3 — прежний оригинал. Все дополнительные проверки прошли; вывод в student-tests.txt.
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
|
||||
TODO: какие отказы выдерживаются и какие нет; что не реализовано; где учебное упрощение; какое улучшение подтверждено, а какое пока гипотеза.
|
||||
Общей транзакции PostgreSQL/S3 нет: авария между операциями может оставить лишний объект, автоматической сверки пока нет. API использует общие административные S3 credentials; отдельные ограниченные права — следующий шаг. Оригинал буферизуется до 10 MiB, декодирование ограничено 40 млн пикселей, но общего лимита памяти для множества параллельных загрузок нет. LB, PostgreSQL, S3 и сам компьютер остаются единичными точками отказа. Миниатюры, фоновые задачи и треки 4/5 не реализованы.
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# Лабораторная 4 — запуск в PowerShell
|
||||
|
||||
Открой эту папку в VS Code и терминал PowerShell. Нужны запущенный Docker
|
||||
Desktop (Linux containers), Python 3.9+; для дополнительных тестов — Node.js 22+.
|
||||
|
||||
```powershell
|
||||
py -X utf8 scripts/setup.py
|
||||
docker compose up -d --build --wait
|
||||
py -X utf8 scripts/check_contract.py
|
||||
py -X utf8 tests/smoke.py --base-url http://localhost:8084
|
||||
```
|
||||
|
||||
API доступен на http://localhost:8084. Порт 8080 предыдущей лабы не занят.
|
||||
Первый запуск скачивает зависимости и собирает MinIO из исходников закреплённого
|
||||
релиза: это может занять несколько минут. Повторная сборка использует кэш.
|
||||
Миграции и создание приватного bucket выполняются автоматически.
|
||||
|
||||
Проверка полного перезапуска и отказа API (запускай по очереди):
|
||||
|
||||
```powershell
|
||||
py -X utf8 tests/resilience.py --mode persistence --action-script scripts/restart.cmd --base-url http://localhost:8084
|
||||
py -X utf8 tests/resilience.py --mode failover --action-script scripts/stop-one.cmd --base-url http://localhost:8084
|
||||
py -X utf8 scripts/control.py restore
|
||||
node student-tests/check.js --lifecycle
|
||||
```
|
||||
|
||||
Дополнительный тест сам загружает PNG/JPEG, проверяет ограничения и права доступа,
|
||||
останавливает принявшую файл реплику, скачивает оригинал через вторую,
|
||||
перезапускает весь стенд и временно отключает S3. После проверок восстанавливает сервисы.
|
||||
|
||||
Остановка с сохранением данных: `docker compose down`.
|
||||
Повторный запуск и сборка: `docker compose up -d --build --wait`.
|
||||
Только для намеренного удаления всех учебных данных этой лабы: `docker compose down -v`.
|
||||
Сохраняй папку secrets между запусками; не добавляй её и .env в Git или архив сдачи.
|
||||
Если переносишь уже работающий стенд в другую папку на этом же компьютере,
|
||||
перенеси `.env` и `secrets/` туда отдельно: существующие Docker volumes ожидают
|
||||
прежние пароли. Для текущего запуска используй уже подготовленную папку проекта.
|
||||
|
||||
Linux: вместо `py -X utf8` используй `python3`, вместо hooks `.cmd` — `.sh`.
|
||||
Перед запуском hooks: `chmod +x scripts/restart.sh scripts/stop-one.sh`.
|
||||
Эквиваленты публичных проверок: `make check` и `make test BASE_URL=http://localhost:8084`.
|
||||
|
||||
Короткий отчёт — REPORT.md; результаты реальных проверок — evidence/.
|
||||
Заголовок PR: **ЛР 4 — БПИ2304 — Кирюпина — на 3**.
|
||||
+106
@@ -0,0 +1,106 @@
|
||||
name: mtusi-lab4
|
||||
x-s3-env: &s3-env
|
||||
S3_ENDPOINT: http://s3:9000
|
||||
S3_REGION: us-east-1
|
||||
S3_BUCKET: messenger-attachments
|
||||
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_access_key_id
|
||||
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_secret_access_key
|
||||
x-api: &api
|
||||
image: mtusi-messenger:lab4
|
||||
build: .
|
||||
environment:
|
||||
<<: *s3-env
|
||||
APP_HOST: 0.0.0.0
|
||||
APP_PORT: 8080
|
||||
SESSION_TTL_SECONDS: ${SESSION_TTL_SECONDS:-3600}
|
||||
IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400}
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
secrets: [database_url, s3_access_key_id, s3_secret_access_key]
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
s3-init:
|
||||
condition: service_completed_successfully
|
||||
healthcheck:
|
||||
test: ["CMD", "node", "-e", "fetch('http://127.0.0.1:8080/health/ready').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"]
|
||||
interval: 2s
|
||||
timeout: 4s
|
||||
retries: 20
|
||||
stop_grace_period: 10s
|
||||
services:
|
||||
db:
|
||||
image: postgres:17.6-alpine
|
||||
environment:
|
||||
POSTGRES_DB: messenger
|
||||
POSTGRES_USER: messenger
|
||||
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
|
||||
secrets: [postgres_password]
|
||||
volumes: [pgdata:/var/lib/postgresql/data]
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U messenger -d messenger"]
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
migrate:
|
||||
image: mtusi-messenger:lab4
|
||||
build: .
|
||||
command: ["node", "src/migrate.js"]
|
||||
environment:
|
||||
DATABASE_URL_FILE: /run/secrets/database_url
|
||||
secrets: [database_url]
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
s3:
|
||||
image: mtusi-minio:RELEASE.2025-04-22T22-12-26Z
|
||||
build:
|
||||
context: .
|
||||
dockerfile: infra/Dockerfile.s3
|
||||
command: ["server", "/data", "--console-address", ":9001"]
|
||||
environment:
|
||||
MINIO_ROOT_USER_FILE: /run/secrets/s3_access_key_id
|
||||
MINIO_ROOT_PASSWORD_FILE: /run/secrets/s3_secret_access_key
|
||||
secrets: [s3_access_key_id, s3_secret_access_key]
|
||||
volumes: [s3data:/data]
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
s3-init:
|
||||
image: mtusi-messenger:lab4
|
||||
build: .
|
||||
command: ["node", "src/storage.js"]
|
||||
environment:
|
||||
<<: *s3-env
|
||||
secrets: [s3_access_key_id, s3_secret_access_key]
|
||||
depends_on:
|
||||
s3:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
api1:
|
||||
<<: *api
|
||||
api2:
|
||||
<<: *api
|
||||
lb:
|
||||
image: haproxy:3.0.12-alpine
|
||||
ports: ["127.0.0.1:${APP_PORT:-8084}:8080"]
|
||||
volumes: [./infra/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro]
|
||||
depends_on:
|
||||
api1:
|
||||
condition: service_healthy
|
||||
api2:
|
||||
condition: service_healthy
|
||||
volumes:
|
||||
pgdata:
|
||||
s3data:
|
||||
secrets:
|
||||
postgres_password:
|
||||
file: ./secrets/postgres_password
|
||||
database_url:
|
||||
file: ./secrets/database_url
|
||||
s3_access_key_id:
|
||||
file: ./secrets/s3_access_key_id
|
||||
s3_secret_access_key:
|
||||
file: ./secrets/s3_secret_access_key
|
||||
@@ -0,0 +1,476 @@
|
||||
Image mtusi-messenger:lab4 Building
|
||||
Image mtusi-minio:RELEASE.2025-04-22T22-12-26Z Building
|
||||
Image mtusi-messenger:lab4 Building
|
||||
Image mtusi-messenger:lab4 Building
|
||||
Image mtusi-messenger:lab4 Building
|
||||
#1 [internal] load local bake definitions
|
||||
#1 reading from stdin 2.92kB 0.0s done
|
||||
#1 DONE 0.0s
|
||||
|
||||
#2 [api1 internal] load build definition from Dockerfile
|
||||
#2 transferring dockerfile: 292B done
|
||||
#2 DONE 0.0s
|
||||
|
||||
#3 [s3 internal] load build definition from Dockerfile.s3
|
||||
#3 transferring dockerfile: 596B done
|
||||
#3 DONE 0.0s
|
||||
|
||||
#4 [migrate internal] load metadata for docker.io/library/node:22.20.0-alpine
|
||||
#4 ...
|
||||
|
||||
#5 [s3 internal] load metadata for docker.io/library/alpine:3.22.1
|
||||
#5 ...
|
||||
|
||||
#6 [s3 internal] load metadata for docker.io/library/golang:1.24.3-alpine
|
||||
#6 ...
|
||||
|
||||
#5 [s3 internal] load metadata for docker.io/library/alpine:3.22.1
|
||||
#5 DONE 12.4s
|
||||
|
||||
#4 [migrate internal] load metadata for docker.io/library/node:22.20.0-alpine
|
||||
#4 ...
|
||||
|
||||
#6 [s3 internal] load metadata for docker.io/library/golang:1.24.3-alpine
|
||||
#6 DONE 12.6s
|
||||
|
||||
#7 [migrate internal] load .dockerignore
|
||||
#7 transferring context: 196B done
|
||||
#7 DONE 0.0s
|
||||
|
||||
#4 [api1 internal] load metadata for docker.io/library/node:22.20.0-alpine
|
||||
#4 DONE 12.6s
|
||||
|
||||
#8 [migrate internal] load build context
|
||||
#8 transferring context: 474B done
|
||||
#8 DONE 0.0s
|
||||
|
||||
#9 [s3 build 1/5] FROM docker.io/library/golang:1.24.3-alpine@sha256:b4f875e650466fa0fe62c6fd3f02517a392123eea85f1d7e69d85f780e4db1c1
|
||||
#9 resolve docker.io/library/golang:1.24.3-alpine@sha256:b4f875e650466fa0fe62c6fd3f02517a392123eea85f1d7e69d85f780e4db1c1 0.0s done
|
||||
#9 DONE 0.0s
|
||||
|
||||
#10 [s3 build 3/5] RUN git clone --depth 1 --branch RELEASE.2025-04-22T22-12-26Z https://github.com/minio/minio.git /src
|
||||
#10 CACHED
|
||||
|
||||
#11 [s3 build 2/5] RUN apk add --no-cache git || apk add --no-cache git || apk add --no-cache git
|
||||
#11 CACHED
|
||||
|
||||
#12 [s3 build 4/5] WORKDIR /src
|
||||
#12 CACHED
|
||||
|
||||
#13 [s3 stage-1 1/3] FROM docker.io/library/alpine:3.22.1@sha256:4bcff63911fcb4448bd4fdacec207030997caf25e9bea4045fa6c8c44de311d1
|
||||
#13 resolve docker.io/library/alpine:3.22.1@sha256:4bcff63911fcb4448bd4fdacec207030997caf25e9bea4045fa6c8c44de311d1 0.0s done
|
||||
#13 DONE 0.1s
|
||||
|
||||
#14 [s3 stage-1 2/3] RUN apk add --no-cache ca-certificates curl
|
||||
#14 CACHED
|
||||
|
||||
#15 [migrate 1/6] FROM docker.io/library/node:22.20.0-alpine@sha256:dbcedd8aeab47fbc0f4dd4bffa55b7c3c729a707875968d467aaaea42d6225af
|
||||
#15 resolve docker.io/library/node:22.20.0-alpine@sha256:dbcedd8aeab47fbc0f4dd4bffa55b7c3c729a707875968d467aaaea42d6225af 0.0s done
|
||||
#15 DONE 0.0s
|
||||
|
||||
#16 [migrate 4/6] RUN npm ci --omit=dev --ignore-scripts
|
||||
#16 CACHED
|
||||
|
||||
#17 [migrate 2/6] WORKDIR /app
|
||||
#17 CACHED
|
||||
|
||||
#18 [migrate 5/6] COPY src/ ./src/
|
||||
#18 CACHED
|
||||
|
||||
#19 [migrate 3/6] COPY package.json package-lock.json ./
|
||||
#19 CACHED
|
||||
|
||||
#20 [migrate 6/6] COPY migrations/ ./migrations/
|
||||
#20 CACHED
|
||||
|
||||
#21 [api1] exporting to image
|
||||
#21 exporting layers done
|
||||
#21 exporting manifest sha256:e366d8aa98f16a1bd289889387fdf3401a716a731cdabf9b0a834d3683c8ae39 done
|
||||
#21 exporting config sha256:0ffd6c566da86551d5fbee607b6d9565c714596a62bc96b7e37a2c03934a0dbd
|
||||
#21 exporting config sha256:0ffd6c566da86551d5fbee607b6d9565c714596a62bc96b7e37a2c03934a0dbd done
|
||||
#21 exporting attestation manifest sha256:b3922880990bdf70372adb55bb1bd697b432717f6a5aaace3d3f6669b7283010
|
||||
#21 exporting attestation manifest sha256:b3922880990bdf70372adb55bb1bd697b432717f6a5aaace3d3f6669b7283010 0.1s done
|
||||
#21 exporting manifest list sha256:0fd8f69f2afa0bc0d3f70e2142c49e708fc53f70f552ff07fa515d6437db2193 0.1s done
|
||||
#21 naming to docker.io/library/mtusi-messenger:lab4 0.0s done
|
||||
#21 unpacking to docker.io/library/mtusi-messenger:lab4
|
||||
#21 unpacking to docker.io/library/mtusi-messenger:lab4 0.0s done
|
||||
#21 DONE 0.3s
|
||||
|
||||
#22 [s3-init] exporting to image
|
||||
#22 exporting layers done
|
||||
#22 exporting manifest sha256:70d2bcedbcf9ca498b7814d5f6a06f8ffd4fdde5d4f3529076a038f240f1d9ab 0.0s done
|
||||
#22 exporting config sha256:84ad10b06644c20475de71e28b2781a193ec8a190de29d725d7d294752759ba0 0.0s done
|
||||
#22 exporting attestation manifest sha256:4c05668b90e165538b4451e3785939ecb6c88ef328be01e8b6185d34542081b5 0.1s done
|
||||
#22 exporting manifest list sha256:caaf8e14d21f60fa57d8c26684b7fff06a4ab1efe9a130c704a409537958a272 0.0s done
|
||||
#22 naming to docker.io/library/mtusi-messenger:lab4 0.0s done
|
||||
#22 unpacking to docker.io/library/mtusi-messenger:lab4 0.0s done
|
||||
#22 DONE 0.3s
|
||||
|
||||
#23 [migrate] exporting to image
|
||||
#23 exporting layers done
|
||||
#23 exporting manifest sha256:a0571ea49777a38abe14d14c27b759b4cab439713afb494ad8b22dc7e3513201 0.0s done
|
||||
#23 exporting config sha256:db473997689da60bee5064a412adb972245d3642e4ed03597357c52af6569d09 0.0s done
|
||||
#23 exporting attestation manifest sha256:ef7595a236175ff992c6293c425c6bc63201ef830e33e60314c95d22af9cbae9 0.1s done
|
||||
#23 exporting manifest list sha256:ecfcc7984ef7a74654816bab8e4a129a2699c1dd2795860d52151ffe771cae80 0.0s done
|
||||
#23 naming to docker.io/library/mtusi-messenger:lab4 0.0s done
|
||||
#23 unpacking to docker.io/library/mtusi-messenger:lab4 0.0s done
|
||||
#23 DONE 0.3s
|
||||
|
||||
#24 [api2] exporting to image
|
||||
#24 exporting layers done
|
||||
#24 exporting manifest sha256:75185eadd9eb95e48fbe46d04ab10dd4513f5bce34fd440f3b420e45e018c34a 0.0s done
|
||||
#24 exporting config sha256:a0bf9ce288f460f565a7c8bf8004bc2e92f86c403e2b07d60679127adaa8694a 0.0s done
|
||||
#24 exporting attestation manifest sha256:10c05168a828f7ce7095b7de63ee3a47287f5a8670da0507bebe958bf7a33b8b 0.1s done
|
||||
#24 exporting manifest list sha256:96ceb3e9a5f8bd0269f913a31d95027d1cafbfdbc48022836ed1a4ea58d9578c 0.0s done
|
||||
#24 naming to docker.io/library/mtusi-messenger:lab4 done
|
||||
#24 unpacking to docker.io/library/mtusi-messenger:lab4 0.0s done
|
||||
#24 DONE 0.3s
|
||||
|
||||
#25 [s3 build 5/5] RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build for attempt in 1 2 3 4 5; do CGO_ENABLED=0 go build -trimpath -o /minio . && exit 0; done; exit 1
|
||||
#25 0.526 go: downloading cloud.google.com/go/storage v1.46.0
|
||||
#25 0.529 go: downloading github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0
|
||||
#25 0.531 go: downloading github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.2
|
||||
#25 0.531 go: downloading github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.0
|
||||
#25 0.531 go: downloading github.com/cheggaaa/pb v1.0.29
|
||||
#25 0.531 go: downloading github.com/cespare/xxhash/v2 v2.3.0
|
||||
#25 0.533 go: downloading github.com/coreos/go-oidc/v3 v3.12.0
|
||||
#25 0.533 go: downloading github.com/coreos/go-systemd/v22 v22.5.0
|
||||
#25 0.534 go: downloading github.com/cosnicolaou/pbzip2 v1.0.5
|
||||
#25 0.551 go: downloading github.com/dchest/siphash v1.2.3
|
||||
#25 ...
|
||||
|
||||
#26 [migrate] resolving provenance for metadata file
|
||||
#26 DONE 0.0s
|
||||
|
||||
#27 [api1] resolving provenance for metadata file
|
||||
#27 DONE 0.1s
|
||||
|
||||
#28 [s3-init] resolving provenance for metadata file
|
||||
#28 DONE 0.1s
|
||||
|
||||
#29 [api2] resolving provenance for metadata file
|
||||
#29 DONE 0.0s
|
||||
|
||||
#25 [s3 build 5/5] RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build for attempt in 1 2 3 4 5; do CGO_ENABLED=0 go build -trimpath -o /minio . && exit 0; done; exit 1
|
||||
#25 2.411 go: downloading github.com/dustin/go-humanize v1.0.1
|
||||
#25 2.420 go: downloading github.com/fatih/color v1.18.0
|
||||
#25 2.421 go: downloading github.com/felixge/fgprof v0.9.5
|
||||
#25 2.453 go: downloading github.com/go-openapi/loads v0.22.0
|
||||
#25 3.495 go: downloading github.com/golang-jwt/jwt/v4 v4.5.2
|
||||
#25 3.631 go: downloading github.com/google/uuid v1.6.0
|
||||
#25 3.631 go: downloading github.com/inconshreveable/mousetrap v1.1.0
|
||||
#25 3.741 go: downloading github.com/json-iterator/go v1.1.12
|
||||
#25 3.743 go: downloading github.com/klauspost/compress v1.18.0
|
||||
#25 3.764 go: downloading cloud.google.com/go v0.118.0
|
||||
#25 3.852 go: downloading github.com/klauspost/filepathx v1.1.1
|
||||
#25 3.855 go: downloading github.com/klauspost/pgzip v1.2.6
|
||||
#25 3.858 go: downloading github.com/klauspost/readahead v1.4.0
|
||||
#25 3.878 go: downloading github.com/klauspost/reedsolomon v1.12.4
|
||||
#25 3.878 go: downloading github.com/lithammer/shortuuid/v4 v4.2.0
|
||||
#25 4.002 go: downloading github.com/miekg/dns v1.1.63
|
||||
#25 4.183 go: downloading github.com/minio/cli v1.24.2
|
||||
#25 4.202 go: downloading github.com/minio/console v1.7.6
|
||||
#25 4.408 go: downloading github.com/minio/dnscache v0.1.1
|
||||
#25 4.409 go: downloading github.com/minio/dperf v0.6.3
|
||||
#25 4.438 go: downloading github.com/minio/highwayhash v1.0.3
|
||||
#25 4.584 go: downloading github.com/minio/kms-go/kes v0.3.1
|
||||
#25 4.654 go: downloading github.com/minio/madmin-go/v3 v3.0.109
|
||||
#25 5.132 go: downloading github.com/minio/minio-go/v7 v7.0.90
|
||||
#25 5.618 go: downloading github.com/minio/mux v1.9.2
|
||||
#25 5.620 go: downloading github.com/minio/pkg/v3 v3.1.0
|
||||
#25 6.131 go: downloading github.com/minio/selfupdate v0.6.0
|
||||
#25 6.131 go: downloading github.com/minio/xxml v0.0.3
|
||||
#25 6.131 go: downloading github.com/minio/sio v0.4.1
|
||||
#25 6.345 go: downloading github.com/minio/zipindex v0.4.0
|
||||
#25 6.354 go: downloading github.com/mitchellh/go-homedir v1.1.0
|
||||
#25 6.438 go: downloading github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c
|
||||
#25 7.265 go: downloading github.com/pierrec/lz4/v4 v4.1.22
|
||||
#25 7.871 go: downloading github.com/pkg/errors v0.9.1
|
||||
#25 8.014 go: downloading github.com/pkg/sftp v1.13.8
|
||||
#25 8.103 go: downloading github.com/pkg/xattr v0.4.10
|
||||
#25 8.840 go: downloading github.com/prometheus/client_golang v1.21.1
|
||||
#25 9.484 go: downloading github.com/prometheus/client_model v0.6.2
|
||||
#25 9.484 go: downloading github.com/prometheus/common v0.63.0
|
||||
#25 9.905 go: downloading github.com/prometheus/procfs v0.16.0
|
||||
#25 10.07 go: downloading github.com/puzpuzpuz/xsync/v3 v3.5.1
|
||||
#25 10.55 go: downloading github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475
|
||||
#25 10.61 go: downloading github.com/rs/cors v1.11.1
|
||||
#25 10.80 go: downloading github.com/secure-io/sio-go v0.3.1
|
||||
#25 10.80 go: downloading github.com/shirou/gopsutil/v3 v3.24.5
|
||||
#25 11.28 go: downloading github.com/tinylib/msgp v1.2.5
|
||||
#25 11.28 go: downloading github.com/valyala/bytebufferpool v1.0.0
|
||||
#25 11.59 go: downloading github.com/zeebo/xxh3 v1.0.2
|
||||
#25 11.94 go: downloading go.etcd.io/etcd/api/v3 v3.5.19
|
||||
#25 12.04 go: downloading go.etcd.io/etcd/client/v3 v3.5.19
|
||||
#25 12.37 go: downloading go.uber.org/atomic v1.11.0
|
||||
#25 12.60 go: downloading goftp.io/server/v2 v2.0.1
|
||||
#25 12.78 go: downloading golang.org/x/crypto v0.37.0
|
||||
#25 14.67 go: downloading golang.org/x/oauth2 v0.28.0
|
||||
#25 14.67 go: downloading golang.org/x/sync v0.13.0
|
||||
#25 14.89 go: downloading golang.org/x/sys v0.32.0
|
||||
#25 17.37 go: downloading golang.org/x/term v0.31.0
|
||||
#25 17.58 go: downloading google.golang.org/api v0.224.0
|
||||
#25 73.71 go: downloading gopkg.in/yaml.v2 v2.4.0
|
||||
#25 73.71 go: downloading gopkg.in/yaml.v3 v3.0.1
|
||||
#25 73.77 go: downloading github.com/mattn/go-runewidth v0.0.16
|
||||
#25 74.07 go: downloading github.com/go-jose/go-jose/v4 v4.0.5
|
||||
#25 74.09 go: downloading github.com/mattn/go-colorable v0.1.14
|
||||
#25 74.61 go: downloading github.com/mattn/go-isatty v0.0.20
|
||||
#25 74.67 go: downloading github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0
|
||||
#25 74.98 go: downloading github.com/AzureAD/microsoft-authentication-library-for-go v1.4.1
|
||||
#25 75.54 go: downloading github.com/google/pprof v0.0.0-20250302191652-9094ed2288e7
|
||||
#25 75.54 go: downloading github.com/go-openapi/analysis v0.23.0
|
||||
#25 75.54 go: downloading github.com/go-openapi/spec v0.21.0
|
||||
#25 78.43 go: downloading github.com/go-openapi/swag v0.23.1
|
||||
#25 79.26 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
|
||||
#25 80.95 go: downloading github.com/modern-go/reflect2 v1.0.2
|
||||
#25 81.17 go: downloading github.com/klauspost/cpuid/v2 v2.2.10
|
||||
#25 81.17 go: downloading golang.org/x/net v0.39.0
|
||||
#25 83.94 go: downloading cloud.google.com/go/compute/metadata v0.6.0
|
||||
#25 83.94 go: downloading cloud.google.com/go/iam v1.3.1
|
||||
#25 84.52 go: downloading github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.51.0
|
||||
#25 84.52 go: downloading github.com/googleapis/gax-go/v2 v2.14.1
|
||||
#25 84.89 go: downloading go.opentelemetry.io/contrib/detectors/gcp v1.35.0
|
||||
#25 85.08 go: downloading go.opentelemetry.io/otel v1.35.0
|
||||
#25 85.30 go: downloading go.opentelemetry.io/otel/sdk/metric v1.35.0
|
||||
#25 87.57 go: downloading go.opentelemetry.io/otel/sdk v1.35.0
|
||||
#25 107.7 go: downloading google.golang.org/genproto v0.0.0-20250106144421-5f5ef82da422
|
||||
#25 107.7 go: downloading google.golang.org/grpc v1.71.0
|
||||
#25 119.9 go: downloading google.golang.org/protobuf v1.36.6
|
||||
#25 125.0 go: downloading github.com/ncw/directio v1.0.5
|
||||
#25 125.0 go: downloading aead.dev/mem v0.2.0
|
||||
#25 125.0 go: downloading github.com/prometheus/prom2json v1.4.2
|
||||
#25 125.0 go: downloading github.com/safchain/ethtool v0.5.10
|
||||
#25 125.0 go: downloading github.com/goccy/go-json v0.10.5
|
||||
#25 125.0 go: downloading github.com/minio/crc64nvme v1.0.1
|
||||
#25 125.3 go: downloading github.com/go-ini/ini v1.67.0
|
||||
#25 125.3 go: downloading github.com/minio/md5-simd v1.1.2
|
||||
#25 125.4 go: downloading golang.org/x/time v0.11.0
|
||||
#25 125.4 go: downloading github.com/rs/xid v1.6.0
|
||||
#25 125.4 go: downloading github.com/beevik/ntp v1.4.3
|
||||
#25 125.4 go: downloading go.uber.org/zap v1.27.0
|
||||
#25 125.4 go: downloading github.com/go-ldap/ldap/v3 v3.4.10
|
||||
#25 125.4 go: downloading github.com/IBM/sarama v1.45.1
|
||||
#25 125.4 go: downloading github.com/eclipse/paho.mqtt.golang v1.5.0
|
||||
#25 125.4 go: downloading github.com/elastic/go-elasticsearch/v7 v7.17.10
|
||||
#25 125.7 go: downloading github.com/go-sql-driver/mysql v1.9.0
|
||||
#25 126.7 go: downloading github.com/gomodule/redigo v1.9.2
|
||||
#25 126.7 go: downloading github.com/nats-io/nats.go v1.39.1
|
||||
#25 126.7 go: downloading github.com/lib/pq v1.10.9
|
||||
#25 126.7 go: downloading github.com/nats-io/stan.go v0.10.4
|
||||
#25 128.4 go: downloading github.com/nsqio/go-nsq v1.1.0
|
||||
#25 128.4 go: downloading github.com/rabbitmq/amqp091-go v1.10.0
|
||||
#25 128.4 go: downloading github.com/xdg/scram v1.0.5
|
||||
#25 129.6 go: downloading github.com/gobwas/ws v1.4.0
|
||||
#25 130.0 go: downloading github.com/buger/jsonparser v1.1.1
|
||||
#25 130.0 go: downloading github.com/minio/kms-go/kms v0.4.0
|
||||
#25 130.0 go: downloading github.com/minio/simdjson-go v0.4.5
|
||||
#25 130.0 go: downloading github.com/rjeczalik/notify v0.9.3
|
||||
#25 130.0 go: downloading github.com/lestrrat-go/jwx/v2 v2.1.4
|
||||
#25 130.2 go: downloading aead.dev/minisign v0.3.0
|
||||
#25 130.2 go: downloading github.com/fatih/structs v1.1.0
|
||||
#25 131.5 go: downloading github.com/kr/fs v0.1.0
|
||||
#25 131.5 go: downloading github.com/beorn7/perks v1.0.1
|
||||
#25 131.5 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822
|
||||
#25 131.5 go: downloading github.com/tklauser/go-sysconf v0.3.15
|
||||
#25 131.5 go: downloading github.com/gogo/protobuf v1.3.2
|
||||
#25 131.5 go: downloading github.com/golang/protobuf v1.5.4
|
||||
#25 133.2 go: downloading go.etcd.io/etcd/client/pkg/v3 v3.5.19
|
||||
#25 135.7 go: downloading github.com/go-openapi/errors v0.22.0
|
||||
#25 135.7 go: downloading github.com/go-openapi/runtime v0.28.0
|
||||
#25 139.2 go: downloading github.com/jessevdk/go-flags v1.6.1
|
||||
#25 139.2 go: downloading github.com/minio/mc v0.0.0-20250312172924-c1d5d4cbb4ca
|
||||
#25 140.4 go: downloading github.com/go-openapi/strfmt v0.23.0
|
||||
#25 140.4 go: downloading github.com/minio/websocket v1.6.0
|
||||
#25 141.0 go: downloading github.com/unrolled/secure v1.17.0
|
||||
#25 142.8 go: downloading github.com/rivo/uniseg v0.4.7
|
||||
#25 142.9 go: downloading github.com/kylelemons/godebug v1.1.0
|
||||
#25 142.9 go: downloading github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c
|
||||
#25 142.9 go: downloading github.com/go-openapi/jsonpointer v0.21.1
|
||||
#25 142.9 go: downloading github.com/mailru/easyjson v0.9.0
|
||||
#25 142.9 go: downloading github.com/go-openapi/jsonreference v0.21.0
|
||||
#25 143.3 go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20250311190419-81fb87f6b8bf
|
||||
#25 144.5 go: downloading go.opentelemetry.io/otel/trace v1.35.0
|
||||
#25 144.9 go: downloading cloud.google.com/go/monitoring v1.23.0
|
||||
#25 144.9 go: downloading github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.51.0
|
||||
#25 144.9 go: downloading github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.27.0
|
||||
#25 145.0 go: downloading go.opentelemetry.io/otel/metric v1.35.0
|
||||
#25 145.7 go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20250311190419-81fb87f6b8bf
|
||||
#25 145.7 go: downloading cloud.google.com/go/auth v0.15.0
|
||||
#25 145.7 go: downloading cloud.google.com/go/auth/oauth2adapt v0.2.7
|
||||
#25 145.7 go: downloading go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0
|
||||
#25 145.7 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4
|
||||
#25 145.8 go: downloading go.uber.org/multierr v1.11.0
|
||||
#25 145.8 go: downloading github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358
|
||||
#25 145.9 go: downloading github.com/go-asn1-ber/asn1-ber v1.5.7
|
||||
#25 145.9 go: downloading github.com/nats-io/nuid v1.0.1
|
||||
#25 145.9 go: downloading github.com/gorilla/websocket v1.5.3
|
||||
#25 146.3 go: downloading filippo.io/edwards25519 v1.1.0
|
||||
#25 146.7 go: downloading github.com/eapache/go-resiliency v1.7.0
|
||||
#25 146.7 go: downloading github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3
|
||||
#25 146.7 go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
|
||||
#25 146.8 go: downloading github.com/eapache/queue v1.1.0
|
||||
#25 146.8 go: downloading github.com/jcmturner/gofork v1.7.6
|
||||
#25 146.8 go: downloading github.com/hashicorp/go-multierror v1.1.1
|
||||
#25 146.8 go: downloading github.com/jcmturner/gokrb5/v8 v8.4.4
|
||||
#25 147.4 go: downloading github.com/minio/csvparser v1.0.0
|
||||
#25 147.4 go: downloading github.com/fraugster/parquet-go v0.12.0
|
||||
#25 148.1 go: downloading github.com/alecthomas/participle v0.7.1
|
||||
#25 148.1 go: downloading github.com/golang/snappy v1.0.0
|
||||
#25 148.1 go: downloading github.com/xdg/stringprep v1.0.3
|
||||
#25 148.1 go: downloading github.com/nats-io/nkeys v0.4.10
|
||||
#25 148.1 go: downloading github.com/gobwas/httphead v0.1.0
|
||||
#25 148.1 go: downloading github.com/coreos/go-semver v0.3.1
|
||||
#25 148.1 go: downloading github.com/gobwas/pool v0.2.1
|
||||
#25 148.5 go: downloading github.com/lestrrat-go/option v1.0.1
|
||||
#25 148.5 go: downloading github.com/lestrrat-go/blackmagic v1.0.2
|
||||
#25 148.5 go: downloading github.com/lestrrat-go/httprc v1.0.6
|
||||
#25 148.5 go: downloading github.com/lestrrat-go/iter v1.0.2
|
||||
#25 148.6 go: downloading github.com/go-openapi/validate v0.24.0
|
||||
#25 148.6 go: downloading github.com/tklauser/numcpus v0.10.0
|
||||
#25 148.6 go: downloading github.com/golang-jwt/jwt/v5 v5.2.2
|
||||
#25 148.7 go: downloading github.com/docker/go-units v0.5.0
|
||||
#25 149.8 go: downloading github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2
|
||||
#25 149.8 go: downloading github.com/mitchellh/mapstructure v1.5.0
|
||||
#25 149.8 go: downloading go.mongodb.org/mongo-driver v1.17.3
|
||||
#25 149.8 go: downloading github.com/oklog/ulid v1.3.1
|
||||
#25 149.8 go: downloading github.com/go-logr/logr v1.4.2
|
||||
#25 149.8 go: downloading go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.59.0
|
||||
#25 149.8 go: downloading github.com/charmbracelet/bubbles v0.20.0
|
||||
#25 152.3 go: downloading github.com/charmbracelet/bubbletea v1.3.4
|
||||
#25 152.3 go: downloading github.com/charmbracelet/lipgloss v1.0.0
|
||||
#25 155.3 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
|
||||
#25 155.9 go: downloading github.com/jedib0t/go-pretty/v6 v6.6.7
|
||||
#25 155.9 go: downloading github.com/mattn/go-ieproxy v0.0.12
|
||||
#25 155.9 go: downloading github.com/minio/colorjson v1.0.8
|
||||
#25 155.9 go: downloading github.com/minio/filepath v1.0.0
|
||||
#25 155.9 go: downloading github.com/muesli/reflow v0.3.0
|
||||
#25 155.9 go: downloading github.com/muesli/termenv v0.16.0
|
||||
#25 156.0 go: downloading github.com/olekukonko/tablewriter v0.0.5
|
||||
#25 157.0 go: downloading github.com/posener/complete v1.2.3
|
||||
#25 157.0 go: downloading github.com/tidwall/gjson v1.18.0
|
||||
#25 157.0 go: downloading github.com/vbauerster/mpb/v8 v8.9.3
|
||||
#25 157.1 go: downloading golang.org/x/text v0.24.0
|
||||
#25 157.4 go: downloading github.com/josharian/intern v1.0.0
|
||||
#25 157.6 go: downloading go.opentelemetry.io/auto/sdk v1.1.0
|
||||
#25 157.6 go: downloading github.com/go-logr/stdr v1.2.2
|
||||
#25 157.6 go: downloading github.com/envoyproxy/go-control-plane/envoy v1.32.4
|
||||
#25 159.2 go: downloading github.com/google/s2a-go v0.1.9
|
||||
#25 161.0 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.3.5
|
||||
#25 161.1 go: downloading github.com/prometheus/prometheus v0.303.0
|
||||
#25 162.0 go: downloading github.com/felixge/httpsnoop v1.0.4
|
||||
#25 164.4 go: downloading github.com/hashicorp/errwrap v1.1.0
|
||||
#25 167.4 go: downloading github.com/jcmturner/dnsutils/v2 v2.0.0
|
||||
#25 169.4 go: downloading github.com/hashicorp/go-uuid v1.0.3
|
||||
#25 171.9 go: downloading github.com/apache/thrift v0.21.0
|
||||
#25 171.9 go: downloading github.com/lestrrat-go/httpcc v1.0.1
|
||||
#25 171.9 go: downloading github.com/charmbracelet/x/ansi v0.8.0
|
||||
#25 172.7 go: downloading github.com/charmbracelet/x/term v0.2.1
|
||||
#25 172.7 go: downloading github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6
|
||||
#25 176.9 go: downloading github.com/muesli/cancelreader v0.2.2
|
||||
#25 176.9 go: downloading github.com/juju/ratelimit v1.0.2
|
||||
#25 176.9 go: downloading github.com/aymanbagabas/go-osc52/v2 v2.0.1
|
||||
#25 177.0 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0
|
||||
#25 180.9 go: downloading github.com/cncf/xds/go v0.0.0-20250121191232-2f005788dc42
|
||||
#25 181.0 go: downloading github.com/tidwall/match v1.1.1
|
||||
#25 182.2 go: downloading github.com/tidwall/pretty v1.2.1
|
||||
#25 182.2 go: downloading github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
|
||||
#25 182.2 go: downloading github.com/VividCortex/ewma v1.2.0
|
||||
#25 182.3 go: downloading github.com/jcmturner/rpc/v2 v2.0.3
|
||||
#25 184.7 go: downloading github.com/envoyproxy/protoc-gen-validate v1.2.1
|
||||
#25 184.7 go: downloading github.com/jcmturner/aescts/v2 v2.0.0
|
||||
#25 184.7 go: downloading cel.dev/expr v0.22.0
|
||||
#25 DONE 256.1s
|
||||
|
||||
#30 [s3 stage-1 3/3] COPY --from=build /minio /usr/local/bin/minio
|
||||
#30 DONE 0.2s
|
||||
|
||||
#31 [s3] exporting to image
|
||||
#31 exporting layers
|
||||
#31 exporting layers 4.3s done
|
||||
#31 exporting manifest sha256:d50e9c88097122a227139120be4c032e21725a22317154dd69648b53dc84dc3f 0.0s done
|
||||
#31 exporting config sha256:c00e48fb21003e068aace42f534d64659fa837adab23e4674cc94bc79e3b1f97 0.0s done
|
||||
#31 exporting attestation manifest sha256:0481668ae887bdd025097902023c7d1cc45060c1cec382e07b282205c4276fe7 0.0s done
|
||||
#31 exporting manifest list sha256:b0483eceaa301523e808ae4f7572d4350f0695344334646c23c5f7bef07a718e 0.0s done
|
||||
#31 naming to docker.io/library/mtusi-minio:RELEASE.2025-04-22T22-12-26Z done
|
||||
#31 unpacking to docker.io/library/mtusi-minio:RELEASE.2025-04-22T22-12-26Z
|
||||
#31 unpacking to docker.io/library/mtusi-minio:RELEASE.2025-04-22T22-12-26Z 0.7s done
|
||||
#31 DONE 5.2s
|
||||
|
||||
#32 [s3] resolving provenance for metadata file
|
||||
#32 DONE 0.0s
|
||||
Image mtusi-messenger:lab4 Built
|
||||
Image mtusi-minio:RELEASE.2025-04-22T22-12-26Z Built
|
||||
Image mtusi-messenger:lab4 Built
|
||||
Image mtusi-messenger:lab4 Built
|
||||
Image mtusi-messenger:lab4 Built
|
||||
Network mtusi-lab4_default Creating
|
||||
Volume mtusi-lab4_pgdata Creating
|
||||
Volume mtusi-lab4_s3data Creating
|
||||
Volume mtusi-lab4_s3data Creating
|
||||
Volume mtusi-lab4_pgdata Creating
|
||||
Network mtusi-lab4_default Creating
|
||||
Volume mtusi-lab4_s3data Created
|
||||
Volume mtusi-lab4_s3data Created
|
||||
Volume mtusi-lab4_pgdata Created
|
||||
Volume mtusi-lab4_pgdata Created
|
||||
Network mtusi-lab4_default Created
|
||||
Network mtusi-lab4_default Created
|
||||
Container mtusi-lab4-db-1 Creating
|
||||
Container mtusi-lab4-s3-1 Creating
|
||||
Container mtusi-lab4-db-1 Created
|
||||
Container mtusi-lab4-migrate-1 Creating
|
||||
Container mtusi-lab4-s3-1 Created
|
||||
Container mtusi-lab4-s3-init-1 Creating
|
||||
Container mtusi-lab4-s3-init-1 Created
|
||||
Container mtusi-lab4-migrate-1 Created
|
||||
Container mtusi-lab4-api1-1 Creating
|
||||
Container mtusi-lab4-api2-1 Creating
|
||||
Container mtusi-lab4-api2-1 Created
|
||||
Container mtusi-lab4-api1-1 Created
|
||||
Container mtusi-lab4-lb-1 Creating
|
||||
Container mtusi-lab4-lb-1 Created
|
||||
Container mtusi-lab4-db-1 Starting
|
||||
Container mtusi-lab4-s3-1 Starting
|
||||
Container mtusi-lab4-db-1 Started
|
||||
Container mtusi-lab4-db-1 Waiting
|
||||
Container mtusi-lab4-s3-1 Started
|
||||
Container mtusi-lab4-s3-1 Waiting
|
||||
Container mtusi-lab4-s3-1 Healthy
|
||||
Container mtusi-lab4-s3-init-1 Starting
|
||||
Container mtusi-lab4-s3-init-1 Started
|
||||
Container mtusi-lab4-db-1 Healthy
|
||||
Container mtusi-lab4-migrate-1 Starting
|
||||
Container mtusi-lab4-migrate-1 Started
|
||||
Container mtusi-lab4-migrate-1 Waiting
|
||||
Container mtusi-lab4-s3-init-1 Waiting
|
||||
Container mtusi-lab4-migrate-1 Waiting
|
||||
Container mtusi-lab4-s3-init-1 Waiting
|
||||
Container mtusi-lab4-migrate-1 Exited
|
||||
Container mtusi-lab4-s3-init-1 Exited
|
||||
Container mtusi-lab4-api2-1 Starting
|
||||
Container mtusi-lab4-s3-init-1 Exited
|
||||
Container mtusi-lab4-migrate-1 Exited
|
||||
Container mtusi-lab4-api1-1 Starting
|
||||
Container mtusi-lab4-api2-1 Started
|
||||
Container mtusi-lab4-api1-1 Started
|
||||
Container mtusi-lab4-api1-1 Waiting
|
||||
Container mtusi-lab4-api2-1 Waiting
|
||||
Container mtusi-lab4-api2-1 Healthy
|
||||
Container mtusi-lab4-api1-1 Healthy
|
||||
Container mtusi-lab4-lb-1 Starting
|
||||
Container mtusi-lab4-lb-1 Started
|
||||
Container mtusi-lab4-api2-1 Waiting
|
||||
Container mtusi-lab4-lb-1 Waiting
|
||||
Container mtusi-lab4-db-1 Waiting
|
||||
Container mtusi-lab4-migrate-1 Waiting
|
||||
Container mtusi-lab4-s3-1 Waiting
|
||||
Container mtusi-lab4-s3-init-1 Waiting
|
||||
Container mtusi-lab4-api1-1 Waiting
|
||||
Container mtusi-lab4-migrate-1 Exited
|
||||
Container mtusi-lab4-s3-init-1 Exited
|
||||
Container mtusi-lab4-api1-1 Healthy
|
||||
Container mtusi-lab4-lb-1 Healthy
|
||||
Container mtusi-lab4-api2-1 Healthy
|
||||
Container mtusi-lab4-db-1 Healthy
|
||||
Container mtusi-lab4-s3-1 Healthy
|
||||
@@ -0,0 +1 @@
|
||||
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
|
||||
@@ -0,0 +1,8 @@
|
||||
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||
mtusi-lab4-api1-1 mtusi-messenger:lab4 "docker-entrypoint.s…" api1 5 minutes ago Up 2 seconds (healthy) 8080/tcp
|
||||
mtusi-lab4-api2-1 mtusi-messenger:lab4 "docker-entrypoint.s…" api2 5 minutes ago Up 2 minutes (healthy) 8080/tcp
|
||||
mtusi-lab4-db-1 postgres:17.6-alpine "docker-entrypoint.s…" db 5 minutes ago Up 2 minutes (healthy) 5432/tcp
|
||||
mtusi-lab4-lb-1 haproxy:3.0.12-alpine "docker-entrypoint.s…" lb 5 minutes ago Up 2 minutes 127.0.0.1:8084->8080/tcp
|
||||
mtusi-lab4-migrate-1 mtusi-messenger:lab4 "docker-entrypoint.s…" migrate 5 minutes ago Exited (0) 5 minutes ago
|
||||
mtusi-lab4-s3-1 mtusi-minio:RELEASE.2025-04-22T22-12-26Z "minio server /data …" s3 5 minutes ago Up 2 minutes (healthy)
|
||||
mtusi-lab4-s3-init-1 mtusi-messenger:lab4 "docker-entrypoint.s…" s3-init 5 minutes ago Exited (0) 5 minutes ago
|
||||
@@ -0,0 +1 @@
|
||||
PASS: 17 original contract/test/task files byte-identical to supplied archive.
|
||||
@@ -0,0 +1 @@
|
||||
Docker=29.9.2; CPUs=12; MemoryBytes=8259383296; OS=Docker Desktop
|
||||
@@ -0,0 +1,3 @@
|
||||
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
|
||||
{"observed_instances_before": 2, "successful_writes": 6, "failed_attempts": 1, "first_recovery_seconds": 5.156}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
@@ -0,0 +1,2 @@
|
||||
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
|
||||
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"before": [
|
||||
{
|
||||
"service": "api1",
|
||||
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:45:29.802007824Z",
|
||||
"finished_at": "2026-11-13T09:45:23.458360231Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "api2",
|
||||
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:45:29.976703587Z",
|
||||
"finished_at": "2026-11-13T09:45:23.459942874Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "db",
|
||||
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:45:24.574681933Z",
|
||||
"finished_at": "2026-11-13T09:45:23.879037942Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_pgdata"
|
||||
]
|
||||
},
|
||||
{
|
||||
"service": "lb",
|
||||
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:45:33.091747442Z",
|
||||
"finished_at": "2026-11-13T09:45:23.177520781Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "s3",
|
||||
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:45:40.582059708Z",
|
||||
"finished_at": "2026-11-13T09:45:34.475313953Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_s3data"
|
||||
]
|
||||
}
|
||||
],
|
||||
"after": [
|
||||
{
|
||||
"service": "api1",
|
||||
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:44.323950141Z",
|
||||
"finished_at": "2026-11-13T09:47:37.999368896Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "api2",
|
||||
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:44.186050153Z",
|
||||
"finished_at": "2026-11-13T09:47:37.999033921Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "db",
|
||||
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:39.127938865Z",
|
||||
"finished_at": "2026-11-13T09:47:38.306703924Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_pgdata"
|
||||
]
|
||||
},
|
||||
{
|
||||
"service": "lb",
|
||||
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:47.418593695Z",
|
||||
"finished_at": "2026-11-13T09:47:37.689985726Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "s3",
|
||||
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:38.986060426Z",
|
||||
"finished_at": "2026-11-13T09:47:38.274698815Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_s3data"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
|
||||
@@ -0,0 +1,96 @@
|
||||
{
|
||||
"target_instance": "a003f158-ce85-43e5-b7b2-ce885eb0fc8a",
|
||||
"selected_service": "api1",
|
||||
"before": [
|
||||
{
|
||||
"service": "api1",
|
||||
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:44.323950141Z",
|
||||
"finished_at": "2026-11-13T09:47:37.999368896Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "api2",
|
||||
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:44.186050153Z",
|
||||
"finished_at": "2026-11-13T09:47:37.999033921Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "db",
|
||||
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:39.127938865Z",
|
||||
"finished_at": "2026-11-13T09:47:38.306703924Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_pgdata"
|
||||
]
|
||||
},
|
||||
{
|
||||
"service": "lb",
|
||||
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:47.418593695Z",
|
||||
"finished_at": "2026-11-13T09:47:37.689985726Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "s3",
|
||||
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:38.986060426Z",
|
||||
"finished_at": "2026-11-13T09:47:38.274698815Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_s3data"
|
||||
]
|
||||
}
|
||||
],
|
||||
"after": [
|
||||
{
|
||||
"service": "api1",
|
||||
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
|
||||
"running": false,
|
||||
"started_at": "2026-11-13T09:47:44.323950141Z",
|
||||
"finished_at": "2026-11-13T09:49:43.202703028Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "api2",
|
||||
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:44.186050153Z",
|
||||
"finished_at": "2026-11-13T09:47:37.999033921Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "db",
|
||||
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:39.127938865Z",
|
||||
"finished_at": "2026-11-13T09:47:38.306703924Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_pgdata"
|
||||
]
|
||||
},
|
||||
{
|
||||
"service": "lb",
|
||||
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:47.418593695Z",
|
||||
"finished_at": "2026-11-13T09:47:37.689985726Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "s3",
|
||||
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:47:38.986060426Z",
|
||||
"finished_at": "2026-11-13T09:47:38.274698815Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_s3data"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
{
|
||||
"target_instance": "b770ec13-25b7-435b-a024-7d805a1c8798",
|
||||
"selected_service": "api2",
|
||||
"before": [
|
||||
{
|
||||
"service": "api1",
|
||||
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:53.775499857Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "api2",
|
||||
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:53.604840647Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "db",
|
||||
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:48.260190707Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_pgdata"
|
||||
]
|
||||
},
|
||||
{
|
||||
"service": "lb",
|
||||
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:56.441675642Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "s3",
|
||||
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:48.465999336Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_s3data"
|
||||
]
|
||||
}
|
||||
],
|
||||
"after": [
|
||||
{
|
||||
"service": "api1",
|
||||
"container": "8a939e77ecc8aa147c42e984c23d22b9014de6fc6fe435685dd33aef57533591",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:53.775499857Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "api2",
|
||||
"container": "61a1827c715014066ea9cd94aa79ada0b511048e9058757ad400d333d88b4eb8",
|
||||
"running": false,
|
||||
"started_at": "2026-11-13T09:44:53.604840647Z",
|
||||
"finished_at": "2026-11-13T09:45:11.765422268Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "db",
|
||||
"container": "7fbb25ba0dd7dac7ef6cd725177bab96397226afbc8a868ac06e1c0e10dc274f",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:48.260190707Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_pgdata"
|
||||
]
|
||||
},
|
||||
{
|
||||
"service": "lb",
|
||||
"container": "feb81abd9faaab0c2c890c1142784e03354a7f20a52a3face2426005a86eeb6c",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:56.441675642Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": []
|
||||
},
|
||||
{
|
||||
"service": "s3",
|
||||
"container": "bca4d0d5a74c482513fb256de70d99b8fc61c433f55a16932875f3798502c51b",
|
||||
"running": true,
|
||||
"started_at": "2026-11-13T09:44:48.465999336Z",
|
||||
"finished_at": "0001-01-01T00:00:00Z",
|
||||
"volumes": [
|
||||
"mtusi-lab4_s3data"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
PASS: PNG/JPEG exact bytes and SHA-256; private bucket 403; member access; invalid filename/MIME/corrupt file; >10 MiB including chunked and >40M pixels; attachment scope; 12 concurrent sends make 1 message.
|
||||
{"scenario":"upload-stop-uploader-download","recovery_ms":4054,"same_bytes":true}
|
||||
{"scenario":"upload-stop-uploader-download","recovery_ms":2199,"same_bytes":true}
|
||||
PASS: full restart preserves metadata, original bytes, session and idempotent attachment message.
|
||||
PASS: stopped S3 makes API readiness 503 while liveness remains 200.
|
||||
PASS: S3 recovery restores download without data loss.
|
||||
@@ -0,0 +1,11 @@
|
||||
FROM golang:1.24.3-alpine AS build
|
||||
RUN apk add --no-cache git || apk add --no-cache git || apk add --no-cache git
|
||||
RUN git clone --depth 1 --branch RELEASE.2025-04-22T22-12-26Z https://github.com/minio/minio.git /src
|
||||
WORKDIR /src
|
||||
RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build \
|
||||
for attempt in 1 2 3 4 5; do CGO_ENABLED=0 go build -trimpath -o /minio . && exit 0; done; exit 1
|
||||
|
||||
FROM alpine:3.22.1
|
||||
RUN apk add --no-cache ca-certificates curl
|
||||
COPY --from=build /minio /usr/local/bin/minio
|
||||
ENTRYPOINT ["minio"]
|
||||
@@ -0,0 +1,36 @@
|
||||
global
|
||||
log stdout format raw local0
|
||||
|
||||
defaults
|
||||
mode http
|
||||
log global
|
||||
timeout connect 1s
|
||||
timeout client 15s
|
||||
timeout server 15s
|
||||
timeout check 1s
|
||||
retries 0
|
||||
|
||||
resolvers docker
|
||||
nameserver dns 127.0.0.11:53
|
||||
resolve_retries 3
|
||||
timeout resolve 1s
|
||||
timeout retry 1s
|
||||
hold valid 1s
|
||||
hold nx 1s
|
||||
hold timeout 1s
|
||||
hold refused 1s
|
||||
hold other 1s
|
||||
hold obsolete 1s
|
||||
|
||||
frontend public
|
||||
bind :8080
|
||||
default_backend messenger
|
||||
|
||||
backend messenger
|
||||
balance roundrobin
|
||||
option httpchk
|
||||
http-check send meth GET uri /health/ready ver HTTP/1.1 hdr Host localhost
|
||||
http-check expect status 200
|
||||
default-server check inter 1s fall 2 rise 1 resolvers docker init-addr last,libc,none
|
||||
server api1 api1:8080
|
||||
server api2 api2:8080
|
||||
@@ -0,0 +1,27 @@
|
||||
# Хранение данных lab4
|
||||
|
||||
PostgreSQL хранит пользователей, хеши паролей, хеши сессий, чаты, участников,
|
||||
сообщения, результаты идемпотентных запросов и метаданные вложений.
|
||||
Миграции 001–002 перенесены из lab3; миграция 003 добавляет вложения.
|
||||
|
||||
`attachments`: UUID, чат, владелец, отображаемое имя, MIME, размер, SHA-256,
|
||||
случайный object_key и время создания/изменения. Object key наружу не выдаётся.
|
||||
`message_attachments`: связь сообщения и вложения, позиция в списке.
|
||||
Байты оригинала находятся только в приватном bucket `messenger-attachments`.
|
||||
API использует ограниченный буфер в RAM, не пишет файл на диск.
|
||||
|
||||
Порядок upload: авторизация → проверка имени/размера/MIME → полное декодирование
|
||||
с лимитом пикселей → S3 PUT → INSERT метаданных → 201 ready.
|
||||
Ошибка INSERT вызывает попытку удалить объект. Общей транзакции S3/PostgreSQL нет:
|
||||
при аварии между операциями возможен лишний объект. Автоматический reconciler
|
||||
не реализован (целевая оценка 3).
|
||||
|
||||
Отправка сообщения и запись результата идемпотентности выполняются одной
|
||||
транзакцией PostgreSQL, с общей блокировкой по пользователю, чату и ключу.
|
||||
Все attachment_ids проверяются на принадлежность чату; порядок сохраняется.
|
||||
Отсутствующий attachment_ids равнозначен пустому списку.
|
||||
|
||||
У API нет собственного volume. PostgreSQL использует pgdata, MinIO — s3data.
|
||||
Оба API читают одни и те же данные. S3 и консоль доступны только внутри Compose.
|
||||
Bootstrap создаёт bucket и удаляет анонимную policy. Для учебного уровня 3
|
||||
API использует общие административные S3 credentials из локальных secret-файлов.
|
||||
@@ -0,0 +1,35 @@
|
||||
CREATE TABLE users (
|
||||
id uuid PRIMARY KEY,
|
||||
username text NOT NULL UNIQUE CHECK (username ~ '^[a-z0-9_]{3,32}$'),
|
||||
display_name text NOT NULL CHECK (char_length(display_name) BETWEEN 1 AND 100),
|
||||
password_hash text NOT NULL,
|
||||
created_at timestamptz(3) NOT NULL DEFAULT clock_timestamp()
|
||||
);
|
||||
CREATE TABLE chats (
|
||||
id uuid PRIMARY KEY,
|
||||
title text NOT NULL CHECK (char_length(title) BETWEEN 1 AND 100),
|
||||
created_at timestamptz(3) NOT NULL DEFAULT clock_timestamp()
|
||||
);
|
||||
CREATE TABLE chat_members (
|
||||
chat_id uuid NOT NULL REFERENCES chats(id),
|
||||
user_id uuid NOT NULL REFERENCES users(id),
|
||||
PRIMARY KEY (chat_id, user_id)
|
||||
);
|
||||
CREATE TABLE messages (
|
||||
id uuid PRIMARY KEY,
|
||||
chat_id uuid NOT NULL,
|
||||
sender_id uuid NOT NULL,
|
||||
text text NOT NULL CHECK (char_length(text) BETWEEN 1 AND 4000),
|
||||
created_at timestamptz(3) NOT NULL DEFAULT clock_timestamp(),
|
||||
FOREIGN KEY (chat_id, sender_id) REFERENCES chat_members(chat_id, user_id)
|
||||
);
|
||||
CREATE TABLE sessions (
|
||||
token_hash text PRIMARY KEY,
|
||||
user_id uuid NOT NULL REFERENCES users(id),
|
||||
expires_at timestamptz(3) NOT NULL
|
||||
);
|
||||
CREATE TABLE app_settings (key text PRIMARY KEY, value text NOT NULL);
|
||||
CREATE INDEX messages_history ON messages(chat_id, created_at, id);
|
||||
CREATE INDEX chats_order ON chats(created_at, id);
|
||||
CREATE INDEX member_chats ON chat_members(user_id, chat_id);
|
||||
CREATE INDEX sessions_expiry ON sessions(expires_at);
|
||||
@@ -0,0 +1,11 @@
|
||||
CREATE TABLE message_requests (
|
||||
user_id uuid NOT NULL REFERENCES users(id),
|
||||
chat_id uuid NOT NULL REFERENCES chats(id),
|
||||
key text NOT NULL,
|
||||
text text NOT NULL,
|
||||
message_id uuid NOT NULL REFERENCES messages(id),
|
||||
response jsonb NOT NULL,
|
||||
expires_at timestamptz NOT NULL,
|
||||
PRIMARY KEY (user_id, chat_id, key)
|
||||
);
|
||||
CREATE INDEX message_requests_expiry ON message_requests(expires_at);
|
||||
@@ -0,0 +1,25 @@
|
||||
CREATE TABLE attachments (
|
||||
id uuid PRIMARY KEY,
|
||||
chat_id uuid NOT NULL REFERENCES chats(id),
|
||||
owner_id uuid NOT NULL REFERENCES users(id),
|
||||
filename text NOT NULL CHECK (char_length(filename) BETWEEN 1 AND 255),
|
||||
content_type text NOT NULL CHECK (content_type IN ('image/png','image/jpeg')),
|
||||
size_bytes integer NOT NULL CHECK (size_bytes BETWEEN 1 AND 10485760),
|
||||
sha256 text NOT NULL,
|
||||
object_key text NOT NULL UNIQUE,
|
||||
created_at timestamptz(3) NOT NULL DEFAULT clock_timestamp(),
|
||||
updated_at timestamptz(3) NOT NULL DEFAULT clock_timestamp(),
|
||||
UNIQUE (id,chat_id)
|
||||
);
|
||||
CREATE INDEX attachments_chat ON attachments(chat_id);
|
||||
ALTER TABLE messages DROP CONSTRAINT messages_text_check;
|
||||
ALTER TABLE messages ADD CONSTRAINT messages_text_check CHECK (char_length(text) BETWEEN 0 AND 4000);
|
||||
CREATE TABLE message_attachments (
|
||||
message_id uuid NOT NULL REFERENCES messages(id),
|
||||
attachment_id uuid NOT NULL REFERENCES attachments(id),
|
||||
position integer NOT NULL,
|
||||
PRIMARY KEY (message_id,attachment_id),
|
||||
UNIQUE (message_id,position)
|
||||
);
|
||||
ALTER TABLE message_requests ADD COLUMN attachment_ids uuid[] NOT NULL DEFAULT '{}';
|
||||
UPDATE message_requests SET response = response || '{"attachment_ids":[]}'::jsonb;
|
||||
Generated
+1449
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"name": "mtusi-lab4-messenger",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"start": "node src/server.js",
|
||||
"migrate": "node src/migrate.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"pg": "8.16.3",
|
||||
"minio": "8.0.5",
|
||||
"sharp": "0.34.3"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
"""Explicit lab4-only lifecycle hooks. Never delete volumes or print credentials."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
os.chdir(ROOT)
|
||||
|
||||
def docker(*args):
|
||||
return subprocess.check_output(['docker', *args], text=True).strip()
|
||||
|
||||
def compose(*args):
|
||||
return docker('compose', *args)
|
||||
|
||||
def snapshot():
|
||||
result = []
|
||||
containers = compose('ps', '-a', '-q').splitlines()
|
||||
if not containers:
|
||||
return result
|
||||
# One bulk inspect keeps the stop hook well below its 10-second budget.
|
||||
# Only these selected fields are written; Config/Env are never logged.
|
||||
for container in json.loads(docker('inspect', *containers)):
|
||||
service = container['Config']['Labels']['com.docker.compose.service']
|
||||
if service not in ('lb', 'api1', 'api2', 'db', 's3'):
|
||||
continue
|
||||
state, mounts = container['State'], container['Mounts']
|
||||
result.append({'service': service, 'container': container['Id'], 'running': state['Running'],
|
||||
'started_at': state['StartedAt'], 'finished_at': state['FinishedAt'],
|
||||
'volumes': [m['Name'] for m in mounts if m['Type'] == 'volume']})
|
||||
return result
|
||||
|
||||
def save(name, value):
|
||||
(ROOT / 'evidence').mkdir(exist_ok=True)
|
||||
(ROOT / 'evidence' / name).write_text(json.dumps(value, indent=2) + '\n', encoding='utf-8')
|
||||
|
||||
def main():
|
||||
action = sys.argv[1]
|
||||
if action == 'restart':
|
||||
before = snapshot()
|
||||
compose('stop', 'lb', 'api1', 'api2', 'db', 's3')
|
||||
compose('up', '-d', '--wait', '--wait-timeout', '30', 'db', 's3')
|
||||
compose('run', '--rm', '--no-deps', 's3-init')
|
||||
compose('run', '--rm', '--no-deps', 'migrate')
|
||||
compose('up', '-d', '--no-deps', '--wait', '--wait-timeout', '30', 'api1', 'api2')
|
||||
compose('up', '-d', '--no-deps', 'lb')
|
||||
save('restart-scope.json', {'before': before, 'after': snapshot()})
|
||||
elif action == 'stop-one':
|
||||
target = os.environ.get('TARGET_INSTANCE_ID')
|
||||
if not target:
|
||||
raise RuntimeError('TARGET_INSTANCE_ID required')
|
||||
before = snapshot()
|
||||
selected = None
|
||||
for service in ('api1', 'api2'):
|
||||
code = "fetch('http://127.0.0.1:8080/health/live').then(r=>console.log(r.headers.get('x-instance-id'))).catch(()=>process.exit(1))"
|
||||
instance = compose('exec', '-T', service, 'node', '-e', code)
|
||||
if instance == target:
|
||||
selected = service
|
||||
break
|
||||
if selected is None:
|
||||
raise RuntimeError('Target is not a running API in this lab')
|
||||
compose('stop', '-t', '2', selected)
|
||||
after = snapshot()
|
||||
assert not next(s for s in after if s['service'] == selected)['running']
|
||||
assert all(s['running'] for s in after if s['service'] != selected)
|
||||
save('stop-' + selected + '.json', {'target_instance': target, 'selected_service': selected, 'before': before, 'after': after})
|
||||
elif action == 'restore':
|
||||
compose('up', '-d', '--no-deps', '--wait', '--wait-timeout', '30', 'api1', 'api2')
|
||||
else:
|
||||
raise RuntimeError('Unknown action')
|
||||
print('PASS: ' + action + ' completed for mtusi-lab4')
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,7 @@
|
||||
@echo off
|
||||
if defined LAB_PYTHON (
|
||||
"%LAB_PYTHON%" -X utf8 "%~dp0control.py" restart
|
||||
) else (
|
||||
py -X utf8 "%~dp0control.py" restart
|
||||
)
|
||||
exit /b %errorlevel%
|
||||
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
exec python3 scripts/control.py restart
|
||||
@@ -0,0 +1,26 @@
|
||||
"""Generate local credentials once; never print or rotate existing secrets."""
|
||||
from pathlib import Path
|
||||
import secrets
|
||||
import shutil
|
||||
|
||||
root = Path(__file__).resolve().parents[1]
|
||||
directory = root / 'secrets'
|
||||
directory.mkdir(exist_ok=True)
|
||||
password_file = directory / 'postgres_password'
|
||||
url_file = directory / 'database_url'
|
||||
if password_file.exists() != url_file.exists():
|
||||
raise SystemExit('Incomplete secrets: restore the missing file; do not rotate an existing database password.')
|
||||
if not password_file.exists():
|
||||
password = secrets.token_hex(32)
|
||||
password_file.write_text(password + '\n', encoding='utf-8')
|
||||
url_file.write_text('postgresql://messenger:' + password + '@db:5432/messenger\n', encoding='utf-8')
|
||||
password_file.chmod(0o600)
|
||||
url_file.chmod(0o600)
|
||||
for filename in ('s3_access_key_id', 's3_secret_access_key'):
|
||||
destination = directory / filename
|
||||
if not destination.exists():
|
||||
destination.write_text(secrets.token_hex(10 if filename == 's3_access_key_id' else 24) + '\n', encoding='utf-8')
|
||||
destination.chmod(0o600)
|
||||
if not (root / '.env').exists():
|
||||
shutil.copyfile(root / '.env.example', root / '.env')
|
||||
print('Local .env and secrets ready; values are not printed. Existing credentials preserved.')
|
||||
@@ -0,0 +1,7 @@
|
||||
@echo off
|
||||
if defined LAB_PYTHON (
|
||||
"%LAB_PYTHON%" -X utf8 "%~dp0control.py" stop-one
|
||||
) else (
|
||||
py -X utf8 "%~dp0control.py" stop-one
|
||||
)
|
||||
exit /b %errorlevel%
|
||||
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
exec python3 scripts/control.py stop-one
|
||||
@@ -0,0 +1,97 @@
|
||||
'use strict';
|
||||
const sharp = require('sharp');
|
||||
const { randomUUID, createHash } = require('node:crypto');
|
||||
const { pool } = require('./db');
|
||||
const { client, bucket, unavailable } = require('./storage');
|
||||
const MAX_BYTES = 10485760, MAX_PIXELS = 40000000;
|
||||
sharp.cache(false);
|
||||
function fail(status, code, message) { throw Object.assign(new Error(message), { status, code }); }
|
||||
function visible(row) {
|
||||
const { object_key, ...data } = row;
|
||||
return { ...data, status: 'ready', variants: [], error_code: null };
|
||||
}
|
||||
function readLimited(req) {
|
||||
return new Promise((resolve, reject) => {
|
||||
let size = 0, failed = false;
|
||||
const chunks = [];
|
||||
const tooLarge = () => {
|
||||
failed = true;
|
||||
reject(Object.assign(new Error('File exceeds 10 MiB'), { status: 413, code: 'payload_too_large' }));
|
||||
};
|
||||
if (Number(req.headers['content-length'] || 0) > MAX_BYTES) { req.resume(); tooLarge(); return; }
|
||||
req.on('data', chunk => {
|
||||
if (failed) return;
|
||||
size += chunk.length;
|
||||
if (size > MAX_BYTES) { chunks.length = 0; tooLarge(); }
|
||||
else chunks.push(chunk);
|
||||
});
|
||||
req.on('end', () => { if (!failed) resolve(Buffer.concat(chunks)); });
|
||||
req.on('error', () => reject(Object.assign(new Error('Upload interrupted'), { status: 400, code: 'invalid_request' })));
|
||||
});
|
||||
}
|
||||
async function validateImage(bytes, contentType) {
|
||||
// Reject other formats before asking the image decoder to process them.
|
||||
const png = bytes.length >= 8 && bytes.subarray(0, 8).equals(Buffer.from([137,80,78,71,13,10,26,10]));
|
||||
const jpeg = bytes.length >= 3 && bytes[0] === 255 && bytes[1] === 216 && bytes[2] === 255;
|
||||
if ((!png && !jpeg) || (png ? 'image/png' : 'image/jpeg') !== contentType)
|
||||
fail(415, 'unsupported_media_type', 'Expected a matching PNG or JPEG');
|
||||
try {
|
||||
const decoder = sharp(bytes, { limitInputPixels: MAX_PIXELS, failOn: 'warning' });
|
||||
const info = await decoder.metadata();
|
||||
if (!info.width || !info.height || info.width * info.height > MAX_PIXELS)
|
||||
fail(413, 'payload_too_large', 'Image exceeds 40 million pixels');
|
||||
if (!['png', 'jpeg'].includes(info.format)) fail(415, 'unsupported_media_type', 'Unsupported image');
|
||||
// Full decoding detects truncated/corrupt payloads; the raw result is discarded.
|
||||
// Pixel limit is enforced before allocation. Original bytes are never transformed.
|
||||
await decoder.raw().toBuffer();
|
||||
} catch (error) {
|
||||
if (error.status) throw error;
|
||||
if (/pixel limit/i.test(error.message)) fail(413, 'payload_too_large', 'Image exceeds 40 million pixels');
|
||||
fail(415, 'unsupported_media_type', 'Corrupt image');
|
||||
}
|
||||
}
|
||||
async function upload(req, query, chat, user) {
|
||||
const filename = query.get('filename');
|
||||
if (query.getAll('filename').length !== 1 || !filename || !filename.isWellFormed() ||
|
||||
/^[\p{White_Space}\uFEFF]*$/u.test(filename) || [...filename].length > 255 || /[\/\\\p{Cc}]/u.test(filename))
|
||||
fail(400, 'invalid_request', 'Invalid filename');
|
||||
const contentType = req.headers['content-type']?.split(';')[0].trim().toLowerCase();
|
||||
if (!['image/png', 'image/jpeg'].includes(contentType)) fail(415, 'unsupported_media_type', 'Expected PNG or JPEG');
|
||||
const bytes = await readLimited(req);
|
||||
await validateImage(bytes, contentType);
|
||||
const id = randomUUID(), objectKey = 'originals/' + randomUUID();
|
||||
const checksum = createHash('sha256').update(bytes).digest('hex');
|
||||
try { await client.putObject(bucket, objectKey, bytes, bytes.length, { 'Content-Type': contentType }); }
|
||||
catch { throw unavailable(); }
|
||||
try {
|
||||
const result = await pool.query(`INSERT INTO attachments(id,chat_id,owner_id,filename,content_type,size_bytes,sha256,object_key)
|
||||
VALUES($1,$2,$3,$4,$5,$6,$7,$8) RETURNING *`, [id, chat.id, user.id, filename, contentType, bytes.length, checksum, objectKey]);
|
||||
return visible(result.rows[0]);
|
||||
} catch (error) {
|
||||
// Best-effort compensation. A process crash between stores still needs a reconciler (not grade 3).
|
||||
await client.removeObject(bucket, objectKey).catch(() => console.error('Object compensation failed'));
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
async function accessible(id, user) {
|
||||
const result = await pool.query(`SELECT a.* FROM attachments a WHERE a.id=$1 AND EXISTS
|
||||
(SELECT 1 FROM chat_members WHERE chat_id=a.chat_id AND user_id=$2)`, [id, user.id]);
|
||||
if (!result.rowCount) fail(404, 'not_found', 'Object not found');
|
||||
return result.rows[0];
|
||||
}
|
||||
async function download(row) {
|
||||
try {
|
||||
const stream = await client.getObject(bucket, row.object_key);
|
||||
const chunks = [];
|
||||
let size = 0;
|
||||
for await (const chunk of stream) {
|
||||
size += chunk.length;
|
||||
if (size > MAX_BYTES) { stream.destroy(); throw new Error(); }
|
||||
chunks.push(chunk);
|
||||
}
|
||||
const bytes = Buffer.concat(chunks);
|
||||
if (bytes.length !== row.size_bytes || createHash('sha256').update(bytes).digest('hex') !== row.sha256) throw new Error();
|
||||
return bytes;
|
||||
} catch { throw unavailable(); }
|
||||
}
|
||||
module.exports = { upload, accessible, visible, download, validateImage };
|
||||
@@ -0,0 +1,22 @@
|
||||
'use strict';
|
||||
const { readFileSync } = require('node:fs');
|
||||
const { Pool } = require('pg');
|
||||
const pool = new Pool({
|
||||
connectionString: readFileSync(process.env.DATABASE_URL_FILE || '/run/secrets/database_url', 'utf8').trim(),
|
||||
max: 10, connectionTimeoutMillis: 3000, idleTimeoutMillis: 10000,
|
||||
statement_timeout: 5000,
|
||||
});
|
||||
pool.on('error', () => console.error('Database connection unavailable'));
|
||||
async function transaction(work) {
|
||||
const client = await pool.connect();
|
||||
try {
|
||||
await client.query('BEGIN');
|
||||
const result = await work(client);
|
||||
await client.query('COMMIT');
|
||||
return result;
|
||||
} catch (error) {
|
||||
await client.query('ROLLBACK').catch(() => {});
|
||||
throw error;
|
||||
} finally { client.release(); }
|
||||
}
|
||||
module.exports = { pool, transaction };
|
||||
@@ -0,0 +1,36 @@
|
||||
'use strict';
|
||||
const { randomUUID } = require('node:crypto');
|
||||
const { transaction } = require('./db');
|
||||
const retention = Number(process.env.IDEMPOTENCY_TTL_SECONDS || 86400);
|
||||
if (!Number.isSafeInteger(retention) || retention < 86400) throw new Error('Idempotency retention must be at least 24 hours');
|
||||
|
||||
async function sendMessage(userId, chatId, key, text, attachmentIds = []) {
|
||||
return transaction(async client => {
|
||||
const attached = await client.query('SELECT id FROM attachments WHERE chat_id=$1 AND id=ANY($2::uuid[])', [chatId, attachmentIds]);
|
||||
if (attached.rowCount !== attachmentIds.length)
|
||||
throw Object.assign(new Error('Object not found'), { status: 404, code: 'not_found' });
|
||||
// Shared PostgreSQL lock serializes this operation across all API replicas.
|
||||
// Hash collisions only serialize unrelated keys; the actual lookup uses the full scope.
|
||||
const scope = JSON.stringify([userId, chatId, key]);
|
||||
await client.query('SELECT pg_advisory_xact_lock(hashtextextended($1, 0))', [scope]);
|
||||
await client.query(`DELETE FROM message_requests WHERE user_id=$1 AND chat_id=$2 AND key=$3
|
||||
AND expires_at <= clock_timestamp()`, [userId, chatId, key]);
|
||||
const prior = await client.query('SELECT text,attachment_ids,response FROM message_requests WHERE user_id=$1 AND chat_id=$2 AND key=$3', [userId, chatId, key]);
|
||||
if (prior.rowCount) {
|
||||
if (prior.rows[0].text !== text || JSON.stringify(prior.rows[0].attachment_ids) !== JSON.stringify(attachmentIds))
|
||||
throw Object.assign(new Error('Key already used for another message'), { status: 409, code: 'conflict' });
|
||||
return prior.rows[0].response;
|
||||
}
|
||||
const message = (await client.query(`INSERT INTO messages(id,chat_id,sender_id,text)
|
||||
VALUES($1,$2,$3,$4) RETURNING *`, [randomUUID(), chatId, userId, text])).rows[0];
|
||||
for (let i = 0; i < attachmentIds.length; i++)
|
||||
await client.query('INSERT INTO message_attachments VALUES($1,$2,$3)', [message.id, attachmentIds[i], i]);
|
||||
message.attachment_ids = attachmentIds;
|
||||
await client.query(`INSERT INTO message_requests(user_id,chat_id,key,text,message_id,response,expires_at,attachment_ids)
|
||||
VALUES($1,$2,$3,$4,$5,$6::jsonb,clock_timestamp()+$7*interval '1 second',$8::uuid[])`,
|
||||
[userId, chatId, key, text, message.id, JSON.stringify(message), retention, attachmentIds]);
|
||||
// The HTTP handler returns 201 only after transaction() completes COMMIT.
|
||||
return message;
|
||||
});
|
||||
}
|
||||
module.exports = { sendMessage };
|
||||
@@ -0,0 +1,37 @@
|
||||
'use strict';
|
||||
const { readFileSync, readdirSync } = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const { createHash, randomBytes } = require('node:crypto');
|
||||
const { pool } = require('./db');
|
||||
async function migrate() {
|
||||
const client = await pool.connect();
|
||||
try {
|
||||
await client.query('SELECT pg_advisory_lock(202602)');
|
||||
await client.query(`CREATE TABLE IF NOT EXISTS schema_migrations
|
||||
(version text PRIMARY KEY, checksum text NOT NULL)`);
|
||||
const dir = path.join(__dirname, '..', 'migrations');
|
||||
for (const version of readdirSync(dir).filter(f => /^\d+.*\.sql$/.test(f)).sort()) {
|
||||
const sql = readFileSync(path.join(dir, version), 'utf8');
|
||||
const checksum = createHash('sha256').update(sql).digest('hex');
|
||||
const existing = await client.query('SELECT checksum FROM schema_migrations WHERE version=$1', [version]);
|
||||
if (existing.rowCount) {
|
||||
if (existing.rows[0].checksum !== checksum) throw new Error('Migration checksum mismatch');
|
||||
continue;
|
||||
}
|
||||
await client.query('BEGIN');
|
||||
try {
|
||||
await client.query(sql);
|
||||
await client.query('INSERT INTO schema_migrations VALUES ($1,$2)', [version, checksum]);
|
||||
await client.query('COMMIT');
|
||||
} catch (error) { await client.query('ROLLBACK'); throw error; }
|
||||
console.log('Applied migration ' + version);
|
||||
}
|
||||
await client.query(`INSERT INTO app_settings VALUES ('cursor_secret',$1) ON CONFLICT DO NOTHING`, [randomBytes(32).toString('hex')]);
|
||||
} finally {
|
||||
await client.query('SELECT pg_advisory_unlock(202602)').catch(() => {});
|
||||
client.release();
|
||||
}
|
||||
}
|
||||
module.exports = { migrate };
|
||||
if (require.main === module) migrate().then(() => console.log('Migrations up to date'))
|
||||
.catch(() => { console.error('Migration failed'); process.exitCode = 1; }).finally(() => pool.end());
|
||||
+256
@@ -0,0 +1,256 @@
|
||||
'use strict';
|
||||
const http = require('node:http');
|
||||
const { randomUUID, randomBytes, createHash, createHmac, scrypt, timingSafeEqual } = require('node:crypto');
|
||||
const { promisify } = require('node:util');
|
||||
const { pool, transaction } = require('./db');
|
||||
const { sendMessage } = require('./messages');
|
||||
const attachments = require('./attachments');
|
||||
const storage = require('./storage');
|
||||
const instanceId = randomUUID();
|
||||
let stopping = false;
|
||||
const derive = promisify(scrypt);
|
||||
const kdfOptions = { N: 32768, r: 8, p: 1, maxmem: 64 * 1024 * 1024 };
|
||||
const ttl = Number(process.env.SESSION_TTL_SECONDS || 3600);
|
||||
if (!Number.isSafeInteger(ttl) || ttl < 1) throw new Error('Invalid SESSION_TTL_SECONDS');
|
||||
let cursorSecret;
|
||||
const userColumns = 'id, username, display_name, created_at';
|
||||
const chatSelect = `SELECT c.id,c.title,c.created_at,
|
||||
ARRAY(SELECT user_id FROM chat_members WHERE chat_id=c.id ORDER BY user_id) AS member_ids FROM chats c`;
|
||||
function fail(status, code, message) { throw Object.assign(new Error(message), { status, code }); }
|
||||
function invalid(message = 'Invalid request') { fail(400, 'invalid_request', message); }
|
||||
function unauthorized() { fail(401, 'unauthorized', 'Invalid credentials or session'); }
|
||||
function uuid(value) {
|
||||
if (typeof value !== 'string' || value.length !== 36 || !/^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$/i.test(value)) invalid('Invalid UUID');
|
||||
return value.toLowerCase();
|
||||
}
|
||||
function fields(body, names) {
|
||||
if (!body || typeof body !== 'object' || Array.isArray(body) ||
|
||||
Object.keys(body).length !== names.length || !names.every(k => Object.hasOwn(body, k))) invalid('Invalid fields');
|
||||
}
|
||||
function string(value, max, min = 1) {
|
||||
if (typeof value !== 'string' || !value.isWellFormed() || /^[\p{White_Space}\uFEFF]*$/u.test(value) ||
|
||||
[...value].length < min || [...value].length > max || value.includes('\0')) invalid('Invalid string');
|
||||
return value;
|
||||
}
|
||||
function username(value) {
|
||||
if (typeof value !== 'string' || value.length < 3 || value.length > 32 || /[^a-z0-9_]/.test(value)) invalid('Invalid username');
|
||||
}
|
||||
function password(value) {
|
||||
if (typeof value !== 'string' || !value.isWellFormed() || [...value].length < 12 || [...value].length > 128 ||
|
||||
/^[\p{White_Space}\uFEFF]*$/u.test(value)) invalid('Invalid password');
|
||||
}
|
||||
function digest(token) { return createHash('sha256').update(token).digest('hex'); }
|
||||
async function hashPassword(value) {
|
||||
const salt = randomBytes(16).toString('hex');
|
||||
const hash = await derive(value, salt, 64, kdfOptions);
|
||||
return `scrypt$32768$8$1$${salt}$${hash.toString('hex')}`;
|
||||
}
|
||||
const dummySalt = randomBytes(16).toString('hex');
|
||||
async function verifyPassword(value, encoded) {
|
||||
const parts = encoded?.split('$');
|
||||
const salt = parts ? parts[4] : dummySalt;
|
||||
const expected = parts ? Buffer.from(parts[5], 'hex') : Buffer.alloc(64);
|
||||
const actual = await derive(value, salt, 64, kdfOptions);
|
||||
return Boolean(encoded) && expected.length === actual.length && timingSafeEqual(expected, actual);
|
||||
}
|
||||
async function rawBody(req) {
|
||||
const chunks = [];
|
||||
let size = 0;
|
||||
for await (const chunk of req) {
|
||||
size += chunk.length;
|
||||
if (size > 1024 * 1024) fail(413, 'payload_too_large', 'Request body too large');
|
||||
chunks.push(chunk);
|
||||
}
|
||||
return Buffer.concat(chunks);
|
||||
}
|
||||
async function body(req) {
|
||||
if (req.headers['content-type']?.split(';')[0].trim().toLowerCase() !== 'application/json')
|
||||
fail(415, 'unsupported_media_type', 'Expected application/json');
|
||||
const bytes = await rawBody(req);
|
||||
try { return JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes)); }
|
||||
catch { invalid('Invalid JSON'); }
|
||||
}
|
||||
async function identity(req) {
|
||||
const match = /^Bearer ([A-Za-z0-9_-]{43})$/i.exec(req.headers.authorization || '');
|
||||
if (!match) unauthorized();
|
||||
const tokenHash = digest(match[1]);
|
||||
const result = await pool.query(`SELECT u.id,u.username,u.display_name,u.created_at
|
||||
FROM sessions s JOIN users u ON u.id=s.user_id
|
||||
WHERE token_hash=$1 AND expires_at > clock_timestamp()`, [tokenHash]);
|
||||
if (!result.rowCount) unauthorized();
|
||||
return { user: result.rows[0], tokenHash };
|
||||
}
|
||||
async function accessible(id, user) {
|
||||
const result = await pool.query(chatSelect + ` WHERE c.id=$1 AND EXISTS
|
||||
(SELECT 1 FROM chat_members WHERE chat_id=c.id AND user_id=$2)`, [uuid(id), user.id]);
|
||||
if (!result.rowCount) fail(404, 'not_found', 'Object not found');
|
||||
return result.rows[0];
|
||||
}
|
||||
function signature(payload) { return createHmac('sha256', cursorSecret).update(payload).digest('base64url'); }
|
||||
function pagination(query, scope) {
|
||||
if (query.getAll('limit').length > 1 || query.getAll('cursor').length > 1) invalid();
|
||||
const raw = query.get('limit') ?? '50';
|
||||
if (!/^\d+$/.test(raw) || Number(raw) < 1 || Number(raw) > 100) invalid('Invalid limit');
|
||||
let after = null;
|
||||
if (query.has('cursor')) {
|
||||
try {
|
||||
const token = query.get('cursor');
|
||||
if (!/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(token)) throw new Error();
|
||||
const [payload, mac] = token.split('.');
|
||||
if (signature(payload) !== mac) throw new Error();
|
||||
after = JSON.parse(Buffer.from(payload, 'base64url').toString('utf8'));
|
||||
if (after.scope !== scope || typeof after.created_at !== 'string' || !Number.isFinite(Date.parse(after.created_at))) throw new Error();
|
||||
uuid(after.id);
|
||||
} catch { invalid('Invalid cursor'); }
|
||||
}
|
||||
return { limit: Number(raw), after, scope };
|
||||
}
|
||||
function page(rows, paging) {
|
||||
const items = rows.slice(0, paging.limit);
|
||||
let next_cursor = null;
|
||||
if (rows.length > paging.limit) {
|
||||
const last = items.at(-1);
|
||||
const payload = Buffer.from(JSON.stringify({ scope: paging.scope, created_at: last.created_at, id: last.id })).toString('base64url');
|
||||
next_cursor = payload + '.' + signature(payload);
|
||||
}
|
||||
return { items, next_cursor };
|
||||
}
|
||||
async function route(req) {
|
||||
const url = new URL(req.url, 'http://localhost');
|
||||
const path = url.pathname, method = req.method;
|
||||
if (method === 'GET' && path === '/health/live') return [200, { status: 'ok' }];
|
||||
if (stopping) fail(503, 'unavailable', 'Server is stopping');
|
||||
if (method === 'GET' && path === '/health/ready') {
|
||||
try { await pool.query('SELECT 1'); }
|
||||
catch { fail(503, 'unavailable', 'Database unavailable'); }
|
||||
await storage.ready();
|
||||
return [200, { status: 'ok' }];
|
||||
}
|
||||
if (method === 'POST' && path === '/api/v1/users') {
|
||||
const data = await body(req);
|
||||
fields(data, ['username', 'display_name', 'password']);
|
||||
username(data.username); string(data.display_name, 100); password(data.password);
|
||||
const hash = await hashPassword(data.password);
|
||||
try {
|
||||
const result = await pool.query(`INSERT INTO users(id,username,display_name,password_hash)
|
||||
VALUES($1,$2,$3,$4) RETURNING ${userColumns}`, [randomUUID(), data.username, data.display_name, hash]);
|
||||
return [201, result.rows[0]];
|
||||
} catch (error) {
|
||||
if (error.code === '23505') fail(409, 'conflict', 'Username already exists');
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
if (method === 'POST' && path === '/api/v1/auth/sessions') {
|
||||
const match = /^Basic ([A-Za-z0-9+/]+={0,2})$/i.exec(req.headers.authorization || '');
|
||||
if (!match) unauthorized();
|
||||
let name, value;
|
||||
try {
|
||||
const bytes = Buffer.from(match[1], 'base64');
|
||||
if (bytes.toString('base64') !== match[1]) throw new Error();
|
||||
const decoded = new TextDecoder('utf-8', { fatal: true }).decode(bytes);
|
||||
const colon = decoded.indexOf(':');
|
||||
if (colon < 0) throw new Error();
|
||||
name = decoded.slice(0, colon); value = decoded.slice(colon + 1);
|
||||
username(name); password(value);
|
||||
} catch { unauthorized(); }
|
||||
if ((await rawBody(req)).length) invalid('Login must not have a body');
|
||||
const result = await pool.query(`SELECT ${userColumns},password_hash FROM users WHERE username=$1`, [name]);
|
||||
const row = result.rows[0];
|
||||
if (!await verifyPassword(value, row?.password_hash)) unauthorized();
|
||||
const { password_hash, ...user } = row;
|
||||
const token = randomBytes(32).toString('base64url');
|
||||
const session = await pool.query(`INSERT INTO sessions(token_hash,user_id,expires_at)
|
||||
VALUES($1,$2,clock_timestamp()+$3*interval '1 second') RETURNING expires_at`, [digest(token), user.id, ttl]);
|
||||
return [201, { token, token_type: 'Bearer', expires_at: session.rows[0].expires_at, user }];
|
||||
}
|
||||
const { user, tokenHash } = await identity(req);
|
||||
if (method === 'DELETE' && path === '/api/v1/auth/sessions/current') {
|
||||
await pool.query('DELETE FROM sessions WHERE token_hash=$1', [tokenHash]);
|
||||
return [204, null];
|
||||
}
|
||||
if (method === 'GET' && path === '/api/v1/users/me') return [200, user];
|
||||
if (path === '/api/v1/chats' && method === 'POST') {
|
||||
const data = await body(req);
|
||||
fields(data, ['title', 'member_ids']); string(data.title, 100);
|
||||
if (!Array.isArray(data.member_ids) || data.member_ids.length < 1 || data.member_ids.length > 100) invalid('Invalid members');
|
||||
const ids = data.member_ids.map(uuid);
|
||||
if (new Set(ids).size !== ids.length || !ids.includes(user.id)) invalid('Invalid members');
|
||||
return [201, await transaction(async client => {
|
||||
const known = await client.query('SELECT id FROM users WHERE id=ANY($1::uuid[])', [ids]);
|
||||
if (known.rowCount !== ids.length) invalid('Unknown member');
|
||||
const chat = (await client.query('INSERT INTO chats(id,title) VALUES($1,$2) RETURNING *', [randomUUID(), data.title])).rows[0];
|
||||
await client.query('INSERT INTO chat_members(chat_id,user_id) SELECT $1,unnest($2::uuid[])', [chat.id, ids]);
|
||||
return { ...chat, member_ids: ids.sort() };
|
||||
})];
|
||||
}
|
||||
if (path === '/api/v1/chats' && method === 'GET') {
|
||||
const paging = pagination(url.searchParams, 'chats:' + user.id);
|
||||
const rows = (await pool.query(chatSelect + ` WHERE EXISTS
|
||||
(SELECT 1 FROM chat_members WHERE chat_id=c.id AND user_id=$1)
|
||||
AND ($2::timestamptz IS NULL OR (c.created_at,c.id)>($2::timestamptz,$3::uuid))
|
||||
ORDER BY c.created_at,c.id LIMIT $4`, [user.id, paging.after?.created_at ?? null, paging.after?.id ?? null, paging.limit + 1])).rows;
|
||||
return [200, page(rows, paging)];
|
||||
}
|
||||
const attachmentPath = path.match(/^\/api\/v1\/attachments\/([^/]+)(\/content)?$/);
|
||||
if (attachmentPath && method === 'GET') {
|
||||
const row = await attachments.accessible(uuid(attachmentPath[1]), user);
|
||||
if (!attachmentPath[2]) return [200, attachments.visible(row)];
|
||||
if (url.searchParams.getAll('variant').length > 1) invalid('Invalid variant');
|
||||
if ((url.searchParams.get('variant') ?? 'original') !== 'original') fail(404, 'not_found', 'Object not found');
|
||||
return [200, await attachments.download(row), { 'Content-Type': row.content_type, 'X-Content-Type-Options': 'nosniff' }];
|
||||
}
|
||||
const match = path.match(/^\/api\/v1\/chats\/([^/]+)(\/messages|\/attachments)?$/);
|
||||
if (match) {
|
||||
const chat = await accessible(match[1], user);
|
||||
if (!match[2] && method === 'GET') return [200, chat];
|
||||
if (match[2] === '/attachments' && method === 'POST') return [201, await attachments.upload(req, url.searchParams, chat, user)];
|
||||
if (match[2] === '/messages' && method === 'POST') {
|
||||
const key = req.headers['idempotency-key'];
|
||||
if (typeof key !== 'string' || key.length < 1 || key.length > 128 || /[^A-Za-z0-9._:-]/.test(key)) invalid('Invalid Idempotency-Key');
|
||||
const data = await body(req);
|
||||
if (!data || typeof data !== 'object' || Array.isArray(data) || !Object.hasOwn(data, 'text') ||
|
||||
Object.keys(data).some(k => !['text', 'attachment_ids'].includes(k))) invalid('Invalid fields');
|
||||
if (typeof data.text !== 'string' || !data.text.isWellFormed() || [...data.text].length > 4000 || data.text.includes('\0')) invalid('Invalid text');
|
||||
const rawIds = Object.hasOwn(data, 'attachment_ids') ? data.attachment_ids : [];
|
||||
if (!Array.isArray(rawIds) || rawIds.length > 10) invalid('Invalid attachments');
|
||||
const ids = rawIds.map(uuid);
|
||||
if (new Set(ids).size !== ids.length || (ids.length === 0 && /^[\p{White_Space}\uFEFF]*$/u.test(data.text))) invalid('Empty message or duplicate attachments');
|
||||
return [201, await sendMessage(user.id, chat.id, key, data.text, ids)];
|
||||
}
|
||||
if (match[2] === '/messages' && method === 'GET') {
|
||||
const paging = pagination(url.searchParams, 'messages:' + chat.id);
|
||||
const result = await pool.query(`SELECT m.*, ARRAY(SELECT attachment_id FROM message_attachments
|
||||
WHERE message_id=m.id ORDER BY position) AS attachment_ids FROM messages m WHERE m.chat_id=$1
|
||||
AND ($2::timestamptz IS NULL OR (m.created_at,m.id)>($2::timestamptz,$3::uuid))
|
||||
ORDER BY m.created_at,m.id LIMIT $4`, [chat.id, paging.after?.created_at ?? null, paging.after?.id ?? null, paging.limit + 1]);
|
||||
return [200, page(result.rows, paging)];
|
||||
}
|
||||
}
|
||||
fail(404, 'not_found', 'Object not found');
|
||||
}
|
||||
const server = http.createServer(async (req, res) => {
|
||||
const request_id = randomUUID();
|
||||
let status, data, responseHeaders;
|
||||
const headers = { 'Content-Type': 'application/json; charset=utf-8', 'X-Request-Id': request_id, 'X-Instance-Id': instanceId };
|
||||
try { [status, data, responseHeaders] = await route(req); }
|
||||
catch (error) {
|
||||
status = error.status || (/^(08|53|57)/.test(error.code || '') || ['ECONNREFUSED', 'ECONNRESET', 'ETIMEDOUT'].includes(error.code) ? 503 : 500);
|
||||
data = { error: { code: error.status ? error.code : status === 503 ? 'unavailable' : 'internal_error',
|
||||
message: error.status ? error.message : status === 503 ? 'Database unavailable' : 'Internal error', request_id } };
|
||||
if (status === 401) headers['WWW-Authenticate'] = req.url.split('?')[0] === '/api/v1/auth/sessions'
|
||||
? 'Basic realm="messenger", charset="UTF-8"' : 'Bearer realm="messenger"';
|
||||
}
|
||||
if (status === 413) headers.Connection = 'close';
|
||||
res.writeHead(status, { ...headers, ...responseHeaders });
|
||||
res.end(status === 204 ? undefined : Buffer.isBuffer(data) ? data : JSON.stringify(data));
|
||||
});
|
||||
async function main() {
|
||||
cursorSecret = (await pool.query("SELECT value FROM app_settings WHERE key='cursor_secret'")).rows[0].value;
|
||||
server.listen(Number(process.env.APP_PORT || 8080), process.env.APP_HOST || '0.0.0.0', () => console.log('Messenger lab4 ready; instance=' + instanceId));
|
||||
}
|
||||
for (const signal of ['SIGTERM', 'SIGINT']) process.on(signal, () => {
|
||||
stopping = true;
|
||||
server.close(() => pool.end());
|
||||
setTimeout(() => process.exit(1), 9000).unref();
|
||||
});
|
||||
main().catch(() => { console.error('Startup failed: check database and migrations'); process.exitCode = 1; pool.end(); });
|
||||
@@ -0,0 +1,36 @@
|
||||
'use strict';
|
||||
const { readFileSync } = require('node:fs');
|
||||
const { Agent } = require('node:http');
|
||||
const { Client } = require('minio');
|
||||
const endpoint = new URL(process.env.S3_ENDPOINT || 'http://s3:9000');
|
||||
const bucket = process.env.S3_BUCKET || 'messenger-attachments';
|
||||
const client = new Client({
|
||||
endPoint: endpoint.hostname, port: Number(endpoint.port || 9000), useSSL: false,
|
||||
region: process.env.S3_REGION || 'us-east-1',
|
||||
accessKey: readFileSync(process.env.S3_ACCESS_KEY_ID_FILE || '/run/secrets/s3_access_key_id', 'utf8').trim(),
|
||||
secretKey: readFileSync(process.env.S3_SECRET_ACCESS_KEY_FILE || '/run/secrets/s3_secret_access_key', 'utf8').trim(),
|
||||
transportAgent: new Agent({ keepAlive: true, timeout: 5000, maxSockets: 16 }),
|
||||
});
|
||||
function unavailable() { return Object.assign(new Error('Object storage unavailable'), { status: 503, code: 'unavailable' }); }
|
||||
async function ready() {
|
||||
try { if (!await client.bucketExists(bucket)) throw new Error(); }
|
||||
catch { throw unavailable(); }
|
||||
}
|
||||
async function bootstrap() {
|
||||
for (let attempt = 0; attempt < 30; attempt++) {
|
||||
try {
|
||||
if (!await client.bucketExists(bucket)) await client.makeBucket(bucket, process.env.S3_REGION || 'us-east-1');
|
||||
// Remove any anonymous policy; newly created buckets are private by default.
|
||||
await client.setBucketPolicy(bucket, '');
|
||||
console.log('Private bucket ready');
|
||||
return;
|
||||
} catch (error) {
|
||||
if (attempt === 29) throw unavailable();
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
}
|
||||
}
|
||||
}
|
||||
module.exports = { client, bucket, ready, unavailable };
|
||||
if (require.main === module) bootstrap().then(() => process.exit(0)).catch(() => {
|
||||
console.error('Bucket initialization failed'); process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,151 @@
|
||||
'use strict';
|
||||
const assert = require('node:assert/strict');
|
||||
const { readFileSync } = require('node:fs');
|
||||
const { randomUUID, createHash } = require('node:crypto');
|
||||
const { execFileSync } = require('node:child_process');
|
||||
const http = require('node:http');
|
||||
const base = process.env.BASE_URL || 'http://localhost:8084';
|
||||
const png = readFileSync('tests/fixtures/sample.png');
|
||||
const sha = bytes => createHash('sha256').update(bytes).digest('hex');
|
||||
const pause = ms => new Promise(resolve => setTimeout(resolve, ms));
|
||||
async function call(method, path, expected = 200, actor, data, headers = {}) {
|
||||
const response = await fetch(base + path, { method, headers: {
|
||||
...(actor ? { Authorization: 'Bearer ' + actor.token } : {}),
|
||||
...(data === undefined ? {} : { 'Content-Type': Buffer.isBuffer(data) ? 'image/png' : 'application/json' }), ...headers },
|
||||
body: data === undefined ? undefined : Buffer.isBuffer(data) ? data : JSON.stringify(data), signal: AbortSignal.timeout(12000) });
|
||||
assert.equal(response.status, expected, method + ' ' + path.split('?')[0]);
|
||||
assert.ok(response.headers.get('x-request-id'));
|
||||
assert.ok(response.headers.get('x-instance-id'));
|
||||
const bytes = Buffer.from(await response.arrayBuffer());
|
||||
const json = response.headers.get('content-type')?.includes('application/json') ? JSON.parse(bytes) : null;
|
||||
if (expected >= 400) assert.equal(json.error.request_id, response.headers.get('x-request-id'));
|
||||
return { data: json, bytes, headers: response.headers, instance: response.headers.get('x-instance-id') };
|
||||
}
|
||||
async function user() {
|
||||
const username = 'u_' + randomUUID().replaceAll('-', '').slice(0, 14), password = 'Secret-' + randomUUID();
|
||||
const created = (await call('POST', '/api/v1/users', 201, null, { username, password, display_name: 'Студент' })).data;
|
||||
const login = (await call('POST', '/api/v1/auth/sessions', 201, null, undefined,
|
||||
{ Authorization: 'Basic ' + Buffer.from(username + ':' + password).toString('base64') })).data;
|
||||
return { id: created.id, token: login.token };
|
||||
}
|
||||
const docker = (...args) => execFileSync('docker', args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();
|
||||
const internal = script => docker('compose', 'exec', '-T', 'api1', 'node', '-e', script);
|
||||
function control(action, target) {
|
||||
const python = process.env.LAB_PYTHON || (process.platform === 'win32' ? 'py' : 'python3');
|
||||
execFileSync(python, ['scripts/control.py', action], { stdio: ['ignore', 'pipe', 'pipe'],
|
||||
env: { ...process.env, ...(target ? { TARGET_INSTANCE_ID: target } : {}) } });
|
||||
}
|
||||
async function ready(actor) {
|
||||
for (let i = 0; i < 40; i++) {
|
||||
try { await call('GET', '/api/v1/users/me', 200, actor); return; } catch { await pause(250); }
|
||||
}
|
||||
throw new Error('Load balancer did not recover');
|
||||
}
|
||||
function crc32(buffer) {
|
||||
let value = 0xffffffff;
|
||||
for (const byte of buffer) {
|
||||
value ^= byte;
|
||||
for (let bit = 0; bit < 8; bit++) value = (value >>> 1) ^ ((value & 1) ? 0xedb88320 : 0);
|
||||
}
|
||||
return (value ^ 0xffffffff) >>> 0;
|
||||
}
|
||||
function chunkedTooLarge(path, actor) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const req = http.request(base + path, { method: 'POST', headers: {
|
||||
Authorization: 'Bearer ' + actor.token, 'Content-Type': 'image/png', 'Transfer-Encoding': 'chunked' } }, res => {
|
||||
let body = '';
|
||||
res.on('data', chunk => body += chunk);
|
||||
res.on('end', () => {
|
||||
try { assert.equal(res.statusCode, 413); assert.equal(JSON.parse(body).error.code, 'payload_too_large'); resolve(); }
|
||||
catch (error) { reject(error); }
|
||||
});
|
||||
});
|
||||
req.on('error', error => { if (!req.res) reject(error); });
|
||||
req.setTimeout(15000, () => req.destroy(new Error('Chunked upload timeout')));
|
||||
req.write(png);
|
||||
for (let i = 0; i < 11; i++) req.write(Buffer.alloc(1024 * 1024));
|
||||
req.end();
|
||||
});
|
||||
}
|
||||
async function main() {
|
||||
const a = await user(), b = await user(), outsider = await user();
|
||||
const chat = (await call('POST', '/api/v1/chats', 201, a, { title: 'Изображения', member_ids: [a.id, b.id] })).data;
|
||||
const uploadPath = '/api/v1/chats/' + chat.id + '/attachments';
|
||||
const messages = '/api/v1/chats/' + chat.id + '/messages';
|
||||
const uploaded = await call('POST', uploadPath + '?filename=' + encodeURIComponent('Фото 😀.png'), 201, a, png);
|
||||
const att = uploaded.data, content = '/api/v1/attachments/' + att.id + '/content';
|
||||
assert.equal(att.status, 'ready'); assert.equal(att.sha256, sha(png)); assert.equal(att.size_bytes, png.length);
|
||||
assert.equal(att.object_key, undefined);
|
||||
const download = await call('GET', content, 200, b);
|
||||
assert.deepEqual(download.bytes, png); assert.equal(download.headers.get('content-type'), 'image/png');
|
||||
for (const suffix of ['', '/content']) await call('GET', '/api/v1/attachments/' + att.id + suffix, 404, outsider);
|
||||
await call('GET', content, 401);
|
||||
await call('GET', content + '?variant=thumbnail', 404, a);
|
||||
const key = randomUUID(), body = { text: '', attachment_ids: [att.id] };
|
||||
const concurrent = await Promise.all(Array.from({ length: 12 }, () => call('POST', messages, 201, a, body, { 'Idempotency-Key': key })));
|
||||
for (const result of concurrent) assert.deepEqual(result.data, concurrent[0].data);
|
||||
assert.equal(new Set(concurrent.map(r => r.instance)).size, 2);
|
||||
assert.deepEqual(concurrent[0].data.attachment_ids, [att.id]);
|
||||
assert.equal((await call('GET', messages, 200, b)).data.items.length, 1);
|
||||
await call('POST', messages, 409, a, { text: 'Changed', attachment_ids: [att.id] }, { 'Idempotency-Key': key });
|
||||
const textKey = randomUUID();
|
||||
const plain = await call('POST', messages, 201, a, { text: 'Text only' }, { 'Idempotency-Key': textKey });
|
||||
const explicit = await call('POST', messages, 201, a, { text: 'Text only', attachment_ids: [] }, { 'Idempotency-Key': textKey });
|
||||
assert.deepEqual(plain.data, explicit.data);
|
||||
await call('POST', messages, 400, a, { text: '', attachment_ids: [att.id, att.id] }, { 'Idempotency-Key': randomUUID() });
|
||||
const otherChat = (await call('POST', '/api/v1/chats', 201, a, { title: 'Other', member_ids: [a.id] })).data;
|
||||
await call('POST', '/api/v1/chats/' + otherChat.id + '/messages', 404, a, body, { 'Idempotency-Key': randomUUID() });
|
||||
for (const name of ['../bad.png', 'a/b.png', 'a\\b.png', 'a\0.png', '\u0085', 'a\n.png', 'x'.repeat(256)])
|
||||
await call('POST', uploadPath + '?filename=' + encodeURIComponent(name), 400, a, png);
|
||||
await call('POST', uploadPath, 400, a, png);
|
||||
await call('POST', uploadPath + '?filename=x.png', 415, a, Buffer.from('not an image'));
|
||||
await call('POST', uploadPath + '?filename=x.png', 415, a, png.subarray(0, 60));
|
||||
await call('POST', uploadPath + '?filename=x.jpg', 415, a, png, { 'Content-Type': 'image/jpeg' });
|
||||
await call('POST', uploadPath + '?filename=x.png', 415, a, png, { 'Content-Type': 'text/plain' });
|
||||
await call('POST', uploadPath + '?filename=x.png', 413, a, Buffer.alloc(10485761));
|
||||
await chunkedTooLarge(uploadPath + '?filename=large.png', a);
|
||||
const huge = Buffer.from(png);
|
||||
huge.writeUInt32BE(8001, 16); huge.writeUInt32BE(5000, 20); huge.writeUInt32BE(crc32(huge.subarray(12, 29)), 29);
|
||||
await call('POST', uploadPath + '?filename=huge.png', 413, a, huge);
|
||||
const jpeg = Buffer.from(internal("require('sharp')({create:{width:20,height:10,channels:3,background:'red'}}).jpeg().toBuffer().then(b=>console.log(b.toString('base64')))"), 'base64');
|
||||
const jpg = await call('POST', uploadPath + '?filename=real.jpg', 201, b, jpeg, { 'Content-Type': 'image/jpeg' });
|
||||
assert.deepEqual((await call('GET', '/api/v1/attachments/' + jpg.data.id + '/content', 200, a)).bytes, jpeg);
|
||||
// Resolve the private storage key inside the API container; never expose it in public metadata.
|
||||
const anonymous = JSON.parse(internal(`const {pool}=require('./src/db');pool.query('SELECT object_key FROM attachments WHERE id=$1',['${att.id}']).then(async r=>{const response=await fetch('http://s3:9000/messenger-attachments/'+r.rows[0].object_key);console.log(JSON.stringify({status:response.status}));}).finally(()=>pool.end());`));
|
||||
assert.equal(anonymous.status, 403);
|
||||
console.log('PASS: PNG/JPEG exact bytes and SHA-256; private bucket 403; member access; invalid filename/MIME/corrupt file; >10 MiB including chunked and >40M pixels; attachment scope; 12 concurrent sends make 1 message.');
|
||||
if (process.argv.includes('--lifecycle')) {
|
||||
let target = uploaded.instance;
|
||||
for (let round = 0; round < 2; round++) {
|
||||
// Stop the exact process that accepted this upload, then download only through LB.
|
||||
const fresh = round === 0 ? uploaded : await call('POST', uploadPath + '?filename=second.png', 201, a, png);
|
||||
target = fresh.instance;
|
||||
const started = Date.now();
|
||||
control('stop-one', target);
|
||||
try {
|
||||
let ok = false;
|
||||
for (let i = 0; i < 30 && Date.now() - started < 10000; i++) {
|
||||
try {
|
||||
const bytes = await call('GET', '/api/v1/attachments/' + fresh.data.id + '/content', 200, b);
|
||||
assert.notEqual(bytes.instance, target); assert.deepEqual(bytes.bytes, png); ok = true; break;
|
||||
} catch { await pause(100); }
|
||||
}
|
||||
assert.ok(ok, 'Download after uploader failure exceeded 10 seconds');
|
||||
console.log(JSON.stringify({ scenario: 'upload-stop-uploader-download', recovery_ms: Date.now() - started, same_bytes: true }));
|
||||
} finally { control('restore'); await ready(a); }
|
||||
}
|
||||
control('restart'); await ready(a);
|
||||
assert.deepEqual((await call('GET', content, 200, b)).bytes, png);
|
||||
assert.deepEqual((await call('POST', messages, 201, a, body, { 'Idempotency-Key': key })).data, concurrent[0].data);
|
||||
console.log('PASS: full restart preserves metadata, original bytes, session and idempotent attachment message.');
|
||||
docker('compose', 'stop', 's3');
|
||||
try {
|
||||
const checks = JSON.parse(internal(`Promise.all(['/health/live','/health/ready'].map(async p=>{const r=await fetch('http://127.0.0.1:8080'+p);return r.status})).then(s=>console.log(JSON.stringify(s)))`));
|
||||
assert.deepEqual(checks, [200, 503]);
|
||||
console.log('PASS: stopped S3 makes API readiness 503 while liveness remains 200.');
|
||||
} finally { docker('compose', 'up', '-d', '--wait', 's3'); await ready(a); }
|
||||
assert.deepEqual((await call('GET', content, 200, a)).bytes, png);
|
||||
console.log('PASS: S3 recovery restores download without data loss.');
|
||||
}
|
||||
}
|
||||
main().catch(error => { console.error('FAIL: ' + error.message); process.exitCode = 1; });
|
||||
Reference in New Issue
Block a user