diff --git a/REPORT.md b/REPORT.md index 370bdad..4d48869 100644 --- a/REPORT.md +++ b/REPORT.md @@ -71,8 +71,8 @@ worker ◀── broker; ──▶ postgres; ──▶ s3 (Read/Write оба buc | Критерий README | Команда/сценарий | Наблюдаемый результат | Файл | | --- | --- | --- | --- | -| Публичный smoke | `make check test` | PASS lab5: 202 → ready, JPEG 256×160 | [make-check-validate-test.txt](evidence/make-check-validate-test.txt) | -| **3:** минимум lab4 сохранён, 201 → 202 | `pytest student-tests` (test_api, test_attachments, test_idempotency, test_balancing, test_sessions, test_failures) | все проходят, 415 на все кейсы рецензий (PNG фильтр 5/0xff, IDAT не zlib, JPEG 27 байт, SOS не из SOF) | [student-tests.txt](evidence/student-tests.txt) | +| Публичный smoke | `make check validate test` | PASS ×3: 202 → ready, JPEG 256×160 | [make-check-validate-test.txt](evidence/make-check-validate-test.txt) | +| **3:** минимум lab4 сохранён, 201 → 202 | `pytest student-tests` (test_api, test_attachments, test_idempotency, test_balancing, test_sessions, test_failures) | 377 passed; 415 на все кейсы рецензий (PNG фильтр 5/0xff, IDAT не zlib, JPEG 27 байт, SOS не из SOF) | [student-tests.txt](evidence/student-tests.txt) | | **3:** 202 только после устойчивого приёма; остановка API не теряет | test_consistency (`after_s3_put`, `after_enqueue_commit`), demo | окно «после staging» — невидимый `pending`, убран janitor; после COMMIT — ready | student-tests.txt | | **3:** отдельные broker и worker, durable, ack после эффекта | compose, `rabbitmqctl list_queues`; `test_worker_crash_after_commit…` | quorum queue durable; redelivery → `noop_finished` | [clean-clone.txt](evidence/clean-clone.txt), student-tests.txt | | **3:** worker недоступен → API не делает resize | `test_worker_stopped…`, demo | 202 queued, content 409, сообщения 201 | [demo-pipeline.txt](evidence/demo-pipeline.txt) | @@ -87,7 +87,7 @@ worker ◀── broker; ──▶ postgres; ──▶ s3 (Read/Write оба buc | **4:** DLQ/реестр failed, безопасный replay | `jobctl list-failed/replay`, `test_bad_message…` | replay → ready; replay готового → `already_ready`; плохое сообщение → DLQ | student-tests.txt | | **4:** crash после записи объекта до ack | `test_worker_crash_after_s3_write…`, `…after_commit…` | один опубликованный набор, мусор попытки 1 удалён | student-tests.txt | | **4:** lease зависшего processing, prefetch, очистка staging | `test_hung_worker…` (docker pause), `test_concurrency…`, `test_queued_staging…`, `test_cleanup_racing…` | попытки `[abandoned, succeeded]`; ≤2 processing; staging `queued` не удаляется, CHECK отвергает | student-tests.txt | -| **5, трек 2** | `bench/pipeline_faults.py`, `make audit` | см. ниже | [evidence/track2/](evidence/track2/), [track2-runs.txt](evidence/track2-runs.txt) | +| **5, трек 2** | `bench/pipeline_faults.py`, `make audit` | 12/12 сценариев, 0 нарушений, аудит ok | [evidence/track2/](evidence/track2/), [track2-runs.txt](evidence/track2-runs.txt) | ## Эксперименты и отказы @@ -114,7 +114,29 @@ worker ◀── broker; ──▶ postgres; ──▶ s3 (Read/Write оба buc | reorder версий в очереди | v2 и v3 в outbox, relay публикует в обратном порядке, 1 worker | | зависший worker | 2 worker, `docker pause` владельца lease | -**Результаты:** РЕЗУЛЬТАТЫ_ТРЕКА_2 +**Результаты** (чистый clone, 2026-10-31 14:48–14:57 UTC, 5 загрузок на «плоский» сценарий; [summary.txt](evidence/track2/summary.txt), подробно — [summary.json](evidence/track2/summary.json)): + +| Сценарий | Загрузок | ready | claim-ов | Нарушений | Последовательность событий задания | +| --- | --- | --- | --- | --- | --- | +| baseline | 5 | 5 | 5 | 0 | created › claimed › succeeded | +| дубль publish | 5 | 5 | 5 | 0 | то же: второе сообщение — `noop_finished` | +| дубль + обратный порядок | 5 | 5 | 5 | 0 | то же | +| crash relay после confirm | 5 | 5 | 5 | 0 | created › claimed › fault_injected › succeeded | +| crash worker после claim | 5 | 5 | 14 | 0 | claimed › fault_injected › lease_expired › claimed › succeeded (*) | +| crash worker после S3, до COMMIT | 5 | 5 | 10 | 0 | claimed › fault_injected › lease_expired › claimed › succeeded | +| crash worker после COMMIT, до ack | 5 | 5 | 5 | 0 | claimed › succeeded › fault_injected (redelivery = no-op) | +| 2 временные ошибки | 5 | 5 | 15 | 0 | 3 claim-а на загрузку, ровно как задано | +| 2 worker + дубль | 5 | 5 | 5 | 0 | один claim при конкурирующих потребителях | +| устаревшая версия | 1 | — | — | 0 | v1 succeeded, v2 (медленная) **superseded** при COMMIT, v3 succeeded; статус всё время `ready`; ручной дубль задания v1 — no-op | +| reorder версий в очереди | 1 | — | 2 | 0 | v3 опубликована первой, v2 **superseded при claim** без работы | +| зависший worker (`docker pause`) | 1 | — | 2 | 0 | claimed › lease_expired › claimed › succeeded; ожившая попытка fenced | +| **аудит всего стенда после матрицы** | | | | **0** | `ok=True`, 522 опубликованных объекта, SHA-256 сверены, `final=523` (+sentinel) | + +(*) claim-ов больше, чем 2 на загрузку: `os._exit` при одном задании убивает **весь процесс**, а с ним и соседнюю попытку другого задания (prefetch 2). Ей тоже приходится ждать истечения lease. Это и есть «crash до побочного эффекта»: лишние claim-ы ничего не публикуют, и ни в одном сценарии нет второго набора вариантов или объектов (столбец «Нарушений» — проверки H2/H4 по БД и S3). + +**Вывод.** H1–H4 подтверждены на всех 12 сценариях: 0 нарушений по сценарным проверкам и по аудиту I1–I7. Время до ready при отказах определяется lease (6 с в эксперименте) + интервалом reaper (5 с): 15–22 с против ~2 с без отказов. + +**Ограничения эксперимента.** Один прогон матрицы на 5 загрузок (плюс прогоны тех же сценариев в `student-tests` и отладочный прогон на 3 загрузки — тоже 0 нарушений). Это доказательство на конечном наборе чередований, а не формальная верификация. Отказы внедряются в заранее выбранных точках кода, а не в случайные моменты. Один Docker-хост. **Почему это не exactly-once.** Доставка остаётся at-least-once: в сценариях дубля в очереди физически два сообщения, а у зависшего worker вычисление выполнялось дважды (попытки `abandoned` и `succeeded`). Exactly-once — только у **видимого эффекта**, за счёт ключа эффекта, fencing и монотонной версии. Чего нет: (1) защиты от двух независимых **сред** с разными БД; (2) гарантии при потере диска одного Docker-хоста (quorum queue и PostgreSQL на одном узле); (3) удаления опубликованных вариантов старых версий — они остаются (rollback-кандидаты), удаляются только неопубликованные. @@ -142,6 +164,23 @@ worker ◀── broker; ──▶ postgres; ──▶ s3 (Read/Write оба buc Ноутбук: AMD Ryzen 5 3400G (4 ядра / 8 потоков), 29,9 ГБ RAM, Windows 10 Enterprise 19045, Docker Desktop 29.8.0 (VM: 8 CPU, 15,7 ГБ). Лимиты контейнеров (фактические, `docker inspect` в clean-clone.txt): api 1 CPU / 512m ×2, worker 2 CPU / 1536m (без swap), broker 1 CPU / 512m, relay 0,5 / 256m, postgres 2 / 1g, s3 1 / 1g, valkey 0,5 / 256m, lb 1 / 128m, migrate/s3-init 0,5 / 256m, auditor 1 / 256m, jobctl 0,5 / 128m. Нагрузочных выводов о производительности в этой лабе не делается (трек 2 — про корректность); `tests/load.py` приложен как контроль регрессии lab3. +## Финальный прогон + +`python scripts/collect_evidence.py` в **новой папке** (`git clone` ветки, commit `cf6e929`): новые secrets, пустые volumes (стенд разработки удалён `down -v`), образы приложения собраны заново (слои базовых образов из кэша Docker). Все шаги `exit 0`: + +| Файл | Что внутри | +| --- | --- | +| [clean-clone.txt](evidence/clean-clone.txt) | init_secrets, `compose config`, `up --build --wait`, migrate (0001–0005), s3-init (вердикты прав), пользователи и права RabbitMQ, очереди, **лимиты `docker inspect`**, версии | +| [make-check-validate-test.txt](evidence/make-check-validate-test.txt) | `make check` PASS, `make validate` PASS, `make test` PASS lab5 | +| [test-persistence.txt](evidence/test-persistence.txt) | PASS; состояния контейнеров до/после (новые pid у lb, api, relay, worker, postgres, valkey, s3, broker) | +| [test-failover.txt](evidence/test-failover.txt) | PASS дважды, остановлены обе реплики по очереди | +| [demo-pipeline.txt](evidence/demo-pipeline.txt) | сценарий защиты шаги 2–3, очередь `messages=1` до и после рестарта брокера, ready, JPEG 256×160 | +| [student-tests.txt](evidence/student-tests.txt) | **377 passed, 4 skipped** (skipped — unit-тесты `djpeg`, которого нет на Windows-хосте; те же случаи проходят против стенда), 25 мин | +| [track2-runs.txt](evidence/track2-runs.txt), [track2/](evidence/track2/) | матрица трека 2, аудит `--verify-checksums`, `jobctl stats` (outbox пуст), `tests/load.py` | +| [load.json](evidence/load.json) | контроль регрессии lab3: 183 успешных RPS, 0 ошибок, p95 44 мс (4 клиента, 30 с; не сравнивается с lab3 — другой стенд) | + +Свап: у api и worker `memswap = memory` (без swap). У инфраструктурных контейнеров остался умолчательный swap Docker (= 2 × memory) — видно в `docker inspect`, для выводов этой лабы не важно. + ## Ограничения и следующие шаги - Один Docker-хост: quorum queue на одном узле (без репликации), PostgreSQL один. «Устойчиво» = переживает рестарт процессов/контейнеров с volumes. diff --git a/docs/PR.md b/docs/PR.md new file mode 100644 index 0000000..9522b21 --- /dev/null +++ b/docs/PR.md @@ -0,0 +1,33 @@ +## Лабораторная 5 + +Автор / группа: Боданюк Алексей, БВТ2302 +Целевая оценка (3/4/5): 5 +Трек 5, если заявлен: трек 2 «Доказанная идемпотентность pipeline» (versioned pipeline + автоматический fault injection) + +Что реализовано и что перенесено: реализация lab4 перенесена отдельным коммитом без изменений (`5112be8`). Добавлено: +- upload 201 → 202: оригинал в приватном staging-bucket + одна транзакция `queued` + задание + outbox; +- RabbitMQ 4.3.7 (quorum queue, publisher confirms, DLX/DLQ, пользователи с минимальными правами, без management UI); +- relay (transactional outbox, reaper истёкших lease); +- отдельный worker: lease с heartbeat и fencing по токену попытки, варианты thumbnail/crop/WebP в дочернем процессе с лимитами памяти, времени и пикселей, ack только после COMMIT; +- retry с full jitter, реестр failed и `jobctl replay/reprocess`; +- уборка staging и мусора попыток под блокировкой строки; +- read-only аудит инвариантов. + +Reconciler lab4 убран, причина — в REPORT. Подробно — REPORT.md, docs/PIPELINE.md, docs/DEFENSE.md. + +Команды запуска из чистого clone: REPORT.md → «Запуск из чистого clone»: `python3 scripts/init_secrets.py`, `cp .env.example .env`, `docker compose --env-file .env up -d --build --wait`. Windows — docs/WINDOWS.md. + +Результаты проверок и ссылка на REPORT.md: REPORT.md → «Доказательства критериев»; вывод прогона из чистого clone — в evidence/: make check/validate/test, persistence, failover, сценарий защиты, свои тесты, матрица трека 2, аудит. + +Ограничения / известные ошибки: REPORT.md → «Ограничения и следующие шаги»: один Docker-хост, опубликованные варианты старых версий не удаляются, fsync SeaweedFS. + +Источники и использование ИИ: REPORT.md, шапка. Код, тесты и отчёт написаны с помощью Claude Code; все прогоны реально выполнены. + +- [x] Весь обязательный минимум текущей и предыдущих работ сохранён с объявленными переходами. +- [x] Контракт и публичные тесты не ослаблены. +- [x] make check и make test выполнены; приложен реальный вывод. +- [x] Сценарии отказов/безопасности своего уровня воспроизведены. +- [x] Все секреты и приватные ключи находятся вне Git; отчёт не содержит tokens. +- [x] README задания сохранён, REPORT.md содержит инструкцию именно моего решения. + +🤖 Generated with [Claude Code](https://claude.com/claude-code) diff --git a/evidence/clean-clone.txt b/evidence/clean-clone.txt new file mode 100644 index 0000000..744febf --- /dev/null +++ b/evidence/clean-clone.txt @@ -0,0 +1,392 @@ +# Clean clone: secrets, config, build, migrations from zero, start, versions +# 2026-10-31T14:18:15Z, commit cf6e929 docs: PIPELINE, DEFENSE, DATA_MODEL, WINDOWS для lab5; черновик REPORT + +$ git status --short --branch +## solution/bvt2302-bodanyuk...origin/solution/bvt2302-bodanyuk +[exit 0, 0.1 s] + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe --version +Python 3.14.6 +[exit 0, 0.0 s] + +$ docker version --format client {{.Client.Version}} / server {{.Server.Version}} +client 29.8.0 / server 29.8.0 +[exit 0, 0.1 s] + +$ docker compose version +Docker Compose version v5.5.1 +[exit 0, 0.1 s] + +$ make --version +GNU Make 4.4.1 +Built for Windows32 +Copyright (C) 1988-2023 Free Software Foundation, Inc. +License GPLv3+: GNU GPL version 3 or later +This is free software: you are free to change and redistribute it. +There is NO WARRANTY, to the extent permitted by law. +[exit 0, 0.0 s] + +$ python scripts/init_secrets.py +created secrets/postgres_password +created secrets/valkey_password +created secrets/database_url +created secrets/cache_url +created secrets/s3_admin_access_key_id +created secrets/s3_admin_secret_access_key +created secrets/s3_api_access_key_id +created secrets/s3_api_secret_access_key +created secrets/s3_worker_access_key_id +created secrets/s3_worker_secret_access_key +created secrets/s3_auditor_access_key_id +created secrets/s3_auditor_secret_access_key +created secrets/rabbitmq_relay_password +created secrets/rabbitmq_worker_password +created secrets/rabbitmq_users.json +created secrets/broker_url_relay +created secrets/broker_url_worker +[exit 0, 0.1 s] + +$ cp .env.example .env + +$ docker compose --env-file .env config --quiet + +[exit 0, 0.2 s] + +$ docker compose --env-file .env up -d --build --wait +#1 [internal] load local bake definitions +#1 reading from stdin 1.32kB 0.0s done +#1 DONE 0.0s + +#2 [migrate internal] load build definition from Dockerfile +#2 transferring dockerfile: 2.96kB done +#2 DONE 0.1s + +#3 [migrate] resolve image config for docker-image://docker.io/docker/dockerfile:1 +#3 DONE 6.2s + +#4 [migrate] docker-image://docker.io/docker/dockerfile:1@sha256:4edf897a3ffa55b89f906fc8cc78afdb3f1834cc9c7083565e611a8a7d5fe99e +#4 resolve docker.io/docker/dockerfile:1@sha256:4edf897a3ffa55b89f906fc8cc78afdb3f1834cc9c7083565e611a8a7d5fe99e 0.0s done +#4 CACHED + +#5 [api internal] load metadata for docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641 +#5 DONE 2.1s + +#6 [migrate internal] load .dockerignore +#6 transferring context: 196B done +#6 DONE 0.1s + +#7 [migrate 1/9] FROM docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641 +#7 resolve docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641 0.1s done +#7 DONE 0.1s + +#8 [api internal] load build context +#8 transferring context: 525.52kB 0.1s done +#8 transferring context: 525.52kB 0.1s done +#8 DONE 0.2s + +#9 [api 2/9] WORKDIR /app +#9 CACHED + +#10 [api 3/9] COPY requirements.txt ./ +#10 CACHED + +#11 [api 4/9] RUN pip install --root-user-action=ignore --require-hashes --no-deps -r requirements.txt +#11 CACHED + +#12 [api 5/9] RUN apt-get update && apt-get install -y --no-install-recommends libjpeg-turbo-progs=1:2.1.5-2 && rm -rf /var/lib/apt/lists/* +#12 CACHED + +#13 [api 6/9] RUN groupadd --system --gid 10001 app && useradd --system --uid 10001 --gid app --no-create-home --shell /usr/sbin/nologin app +#13 CACHED + +#14 [api 7/9] COPY src/ ./src/ +#14 DONE 0.1s + +#15 [api 8/9] COPY migrations/ ./migrations/ +#15 DONE 0.1s + +#16 [migrate 9/9] COPY contracts/openapi.json ./contracts/openapi.json +#16 DONE 0.1s + +#17 [api] exporting to image +#17 exporting layers +#17 exporting layers 0.3s done +#17 exporting manifest sha256:d5ce436c70dc889fae295b9c569550143a49023a8c8c276e83ec5b3e3e156d4d 0.0s done +#17 exporting config sha256:bd8b08e1de339150644db0527530d2ee10f7fdac39fb495916259bb92f8157ea 0.1s done +#17 exporting attestation manifest sha256:88b54cc6ae37ccc6d7bed5b2f499e6d2ad7b184c25c8f8da067db4010c4f341f 0.1s done +#17 exporting manifest list sha256:7689cb4ef3a9f3649524415f9528e0c84610759d6536e522dc5ea0d4305e3878 +#17 exporting manifest list sha256:7689cb4ef3a9f3649524415f9528e0c84610759d6536e522dc5ea0d4305e3878 0.0s done +#17 naming to docker.io/library/mtusi-messenger:lab5 0.0s done +#17 unpacking to docker.io/library/mtusi-messenger:lab5 +#17 unpacking to docker.io/library/mtusi-messenger:lab5 0.2s done +#17 DONE 0.8s + +#18 [migrate] exporting to image +#18 exporting layers 0.3s done +#18 exporting manifest sha256:9b9171737d004d0514b7adf53f37158be51e19440ab86d15192ef7d71bcf1ec0 0.1s done +#18 exporting config sha256:2b03ff83f38876a18738b5fcf9676e1c840ca6162cdee4dd6252e1cab242448e 0.1s done +#18 exporting attestation manifest sha256:b62c3a0d59a966c45c4de062852cb7c68895fe85e1e669dce3bfe885f907ac72 0.1s done +#18 exporting manifest list sha256:a3c335533065814c753479d7820efef885c03a58eb62edbf2e21d0f267be982a 0.0s done +#18 naming to docker.io/library/mtusi-messenger:lab5 done +#18 unpacking to docker.io/library/mtusi-messenger:lab5 0.2s done +#18 DONE 0.9s + +#19 [api] resolving provenance for metadata file +#19 DONE 0.0s + +#20 [migrate] resolving provenance for metadata file +#20 DONE 0.0s + Image mtusi-messenger:lab5 Building + Image mtusi-messenger:lab5 Building + Image mtusi-messenger:lab5 Built + Image mtusi-messenger:lab5 Built + Network mtusi-lab5_data Creating + Network mtusi-lab5_edge Creating + Network mtusi-lab5_data Creating + Network mtusi-lab5_queue Creating + Network mtusi-lab5_queue Creating + Volume mtusi-lab5_pgdata Creating + Volume mtusi-lab5_pgdata Creating + Volume mtusi-lab5_s3data Creating + Volume mtusi-lab5_s3data Creating + Volume mtusi-lab5_rabbitmqdata Creating + Volume mtusi-lab5_rabbitmqdata Creating + Volume mtusi-lab5_valkeydata Creating + Volume mtusi-lab5_valkeydata Creating + Network mtusi-lab5_edge Creating + Network mtusi-lab5_storage Creating + Network mtusi-lab5_storage Creating + Volume mtusi-lab5_valkeydata Created + Volume mtusi-lab5_valkeydata Created + Volume mtusi-lab5_pgdata Created + Volume mtusi-lab5_pgdata Created + Volume mtusi-lab5_s3data Created + Volume mtusi-lab5_s3data Created + Volume mtusi-lab5_rabbitmqdata Created + Volume mtusi-lab5_rabbitmqdata Created + Network mtusi-lab5_data Created + Network mtusi-lab5_data Created + Container mtusi-lab5-valkey-1 Creating + Container mtusi-lab5-postgres-1 Creating + Network mtusi-lab5_storage Created + Network mtusi-lab5_storage Created + Container mtusi-lab5-s3-1 Creating + Network mtusi-lab5_queue Created + Network mtusi-lab5_queue Created + Container mtusi-lab5-broker-1 Creating + Network mtusi-lab5_edge Created + Network mtusi-lab5_edge Created + Container mtusi-lab5-postgres-1 Created + Container mtusi-lab5-migrate-1 Creating + Container mtusi-lab5-valkey-1 Created + Container mtusi-lab5-broker-1 Created + Container mtusi-lab5-s3-1 Created + Container mtusi-lab5-migrate-1 Created + Container mtusi-lab5-s3-init-1 Creating + Container mtusi-lab5-relay-1 Creating + Container mtusi-lab5-s3-init-1 Created + Container mtusi-lab5-api-2 Creating + Container mtusi-lab5-api-1 Creating + Container mtusi-lab5-worker-1 Creating + Container mtusi-lab5-relay-1 Created + Container mtusi-lab5-worker-1 Created + Container mtusi-lab5-api-1 Created + Container mtusi-lab5-api-2 Created + Container mtusi-lab5-lb-1 Creating + Container mtusi-lab5-lb-1 Created + Container mtusi-lab5-s3-1 Starting + Container mtusi-lab5-valkey-1 Starting + Container mtusi-lab5-postgres-1 Starting + Container mtusi-lab5-broker-1 Starting + Container mtusi-lab5-s3-1 Started + Container mtusi-lab5-valkey-1 Started + Container mtusi-lab5-postgres-1 Started + Container mtusi-lab5-postgres-1 Waiting + Container mtusi-lab5-broker-1 Started + Container mtusi-lab5-postgres-1 Healthy + Container mtusi-lab5-migrate-1 Starting + Container mtusi-lab5-migrate-1 Started + Container mtusi-lab5-s3-1 Waiting + Container mtusi-lab5-postgres-1 Waiting + Container mtusi-lab5-broker-1 Waiting + Container mtusi-lab5-migrate-1 Waiting + Container mtusi-lab5-migrate-1 Waiting + Container mtusi-lab5-s3-1 Healthy + Container mtusi-lab5-postgres-1 Healthy + Container mtusi-lab5-migrate-1 Exited + Container mtusi-lab5-migrate-1 Exited + Container mtusi-lab5-s3-init-1 Starting + Container mtusi-lab5-s3-init-1 Started + Container mtusi-lab5-postgres-1 Waiting + Container mtusi-lab5-valkey-1 Waiting + Container mtusi-lab5-migrate-1 Waiting + Container mtusi-lab5-s3-1 Waiting + Container mtusi-lab5-s3-init-1 Waiting + Container mtusi-lab5-migrate-1 Waiting + Container mtusi-lab5-postgres-1 Waiting + Container mtusi-lab5-broker-1 Waiting + Container mtusi-lab5-s3-init-1 Waiting + Container mtusi-lab5-s3-1 Waiting + Container mtusi-lab5-postgres-1 Healthy + Container mtusi-lab5-migrate-1 Exited + Container mtusi-lab5-s3-1 Healthy + Container mtusi-lab5-s3-1 Healthy + Container mtusi-lab5-postgres-1 Healthy + Container mtusi-lab5-valkey-1 Healthy + Container mtusi-lab5-migrate-1 Exited + Container mtusi-lab5-broker-1 Healthy + Container mtusi-lab5-broker-1 Healthy + Container mtusi-lab5-relay-1 Starting + Container mtusi-lab5-relay-1 Started + Container mtusi-lab5-s3-init-1 Exited + Container mtusi-lab5-worker-1 Starting + Container mtusi-lab5-s3-init-1 Exited + Container mtusi-lab5-api-2 Starting + Container mtusi-lab5-worker-1 Started + Container mtusi-lab5-api-2 Started + Container mtusi-lab5-api-1 Starting + Container mtusi-lab5-api-1 Started + Container mtusi-lab5-api-2 Waiting + Container mtusi-lab5-api-1 Waiting + Container mtusi-lab5-api-2 Healthy + Container mtusi-lab5-api-1 Healthy + Container mtusi-lab5-lb-1 Starting + Container mtusi-lab5-lb-1 Started + Container mtusi-lab5-valkey-1 Waiting + Container mtusi-lab5-postgres-1 Waiting + Container mtusi-lab5-s3-init-1 Waiting + Container mtusi-lab5-migrate-1 Waiting + Container mtusi-lab5-api-2 Waiting + Container mtusi-lab5-api-1 Waiting + Container mtusi-lab5-s3-1 Waiting + Container mtusi-lab5-relay-1 Waiting + Container mtusi-lab5-broker-1 Waiting + Container mtusi-lab5-lb-1 Waiting + Container mtusi-lab5-worker-1 Waiting + Container mtusi-lab5-valkey-1 Healthy + Container mtusi-lab5-broker-1 Healthy + Container mtusi-lab5-worker-1 Healthy + Container mtusi-lab5-relay-1 Healthy + Container mtusi-lab5-postgres-1 Healthy + Container mtusi-lab5-migrate-1 Exited + Container mtusi-lab5-s3-1 Healthy + Container mtusi-lab5-s3-init-1 Exited + Container mtusi-lab5-api-2 Healthy + Container mtusi-lab5-api-1 Healthy + Container mtusi-lab5-lb-1 Healthy +[exit 0, 41.2 s] + +$ docker compose ps -a --format table {{.Service}} {{.Image}} {{.Status}} +SERVICE IMAGE STATUS +api mtusi-messenger:lab5 Up 11 seconds (healthy) +api mtusi-messenger:lab5 Up 12 seconds (healthy) +broker rabbitmq:4.3.7-alpine@sha256:b1f997f74103d08fc1f20a13890dc44fe6cfa51d528cf521b480ada988d3ffa5 Up 26 seconds (healthy) +lb haproxy:3.2.26-alpine@sha256:254e6370f01cd654e0744edf607c67f33777c1b6a9dc0d4b28bcba7171460122 Up 5 seconds (healthy) +migrate mtusi-messenger:lab5 Exited (0) 19 seconds ago +postgres postgres:18.6-bookworm@sha256:afc7e2d441324c0388fa80c3d24f733b4194a4eb7f47dd8ee2b08eb1a24a647c Up 26 seconds (healthy) +relay mtusi-messenger:lab5 Up 15 seconds (healthy) +s3 chrislusf/seaweedfs:4.46@sha256:08d516132314207d10c8e37cbffc1f32b147d870169688734cc61c6231625b62 Up 27 seconds (healthy) +s3-init mtusi-messenger:lab5 Exited (0) 12 seconds ago +valkey valkey/valkey:8.1.3-bookworm@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571 Up 27 seconds (healthy) +worker mtusi-messenger:lab5 Up 12 seconds (healthy) +[exit 0, 0.3 s] + +$ docker compose logs --no-log-prefix migrate +migrate: 0001_init: applied +migrate: 0002_idempotency_keys: applied +migrate: 0003_post_message_function: applied +migrate: 0004_attachments: applied +migrate: 0005_pipeline: applied +migrate: done, 5 migration file(s) checked +[exit 0, 0.3 s] + +$ docker compose logs --no-log-prefix s3-init +s3-bootstrap: bucket messenger-attachments: created +s3-bootstrap: bucket messenger-staging: created +s3-bootstrap: sentinel health/sentinel: written +s3-bootstrap: anonymous GET object of messenger-attachments: HTTP 403 -> refused +s3-bootstrap: anonymous LIST of messenger-attachments: HTTP 403 -> refused +s3-bootstrap: anonymous GET object of messenger-staging: HTTP 403 -> refused +s3-bootstrap: anonymous LIST of messenger-staging: HTTP 403 -> refused +s3-bootstrap: api: write staging: allowed -> OK +s3-bootstrap: api: delete staging (janitor): allowed -> OK +s3-bootstrap: api: read staging: refused (403) -> OK +s3-bootstrap: api: read final: allowed -> OK +s3-bootstrap: api: write final: refused (AccessDenied) -> OK +s3-bootstrap: api: list final: refused (AccessDenied) -> OK +s3-bootstrap: api: list staging: refused (AccessDenied) -> OK +s3-bootstrap: api: create bucket: refused (AccessDenied) -> OK +s3-bootstrap: api: read another bucket: refused (403) -> OK +s3-bootstrap: worker: read staging: allowed -> OK +s3-bootstrap: worker: write final: allowed -> OK +s3-bootstrap: worker: delete final: allowed -> OK +s3-bootstrap: worker: write staging: allowed -> OK +s3-bootstrap: worker: delete staging: allowed -> OK +s3-bootstrap: worker: list final: refused (AccessDenied) -> OK +s3-bootstrap: worker: create bucket: refused (AccessDenied) -> OK +s3-bootstrap: worker: read another bucket: refused (403) -> OK +s3-bootstrap: auditor: list final: allowed -> OK +s3-bootstrap: auditor: list staging: allowed -> OK +s3-bootstrap: auditor: read final: allowed -> OK +s3-bootstrap: auditor: write final: refused (AccessDenied) -> OK +s3-bootstrap: auditor: delete staging: refused (AccessDenied) -> OK +s3-bootstrap: api: buckets visible in ListBuckets: [] -> OK +s3-bootstrap: done: private buckets, least-privilege keys verified +[exit 0, 0.3 s] + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py replicas +mtusi-lab5-api-1 X-Instance-Id=d08185c0e3f3 running health=healthy pid=83074 started=2026-10-31T14:18:45.487 ip=172.25.0.7,172.26.0.5,172.28.0.3 +mtusi-lab5-api-2 X-Instance-Id=85c7cde066aa running health=healthy pid=82998 started=2026-10-31T14:18:45.132 ip=172.25.0.6,172.26.0.4,172.28.0.2 +[exit 0, 0.5 s] + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py status +api1 172.28.0.3 UP mtusi-lab5-api-1 X-Instance-Id=d08185c0e3f3 +api2 172.28.0.2 UP mtusi-lab5-api-2 X-Instance-Id=85c7cde066aa +(other slots: MAINT, no container) +[exit 0, 0.7 s] + +$ docker compose exec -T broker rabbitmqctl -q list_users +user tags +relay [] +worker [] +[exit 0, 2.0 s] + +$ docker compose exec -T broker rabbitmqctl -q list_queues name type durable messages consumers arguments +name type durable messages consumers arguments +messenger.attachment-jobs quorum true 0 1 [{"x-dead-letter-exchange","messenger.jobs.dlx"},{"x-delivery-limit",20},{"x-queue-type","quorum"}] +messenger.attachment-jobs.dlq quorum true 0 0 [{"x-queue-type","quorum"}] +[exit 0, 2.0 s] + +$ docker compose exec -T broker rabbitmqctl -q list_permissions +user configure write read +relay ^$ ^messenger\.jobs$ ^$ +worker ^$ ^$ ^messenger\.attachment-jobs$ +[exit 0, 3.9 s] + +$ docker inspect (limits) +/mtusi-lab5-api-1 cpus=1000000000 memory=536870912 memswap=536870912 image=mtusi-messenger:lab5 +/mtusi-lab5-api-2 cpus=1000000000 memory=536870912 memswap=536870912 image=mtusi-messenger:lab5 +/mtusi-lab5-broker-1 cpus=1000000000 memory=536870912 memswap=1073741824 image=rabbitmq:4.3.7-alpine@sha256:b1f997f74103d08fc1f20a13890dc44fe6cfa51d528cf521b480ada988d3ffa5 +/mtusi-lab5-lb-1 cpus=1000000000 memory=134217728 memswap=268435456 image=haproxy:3.2.26-alpine@sha256:254e6370f01cd654e0744edf607c67f33777c1b6a9dc0d4b28bcba7171460122 +/mtusi-lab5-postgres-1 cpus=2000000000 memory=1073741824 memswap=2147483648 image=postgres:18.6-bookworm@sha256:afc7e2d441324c0388fa80c3d24f733b4194a4eb7f47dd8ee2b08eb1a24a647c +/mtusi-lab5-relay-1 cpus=500000000 memory=268435456 memswap=536870912 image=mtusi-messenger:lab5 +/mtusi-lab5-s3-1 cpus=1000000000 memory=1073741824 memswap=2147483648 image=chrislusf/seaweedfs:4.46@sha256:08d516132314207d10c8e37cbffc1f32b147d870169688734cc61c6231625b62 +/mtusi-lab5-valkey-1 cpus=500000000 memory=268435456 memswap=536870912 image=valkey/valkey:8.1.3-bookworm@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571 +/mtusi-lab5-worker-1 cpus=2000000000 memory=1610612736 memswap=1610612736 image=mtusi-messenger:lab5 +[exit 0, 0.1 s] + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe -m venv .venv + +[exit 0, 7.0 s] + +$ C:\Users\wfa22\Desktop\Высоконагруженные приложения\lab5-clean\.venv\Scripts\python.exe -m pip install -q --require-hashes --no-deps -r requirements.txt + +[notice] A new release of pip is available: 26.1.2 -> 26.2.2 +[notice] To update, run: C:\Users\wfa22\Desktop\Высоконагруженные приложения\lab5-clean\.venv\Scripts\python.exe -m pip install --upgrade pip +[exit 0, 28.9 s] + +$ C:\Users\wfa22\Desktop\Высоконагруженные приложения\lab5-clean\.venv\Scripts\python.exe -m pip install -q -r requirements-test.txt -r requirements-dev.txt + +[notice] A new release of pip is available: 26.1.2 -> 26.2.2 +[notice] To update, run: C:\Users\wfa22\Desktop\Высоконагруженные приложения\lab5-clean\.venv\Scripts\python.exe -m pip install --upgrade pip +[exit 0, 16.0 s] diff --git a/evidence/demo-pipeline.txt b/evidence/demo-pipeline.txt new file mode 100644 index 0000000..39716e1 --- /dev/null +++ b/evidence/demo-pipeline.txt @@ -0,0 +1,60 @@ +# Defense steps 2-3: worker stopped, upload 202, content 409, restart API and broker with volumes, worker started, ready + thumbnail +# 2026-10-31T14:22:23Z, commit cf6e929 docs: PIPELINE, DEFENSE, DATA_MODEL, WINDOWS для lab5; черновик REPORT + +Containers BEFORE: +mtusi-lab5-api-1 host=d08185c0e3f3 running exit=0 pid=94963 started=2026-10-31T14:21:47.426 +mtusi-lab5-api-2 host=85c7cde066aa running exit=0 pid=96896 started=2026-10-31T14:22:15.716 +mtusi-lab5-broker-1 host=broker running exit=0 pid=89550 started=2026-10-31T14:20:17.480 +mtusi-lab5-lb-1 host=a22e1c19fc90 running exit=0 pid=92390 started=2026-10-31T14:21:12.759 +mtusi-lab5-migrate-1 host=fa0a8a1c9191 exited exit=0 pid=0 started=2026-10-31T14:21:14.217 +mtusi-lab5-postgres-1 host=7961919f90c9 running exit=0 pid=89433 started=2026-10-31T14:20:16.605 +mtusi-lab5-relay-1 host=9b6249446ece running exit=0 pid=92854 started=2026-10-31T14:21:16.976 +mtusi-lab5-s3-1 host=bdee3ea030a4 running exit=0 pid=90402 started=2026-10-31T14:20:36.421 +mtusi-lab5-s3-init-1 host=2c85c6921c88 exited exit=0 pid=0 started=2026-10-31T14:21:17.299 +mtusi-lab5-valkey-1 host=d53b8a88e5b5 running exit=0 pid=89376 started=2026-10-31T14:20:16.424 +mtusi-lab5-worker-1 host=ee8c11f59aa8 running exit=0 pid=93271 started=2026-10-31T14:21:22.639 + +$ python scripts/demo_pipeline.py + +[17:22:24] 1. docker compose stop worker + worker exited + +[17:22:26] 2. upload while the worker is stopped + POST /chats/{id}/attachments -> 202 [X-Instance-Id 85c7cde066aa] {"id":"e9d4a7b9-a5da-4c05-a12d-abb95c9a1ff5","chat_id":"d60cc7c6-9ce0-4a81-90ca-3fb3e0e4abbb","owner_id":"7ef6ca59-758c-4753-8037-d00537802aba","filename":"sample.png","content_type":"image/png","size_bytes":111817,"sha256":"99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75","status": + attachment_id = e9d4a7b9-a5da-4c05-a12d-abb95c9a1ff5 status = queued + GET /attachments/{id}/content -> 409 [X-Instance-Id d08185c0e3f3] {"error":{"code":"conflict","message":"Вложение ещё не готово","request_id":"3c6fd35c-0ce9-4f9f-8551-8b9a4eebb88c"}} + GET /attachments/{id}/content?variant=thumbnail -> 409 [X-Instance-Id 85c7cde066aa] {"error":{"code":"conflict","message":"Вложение ещё не готово","request_id":"d017985f-e358-4a88-9459-70e377707782"}} + POST /messages (text only) -> 201 [X-Instance-Id d08185c0e3f3] {"id":"fda54822-f633-4f82-946c-fcee50e0e124","chat_id":"d60cc7c6-9ce0-4a81-90ca-3fb3e0e4abbb","sender_id":"41252a17-4106-481a-8a43-a5187721c7b9","text":"обычное сообщение работает","created_at":"2026-10-31T14:22:26.348105Z","attachment_ids":[]} + POST /messages attachment_ids=[queued id] -> 201 [X-Instance-Id 85c7cde066aa] {"id":"279910fd-0f7c-4123-bd16-ae3fe1e52833","chat_id":"d60cc7c6-9ce0-4a81-90ca-3fb3e0e4abbb","sender_id":"7ef6ca59-758c-4753-8037-d00537802aba","text":"фото","created_at":"2026-10-31T14:22:26.360988Z","attachment_ids":["e9d4a7b9-a5da-4c05-a12d-abb95c9a1ff5"]} + job + outbox in PostgreSQL: c59c75a5-67d1-48e6-95cd-b6af894c4e4b state=queued attempt=0 | outbox published_at=2026-10-31 14:22:26.626499+00 + RabbitMQ queue messenger.attachment-jobs: messages=1 consumers=0 + +[17:22:38] 3. docker compose restart api broker (volumes preserved) + RabbitMQ queue after the restart: messages=1 consumers=0 + GET /attachments/{id} -> 200 [X-Instance-Id 85c7cde066aa] {"id":"e9d4a7b9-a5da-4c05-a12d-abb95c9a1ff5","chat_id":"d60cc7c6-9ce0-4a81-90ca-3fb3e0e4abbb","owner_id":"7ef6ca59-758c-4753-8037-d00537802aba","filename":"sample.png","content_type":"image/png","size_bytes":111817,"sha256":"99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75","status": + +[17:23:00] 4. docker compose start worker + GET /attachments/{id} until ready -> 200 [X-Instance-Id 85c7cde066aa] {"id":"e9d4a7b9-a5da-4c05-a12d-abb95c9a1ff5","chat_id":"d60cc7c6-9ce0-4a81-90ca-3fb3e0e4abbb","owner_id":"7ef6ca59-758c-4753-8037-d00537802aba","filename":"sample.png","content_type":"image/png","size_bytes":111817,"sha256":"99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75","status": + GET /content?variant=thumbnail -> 200 [X-Instance-Id d08185c0e3f3] 4679 bytes, Content-Type image/jpeg + thumbnail: real JPEG 256x160, sha256 matches metadata: True + original unchanged byte for byte: True + message 279910fd-0f7c-4123-bd16-ae3fe1e52833 still references it: {e9d4a7b9-a5da-4c05-a12d-abb95c9a1ff5} + +[17:23:13] 5. job history (jobctl show) + 2026-10-31 14:22:26.316695+00:00 created attempt=None actor=api:85c7cde066aa + 2026-10-31 14:23:12.130790+00:00 claimed attempt=1 actor=worker:ee8c11f59aa8 + 2026-10-31 14:23:12.981082+00:00 succeeded attempt=1 actor=worker:ee8c11f59aa8 +[exit 0, 53.7 s] + +Containers AFTER (api-*, broker restarted; worker stopped and started): +mtusi-lab5-api-1 host=d08185c0e3f3 running exit=0 pid=99058 started=2026-10-31T14:22:43.321 +mtusi-lab5-api-2 host=85c7cde066aa running exit=0 pid=98980 started=2026-10-31T14:22:42.925 +mtusi-lab5-broker-1 host=broker running exit=0 pid=98657 started=2026-10-31T14:22:40.329 +mtusi-lab5-lb-1 host=a22e1c19fc90 running exit=0 pid=92390 started=2026-10-31T14:21:12.759 +mtusi-lab5-migrate-1 host=fa0a8a1c9191 exited exit=0 pid=0 started=2026-10-31T14:23:01.939 +mtusi-lab5-postgres-1 host=7961919f90c9 running exit=0 pid=89433 started=2026-10-31T14:20:16.605 +mtusi-lab5-relay-1 host=9b6249446ece running exit=0 pid=92854 started=2026-10-31T14:21:16.976 +mtusi-lab5-s3-1 host=bdee3ea030a4 running exit=0 pid=90402 started=2026-10-31T14:20:36.421 +mtusi-lab5-s3-init-1 host=2c85c6921c88 exited exit=0 pid=0 started=2026-10-31T14:23:04.208 +mtusi-lab5-valkey-1 host=d53b8a88e5b5 running exit=0 pid=89376 started=2026-10-31T14:20:16.424 +mtusi-lab5-worker-1 host=ee8c11f59aa8 running exit=0 pid=1259 started=2026-10-31T14:23:09.985 diff --git a/evidence/load.json b/evidence/load.json new file mode 100644 index 0000000..6b56abf --- /dev/null +++ b/evidence/load.json @@ -0,0 +1,17 @@ +{ + "lab": 5, + "workload": "closed-loop POST messages; no automatic retries", + "requested_duration_seconds": 30.0, + "elapsed_seconds": 30.066, + "concurrency": 4, + "successful_requests": 5515, + "failed_requests": 0, + "error_ratio": 0.0, + "successful_rps": 183.429, + "successful_latency_ms": { + "p50": 17.96, + "p95": 43.491, + "p99": 58.653 + }, + "note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead." +} diff --git a/evidence/make-check-validate-test.txt b/evidence/make-check-validate-test.txt new file mode 100644 index 0000000..de92cbc --- /dev/null +++ b/evidence/make-check-validate-test.txt @@ -0,0 +1,17 @@ +# make check, make validate (venv), make test +# 2026-10-31T14:19:59Z, commit cf6e929 docs: PIPELINE, DEFENSE, DATA_MODEL, WINDOWS для lab5; черновик REPORT + +$ make check +python scripts/check_contract.py +PASS: offline skeleton structure, local refs and Python syntax; application NOT tested. +[exit 0, 1.2 s] + +$ make validate PYTHON=.venv/Scripts/python.exe +.venv/Scripts/python.exe scripts/validate_openapi.py +PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested. +[exit 0, 2.3 s] + +$ make test +python tests/smoke.py --base-url "http://localhost:8080" +PASS: lab5 public API scenario; infrastructure and grading evidence remain separate. +[exit 0, 3.3 s] diff --git a/evidence/student-tests.txt b/evidence/student-tests.txt new file mode 100644 index 0000000..d202f43 --- /dev/null +++ b/evidence/student-tests.txt @@ -0,0 +1,394 @@ +# Own tests: unit + stand (pytest -v); failures restored in finally +# 2026-10-31T14:23:18Z, commit cf6e929 docs: PIPELINE, DEFENSE, DATA_MODEL, WINDOWS для lab5; черновик REPORT + +$ python -m pytest student-tests -v +============================= test session starts ============================= +platform win32 -- Python 3.14.6, pytest-9.1.1, pluggy-1.7.0 -- C:\Users\wfa22\Desktop\Высоконагруженные приложения\lab5-clean\.venv\Scripts\python.exe +rootdir: C:\Users\wfa22\Desktop\Высоконагруженные приложения\lab5-clean +plugins: anyio-4.15.1 +collecting ... collected 381 items + +student-tests/test_api.py::test_register_never_returns_password_and_login_returns_bearer PASSED [ 0%] +student-tests/test_api.py::test_duplicate_username_is_409 PASSED [ 0%] +student-tests/test_api.py::test_bad_password_on_registration_is_400[short-11ch] PASSED [ 0%] +student-tests/test_api.py::test_bad_password_on_registration_is_400[ ] PASSED [ 1%] +student-tests/test_api.py::test_bad_password_on_registration_is_400[ \t \n \t \n \t \n ] PASSED [ 1%] +student-tests/test_api.py::test_bad_password_on_registration_is_400[\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f] PASSED [ 1%] +student-tests/test_api.py::test_edge_passwords_register_and_login[\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f] PASSED [ 1%] +student-tests/test_api.py::test_edge_passwords_register_and_login[with:colons:inside] PASSED [ 2%] +student-tests/test_api.py::test_edge_passwords_register_and_login[Twelve chars] PASSED [ 2%] +student-tests/test_api.py::test_wrong_password_and_unknown_user_give_the_same_401 PASSED [ 2%] +student-tests/test_api.py::test_malformed_basic_is_401_not_500[Basic \xe9] PASSED [ 2%] +student-tests/test_api.py::test_malformed_basic_is_401_not_500[Basic !!!] PASSED [ 3%] +student-tests/test_api.py::test_malformed_basic_is_401_not_500[Basic bm9fY29sb24=] PASSED [ 3%] +student-tests/test_api.py::test_malformed_basic_is_401_not_500[Basic //46eA==] PASSED [ 3%] +student-tests/test_api.py::test_malformed_basic_is_401_not_500[Bearer aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 3%] +student-tests/test_api.py::test_malformed_basic_is_401_not_500[] PASSED [ 4%] +student-tests/test_api.py::test_logout_revokes_only_this_token PASSED [ 4%] +student-tests/test_api.py::test_x_user_id_basic_and_garbage_do_not_authenticate PASSED [ 4%] +student-tests/test_api.py::test_malformed_json_is_400[{] PASSED [ 4%] +student-tests/test_api.py::test_malformed_json_is_400[] PASSED [ 5%] +student-tests/test_api.py::test_malformed_json_is_400[[]] PASSED [ 5%] +student-tests/test_api.py::test_malformed_json_is_400[null] PASSED [ 5%] +student-tests/test_api.py::test_malformed_json_is_400[{"username": "abc_def", "display_name": NaN, "password": "Twelve-chars"}] PASSED [ 6%] +student-tests/test_api.py::test_malformed_json_is_400[{"username": "abc_def", "display_name": "x", "password": "Twelve-chars"}{"a": 1}] PASSED [ 6%] +student-tests/test_api.py::test_malformed_json_is_400[{"username": "abc_def", "display_name": "\ud800", "password": "Twelve-chars"}] PASSED [ 6%] +student-tests/test_api.py::test_invalid_registration_fields_are_400[body0] PASSED [ 6%] +student-tests/test_api.py::test_invalid_registration_fields_are_400[body1] PASSED [ 7%] +student-tests/test_api.py::test_invalid_registration_fields_are_400[body2] PASSED [ 7%] +student-tests/test_api.py::test_invalid_registration_fields_are_400[body3] PASSED [ 7%] +student-tests/test_api.py::test_invalid_registration_fields_are_400[body4] PASSED [ 7%] +student-tests/test_api.py::test_invalid_registration_fields_are_400[body5] PASSED [ 8%] +student-tests/test_api.py::test_unicode_limits_are_code_points_and_nul_is_stored PASSED [ 8%] +student-tests/test_api.py::test_oversized_body_and_wrong_content_type PASSED [ 8%] +student-tests/test_api.py::test_chat_validation PASSED [ 8%] +student-tests/test_api.py::test_foreign_chat_is_indistinguishable_from_missing PASSED [ 9%] +student-tests/test_api.py::test_chat_list_order_pagination_and_isolation PASSED [ 9%] +student-tests/test_api.py::test_history_pages_without_loss_or_duplicates PASSED [ 9%] +student-tests/test_api.py::test_bad_limit_or_cursor_is_400[limit=0] PASSED [ 9%] +student-tests/test_api.py::test_bad_limit_or_cursor_is_400[limit=101] PASSED [ 10%] +student-tests/test_api.py::test_bad_limit_or_cursor_is_400[limit=abc] PASSED [ 10%] +student-tests/test_api.py::test_bad_limit_or_cursor_is_400[limit=1&limit=2] PASSED [ 10%] +student-tests/test_api.py::test_bad_limit_or_cursor_is_400[cursor=] PASSED [ 11%] +student-tests/test_api.py::test_bad_limit_or_cursor_is_400[cursor=garbage] PASSED [ 11%] +student-tests/test_api.py::test_bad_limit_or_cursor_is_400[cursor=djF8bXx4fDF8Mg] PASSED [ 11%] +student-tests/test_api.py::test_bad_limit_or_cursor_is_400[cursor=djF8bXx4fDk5OTk5OTk5OTk5OTk5OTk5OTk5fHk=] PASSED [ 11%] +student-tests/test_api.py::test_cursor_bound_to_chat PASSED [ 12%] +student-tests/test_api.py::test_read_after_write PASSED [ 12%] +student-tests/test_api.py::test_malformed_cursor_values_are_400_on_both_lists PASSED [ 12%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[\xff] PASSED [ 12%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"first","TEXT":"second"}] PASSED [ 13%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"a","text":"b"}] PASSED [ 13%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":123}] PASSED [ 13%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"valid"}}] PASSED [ 13%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"valid"}]] PASSED [ 14%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"x"} {}] PASSED [ 14%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"x","sender_id":"11111111-1111-4111-8111-111111111111"}] PASSED [ 14%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"x","attachment_ids":null}] PASSED [ 14%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"x","attachment_ids":["nope"]}] PASSED [ 15%] +student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":" \n\t "}] PASSED [ 15%] +student-tests/test_api.py::test_ff_byte_body_on_registration_is_contract_error PASSED [ 15%] +student-tests/test_api.py::test_text_is_stored_verbatim_and_length_counts_raw_text PASSED [ 16%] +student-tests/test_api.py::test_unknown_attachment_ids_are_404 PASSED [ 16%] +student-tests/test_api.py::test_member_ids_are_compared_as_uuids_not_strings PASSED [ 16%] +student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[Bearer \xe9] PASSED [ 16%] +student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[Bearer .] PASSED [ 17%] +student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[Bearer a b] PASSED [ 17%] +student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[Bearer \xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9] PASSED [ 17%] +student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[bearer\taaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 17%] +student-tests/test_api.py::test_login_with_a_body_is_rejected PASSED [ 18%] +student-tests/test_api.py::test_password_range_is_12_to_128_code_points PASSED [ 18%] +student-tests/test_api.py::test_every_api_response_carries_instance_and_request_id PASSED [ 18%] +student-tests/test_api.py::test_unicode_whitespace_only_strings_are_400[\xa0] PASSED [ 18%] +student-tests/test_api.py::test_unicode_whitespace_only_strings_are_400[\u2003\u2003] PASSED [ 19%] +student-tests/test_api.py::test_unicode_whitespace_only_strings_are_400[ \xa0\t\u3000] PASSED [ 19%] +student-tests/test_api.py::test_odd_registration_bodies_are_400_and_the_api_survives[{"username":"num_probe","display_name":"x","password":1e1000}] PASSED [ 19%] +student-tests/test_api.py::test_odd_registration_bodies_are_400_and_the_api_survives[{"username":"ff_probe","display_name":"x","password":"12345678901\xff"}] PASSED [ 19%] +student-tests/test_api.py::test_odd_registration_bodies_are_400_and_the_api_survives[{"username":"num_probe","display_name":1,"password":"Twelve-chars-ok"}] PASSED [ 20%] +student-tests/test_api.py::test_more_than_100_members_is_400 PASSED [ 20%] +student-tests/test_api.py::test_cursor_of_one_list_is_rejected_by_the_other PASSED [ 20%] +student-tests/test_attachments.py::test_png_and_jpeg_roundtrip_byte_for_byte PASSED [ 20%] +student-tests/test_attachments.py::test_metadata_reveals_no_storage_details PASSED [ 21%] +student-tests/test_attachments.py::test_original_is_not_stored_on_api_containers PASSED [ 21%] +student-tests/test_attachments.py::test_outsider_and_unknown_get_404 PASSED [ 21%] +student-tests/test_attachments.py::test_attachment_ids_are_checked_per_chat PASSED [ 22%] +student-tests/test_attachments.py::test_message_with_attachments_and_history PASSED [ 22%] +student-tests/test_attachments.py::test_idempotent_replay_with_attachments PASSED [ 22%] +student-tests/test_attachments.py::test_message_body_validation[{"attachment_ids":["%(a)s"]}] PASSED [ 22%] +student-tests/test_attachments.py::test_message_body_validation[{"text":null,"attachment_ids":["%(a)s"]}] PASSED [ 23%] +student-tests/test_attachments.py::test_message_body_validation[{"text":1,"attachment_ids":["%(a)s"]}] PASSED [ 23%] +student-tests/test_attachments.py::test_message_body_validation[{"text":"x","attachment_ids":null}] PASSED [ 23%] +student-tests/test_attachments.py::test_message_body_validation[{"text":"x","attachment_ids":"%(a)s"}] PASSED [ 23%] +student-tests/test_attachments.py::test_message_body_validation[{"text":"x","attachment_ids":{}}] PASSED [ 24%] +student-tests/test_attachments.py::test_message_body_validation[{"text":"x","attachment_ids":["%(a)s","%] PASSED [ 24%] +student-tests/test_attachments.py::test_message_body_validation[{"text":"x","attachment_ids":["%(a)s",1]] PASSED [ 24%] +student-tests/test_attachments.py::test_message_body_validation[{"text":"x","attachment_ids":["not-a-uui] PASSED [ 24%] +student-tests/test_attachments.py::test_message_body_validation[{"text":"","attachment_ids":[]}] PASSED [ 25%] +student-tests/test_attachments.py::test_message_body_validation[{"text":" "}] PASSED [ 25%] +student-tests/test_attachments.py::test_message_body_validation[{"text":"x","attachment_ids":[],"sender_] PASSED [ 25%] +student-tests/test_attachments.py::test_at_most_ten_attachments PASSED [ 25%] +student-tests/test_attachments.py::test_size_limit_through_the_load_balancer PASSED [ 26%] +student-tests/test_attachments.py::test_pixel_limit_before_decoding PASSED [ 26%] +student-tests/test_attachments.py::test_exactly_40_megapixels_is_accepted PASSED [ 26%] +student-tests/test_attachments.py::test_bad_content_is_415[png: first 33 bytes of the fixture] PASSED [ 27%] +student-tests/test_attachments.py::test_bad_content_is_415[png: truncated] PASSED [ 27%] +student-tests/test_attachments.py::test_bad_content_is_415[png: corrupt IDAT, valid CRC] PASSED [ 27%] +student-tests/test_attachments.py::test_bad_content_is_415[png: fewer rows than IHDR] PASSED [ 27%] +student-tests/test_attachments.py::test_bad_content_is_415[png: flipped byte] PASSED [ 28%] +student-tests/test_attachments.py::test_bad_content_is_415[png: bytes after IEND] PASSED [ 28%] +student-tests/test_attachments.py::test_bad_content_is_415[jpeg: no scan data after SOS] PASSED [ 28%] +student-tests/test_attachments.py::test_bad_content_is_415[jpeg: truncated] PASSED [ 28%] +student-tests/test_attachments.py::test_bad_content_is_415[review: png row filter 5] PASSED [ 29%] +student-tests/test_attachments.py::test_bad_content_is_415[review: png row filter 0xff] PASSED [ 29%] +student-tests/test_attachments.py::test_bad_content_is_415[review: png IDAT not-a-zlib-stream] PASSED [ 29%] +student-tests/test_attachments.py::test_bad_content_is_415[review: jpeg 27 bytes, no tables/scan] PASSED [ 29%] +student-tests/test_attachments.py::test_bad_content_is_415[review: jpeg SOS component not in SOF] PASSED [ 30%] +student-tests/test_attachments.py::test_bad_content_is_415[mime: png bytes declared jpeg] PASSED [ 30%] +student-tests/test_attachments.py::test_bad_content_is_415[mime: jpeg bytes declared png] PASSED [ 30%] +student-tests/test_attachments.py::test_bad_content_is_415[mime: gif declared png] PASSED [ 30%] +student-tests/test_attachments.py::test_bad_content_is_415[mime: html declared png] PASSED [ 31%] +student-tests/test_attachments.py::test_bad_content_is_415[type: text/plain] PASSED [ 31%] +student-tests/test_attachments.py::test_bad_content_is_415[type: octet-stream] PASSED [ 31%] +student-tests/test_attachments.py::test_bad_content_is_415[type: multipart] PASSED [ 32%] +student-tests/test_attachments.py::test_bad_content_is_415[type: missing] PASSED [ 32%] +student-tests/test_attachments.py::test_empty_body_is_400 PASSED [ 32%] +student-tests/test_attachments.py::test_bad_filename_is_400[missing] PASSED [ 32%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=] PASSED [ 33%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=%20%20%20] PASSED [ 33%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=.] PASSED [ 33%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=..] PASSED [ 33%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=a%2Fb.png] PASSED [ 34%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=a%5Cb.png] PASSED [ 34%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=a%00b.png] PASSED [ 34%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=c1%C2%85.png] PASSED [ 34%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=tab%09.png] PASSED [ 35%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=photo%E2%80%AEgnp.exe] PASSED [ 35%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=%FF%FE.png] PASSED [ 35%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=a.png&filename=b.png] PASSED [ 35%] +student-tests/test_attachments.py::test_bad_filename_is_400[filename=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 36%] +student-tests/test_attachments.py::test_unicode_filename_is_kept_and_safely_exposed PASSED [ 36%] +student-tests/test_attachments.py::test_variant_parameter PASSED [ 36%] +student-tests/test_balancing.py::test_one_session_is_served_by_every_real_replica PASSED [ 37%] +student-tests/test_balancing.py::test_write_on_one_replica_is_read_on_the_other PASSED [ 37%] +student-tests/test_balancing.py::test_lb_stops_routing_to_a_replica_that_is_alive_but_not_ready[0-pg_hba] PASSED [ 37%] +student-tests/test_balancing.py::test_lb_stops_routing_to_a_replica_that_is_alive_but_not_ready[0-network] PASSED [ 37%] +student-tests/test_balancing.py::test_lb_stops_routing_to_a_replica_that_is_alive_but_not_ready[1-pg_hba] PASSED [ 38%] +student-tests/test_balancing.py::test_lb_stops_routing_to_a_replica_that_is_alive_but_not_ready[1-network] PASSED [ 38%] +student-tests/test_balancing.py::test_graceful_stop_of_each_replica_loses_no_request[0] PASSED [ 38%] +student-tests/test_balancing.py::test_graceful_stop_of_each_replica_loses_no_request[1] PASSED [ 38%] +student-tests/test_balancing.py::test_killed_replica_bounded_errors_recovery_and_no_duplicates[0] PASSED [ 39%] +student-tests/test_balancing.py::test_killed_replica_bounded_errors_recovery_and_no_duplicates[1] PASSED [ 39%] +student-tests/test_balancing.py::test_graceful_shutdown_order_in_logs PASSED [ 39%] +student-tests/test_balancing.py::test_full_restart_lb_rediscovers_all_replicas PASSED [ 39%] +student-tests/test_balancing.py::test_scale_up_and_down_is_picked_up_without_errors PASSED [ 40%] +student-tests/test_balancing.py::test_compose_policy_limits_ports_secrets_images PASSED [ 40%] +student-tests/test_balancing.py::test_running_containers_have_limits_and_no_bypass_ports PASSED [ 40%] +student-tests/test_balancing.py::test_migrations_ran_once_by_the_migrate_service PASSED [ 40%] +student-tests/test_consistency.py::test_interrupted_and_stalled_uploads_leave_nothing PASSED [ 41%] +student-tests/test_consistency.py::test_crash_after_staging_before_job_is_cleaned_by_janitor PASSED [ 41%] +student-tests/test_consistency.py::test_lost_answer_after_job_commit_still_becomes_ready PASSED [ 41%] +student-tests/test_consistency.py::test_fresh_pending_upload_is_not_touched_by_janitor PASSED [ 41%] +student-tests/test_db.py::test_registration_race_exactly_one_201_and_seven_409 PASSED [ 42%] +student-tests/test_db.py::test_create_chat_rolls_back_after_the_chat_row_was_inserted PASSED [ 42%] +student-tests/test_db.py::test_migrations_rerun_is_a_noop PASSED [ 42%] +student-tests/test_db.py::test_100_messages_from_10_clients_keep_order_and_cursor_walk PASSED [ 43%] +student-tests/test_db.py::test_secrets_are_not_stored_in_plain_text PASSED [ 43%] +student-tests/test_explain.py::test_late_page_uses_index_range_not_filter PASSED [ 43%] +student-tests/test_failures.py::test_valkey_failure[stop] PASSED [ 43%] +student-tests/test_failures.py::test_valkey_failure[pause] PASSED [ 44%] +student-tests/test_failures.py::test_postgres_failure[stop] PASSED [ 44%] +student-tests/test_failures.py::test_postgres_failure[pause] PASSED [ 44%] +student-tests/test_failures.py::test_locked_table_gives_bounded_503 PASSED [ 44%] +student-tests/test_failures.py::test_logout_during_valkey_outage_is_never_lost PASSED [ 45%] +student-tests/test_idempotency.py::test_replay_returns_the_same_201_and_message_from_every_replica PASSED [ 45%] +student-tests/test_idempotency.py::test_other_body_with_the_same_key_is_409_and_keeps_the_original PASSED [ 45%] +student-tests/test_idempotency.py::test_key_order_and_missing_attachment_ids_are_the_same_body PASSED [ 45%] +student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[] PASSED [ 46%] +student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 46%] +student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[\u043a\u043b\u044e\u0447] PASSED [ 46%] +student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[a b] PASSED [ 46%] +student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[a/b] PASSED [ 47%] +student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[a,b] PASSED [ 47%] +student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[a\tb] PASSED [ 47%] +student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[*] PASSED [ 48%] +student-tests/test_idempotency.py::test_repeated_key_header_is_400 PASSED [ 48%] +student-tests/test_idempotency.py::test_longest_key_and_every_allowed_character PASSED [ 48%] +student-tests/test_idempotency.py::test_scope_is_user_plus_chat_plus_key PASSED [ 48%] +student-tests/test_idempotency.py::test_outsider_and_revoked_session_never_see_the_stored_result PASSED [ 49%] +student-tests/test_idempotency.py::test_concurrent_same_key_through_different_replicas_creates_exactly_one PASSED [ 49%] +student-tests/test_idempotency.py::test_concurrent_same_key_different_bodies_one_wins PASSED [ 49%] +student-tests/test_idempotency.py::test_failure_after_the_key_insert_rolls_everything_back PASSED [ 49%] +student-tests/test_idempotency.py::test_client_timeout_then_retry_with_the_same_key_gives_the_committed_message PASSED [ 50%] +student-tests/test_idempotency.py::test_lock_timeout_is_a_transient_503_that_is_not_recorded PASSED [ 50%] +student-tests/test_idempotency.py::test_replay_after_the_serving_replica_is_stopped PASSED [ 50%] +student-tests/test_pipeline_flow.py::test_upload_is_202_queued_then_ready_with_contract_variants PASSED [ 50%] +student-tests/test_pipeline_flow.py::test_queued_attachment_is_attachable_failed_is_not PASSED [ 51%] +student-tests/test_pipeline_flow.py::test_worker_stopped_upload_202_content_409_restart_api_and_broker_then_ready PASSED [ 51%] +student-tests/test_pipeline_flow.py::test_broker_down_commit_to_publish_gap_is_closed_by_the_outbox PASSED [ 51%] +student-tests/test_pipeline_flow.py::test_relay_down_job_waits_in_outbox_not_lost PASSED [ 51%] +student-tests/test_pipeline_flow.py::test_crash_after_publish_before_confirm_is_recorded_gives_a_harmless_duplicate PASSED [ 52%] +student-tests/test_pipeline_flow.py::test_duplicate_and_reordered_deliveries_are_noops PASSED [ 52%] +student-tests/test_pipeline_flow.py::test_worker_crash_before_side_effect_lease_expires_and_job_completes PASSED [ 52%] +student-tests/test_pipeline_flow.py::test_worker_crash_after_s3_write_before_commit_republishes_without_duplicates PASSED [ 53%] +student-tests/test_pipeline_flow.py::test_worker_crash_after_commit_before_ack_redelivery_is_noop PASSED [ 53%] +student-tests/test_pipeline_flow.py::test_transient_errors_are_retried_with_jittered_backoff PASSED [ 53%] +student-tests/test_pipeline_flow.py::test_retry_exhausted_failed_registry_then_replay PASSED [ 53%] +student-tests/test_pipeline_flow.py::test_permanent_error_fails_at_once_without_retries PASSED [ 54%] +student-tests/test_pipeline_flow.py::test_memory_and_time_limits_of_the_worker PASSED [ 54%] +student-tests/test_pipeline_flow.py::test_bad_message_is_dead_lettered_not_processed PASSED [ 54%] +student-tests/test_pipeline_flow.py::test_hung_worker_loses_its_lease_and_cannot_publish PASSED [ 54%] +student-tests/test_pipeline_flow.py::test_graceful_worker_stop_finishes_the_running_attempt PASSED [ 55%] +student-tests/test_pipeline_flow.py::test_concurrency_is_bounded_by_prefetch PASSED [ 55%] +student-tests/test_pipeline_flow.py::test_queued_staging_is_never_deleted_even_after_retention PASSED [ 55%] +student-tests/test_pipeline_flow.py::test_retention_deletes_failed_staging_then_replay_is_refused PASSED [ 55%] +student-tests/test_pipeline_flow.py::test_cleanup_racing_a_replay_never_deletes_the_replayed_source PASSED [ 56%] +student-tests/test_pipeline_flow.py::test_reprocess_of_a_not_ready_upload_is_refused PASSED [ 56%] +student-tests/test_pipeline_flow.py::test_zz_invariants_hold_after_all_scenarios PASSED [ 56%] +student-tests/test_sessions.py::test_short_ttl_expires[valkey] PASSED [ 56%] +student-tests/test_sessions.py::test_short_ttl_expires[postgres] PASSED [ 57%] +student-tests/test_sessions.py::test_restart_does_not_extend_ttl PASSED [ 57%] +student-tests/test_sessions.py::test_postgres_session_store PASSED [ 57%] +student-tests/test_storage_failures.py::test_s3_outage_is_503_and_text_keeps_working[stop] PASSED [ 58%] +student-tests/test_storage_failures.py::test_s3_outage_is_503_and_text_keeps_working[pause] PASSED [ 58%] +student-tests/test_storage_failures.py::test_upload_survives_failover_of_the_serving_replica PASSED [ 58%] +student-tests/test_storage_failures.py::test_bucket_is_private_outside_the_api PASSED [ 58%] +student-tests/test_storage_failures.py::test_s3_restart_keeps_objects PASSED [ 59%] +student-tests/unit/test_images.py::test_public_fixture_is_accepted PASSED [ 59%] +student-tests/unit/test_images.py::test_valid_png_kinds[rgba] PASSED [ 59%] +student-tests/unit/test_images.py::test_valid_png_kinds[palette] PASSED [ 59%] +student-tests/unit/test_images.py::test_valid_png_kinds[1bit] PASSED [ 60%] +student-tests/unit/test_images.py::test_valid_png_kinds[gray] PASSED [ 60%] +student-tests/unit/test_images.py::test_valid_png_kinds[noise] PASSED [ 60%] +student-tests/unit/test_images.py::test_valid_png_kinds[1x1] PASSED [ 60%] +student-tests/unit/test_images.py::test_interlaced_png_is_accepted PASSED [ 61%] +student-tests/unit/test_images.py::test_broken_png_is_415[first 33 bytes of the fixture] PASSED [ 61%] +student-tests/unit/test_images.py::test_broken_png_is_415[truncated (no IEND)] PASSED [ 61%] +student-tests/unit/test_images.py::test_broken_png_is_415[truncated in IDAT] PASSED [ 61%] +student-tests/unit/test_images.py::test_broken_png_is_415[flipped byte -> CRC error] PASSED [ 62%] +student-tests/unit/test_images.py::test_broken_png_is_415[garbage IDAT with valid CRC] PASSED [ 62%] +student-tests/unit/test_images.py::test_broken_png_is_415[fewer rows than IHDR] PASSED [ 62%] +student-tests/unit/test_images.py::test_broken_png_is_415[bytes after IEND (polyglot)] PASSED [ 62%] +student-tests/unit/test_images.py::test_broken_png_is_415[text] PASSED [ 63%] +student-tests/unit/test_images.py::test_broken_png_is_415[gif] PASSED [ 63%] +student-tests/unit/test_images.py::test_broken_png_is_415[two bytes of a signature] PASSED [ 63%] +student-tests/unit/test_images.py::test_pixel_limit_is_checked_from_the_header PASSED [ 64%] +student-tests/unit/test_images.py::test_png_verification_is_streaming_and_fast PASSED [ 64%] +student-tests/unit/test_images.py::test_jpeg_header_is_parsed_incrementally PASSED [ 64%] +student-tests/unit/test_images.py::test_valid_jpeg[base] SKIPPED (dj...) [ 64%] +student-tests/unit/test_images.py::test_valid_jpeg[prog] SKIPPED (dj...) [ 65%] +student-tests/unit/test_images.py::test_broken_jpeg_is_415[no scan data after SOS-] SKIPPED [ 65%] +student-tests/unit/test_images.py::test_broken_jpeg_is_415[truncated-] SKIPPED [ 65%] +student-tests/unit/test_images.py::test_good_filenames[photo.png] PASSED [ 65%] +student-tests/unit/test_images.py::test_good_filenames[\u0444\u043e\u0442\u043e \u043e\u0442\u043f\u0443\u0441\u043a\u0430.jpg] PASSED [ 66%] +student-tests/unit/test_images.py::test_good_filenames[aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 66%] +student-tests/unit/test_images.py::test_good_filenames[emoji \U0001f468\u200d\U0001f469\u200d\U0001f467.png] PASSED [ 66%] +student-tests/unit/test_images.py::test_good_filenames[ x ] PASSED [ 66%] +student-tests/unit/test_images.py::test_good_filenames[..x] PASSED [ 67%] +student-tests/unit/test_images.py::test_bad_filenames[] PASSED [ 67%] +student-tests/unit/test_images.py::test_bad_filenames[ ] PASSED [ 67%] +student-tests/unit/test_images.py::test_bad_filenames[\u3000] PASSED [ 67%] +student-tests/unit/test_images.py::test_bad_filenames[.] PASSED [ 68%] +student-tests/unit/test_images.py::test_bad_filenames[..] PASSED [ 68%] +student-tests/unit/test_images.py::test_bad_filenames[a/b.png] PASSED [ 68%] +student-tests/unit/test_images.py::test_bad_filenames[a\\b.png] PASSED [ 69%] +student-tests/unit/test_images.py::test_bad_filenames[a\x00b] PASSED [ 69%] +student-tests/unit/test_images.py::test_bad_filenames[c1\x85.png] PASSED [ 69%] +student-tests/unit/test_images.py::test_bad_filenames[tab\t.png] PASSED [ 69%] +student-tests/unit/test_images.py::test_bad_filenames[del\x7f.png] PASSED [ 70%] +student-tests/unit/test_images.py::test_bad_filenames[x\u2028y] PASSED [ 70%] +student-tests/unit/test_images.py::test_bad_filenames[photo\u202egnp.exe] PASSED [ 70%] +student-tests/unit/test_images.py::test_bad_filenames[aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 70%] +student-tests/unit/test_images.py::test_filename_must_be_single_and_present PASSED [ 71%] +student-tests/unit/test_images.py::test_query_must_be_valid_utf8 PASSED [ 71%] +student-tests/unit/test_pipeline.py::test_fit_rounds_down_without_upscaling[size0-256-expected0] PASSED [ 71%] +student-tests/unit/test_pipeline.py::test_fit_rounds_down_without_upscaling[size1-256-expected1] PASSED [ 71%] +student-tests/unit/test_pipeline.py::test_fit_rounds_down_without_upscaling[size2-256-expected2] PASSED [ 72%] +student-tests/unit/test_pipeline.py::test_fit_rounds_down_without_upscaling[size3-256-expected3] PASSED [ 72%] +student-tests/unit/test_pipeline.py::test_fit_rounds_down_without_upscaling[size4-256-expected4] PASSED [ 72%] +student-tests/unit/test_pipeline.py::test_fit_rounds_down_without_upscaling[size5-256-expected5] PASSED [ 72%] +student-tests/unit/test_pipeline.py::test_fit_rounds_down_without_upscaling[size6-256-expected6] PASSED [ 73%] +student-tests/unit/test_pipeline.py::test_fit_rounds_down_without_upscaling[size7-1280-expected7] PASSED [ 73%] +student-tests/unit/test_pipeline.py::test_center_square PASSED [ 73%] +student-tests/unit/test_pipeline.py::test_public_fixture_variants_match_the_contract PASSED [ 74%] +student-tests/unit/test_pipeline.py::test_small_image_is_not_upscaled_but_crop_is PASSED [ 74%] +student-tests/unit/test_pipeline.py::test_big_image_is_bounded_by_1280 PASSED [ 74%] +student-tests/unit/test_pipeline.py::test_exif_orientation_is_applied_and_dropped PASSED [ 74%] +student-tests/unit/test_pipeline.py::test_alpha_is_composited_on_white_for_jpeg_and_kept_for_webp PASSED [ 75%] +student-tests/unit/test_pipeline.py::test_metadata_is_not_copied PASSED [ 75%] +student-tests/unit/test_pipeline.py::test_png_modes_are_processed[RGBA] PASSED [ 75%] +student-tests/unit/test_pipeline.py::test_png_modes_are_processed[P] PASSED [ 75%] +student-tests/unit/test_pipeline.py::test_png_modes_are_processed[1] PASSED [ 76%] +student-tests/unit/test_pipeline.py::test_png_modes_are_processed[L] PASSED [ 76%] +student-tests/unit/test_pipeline.py::test_png_modes_are_processed[LA] PASSED [ 76%] +student-tests/unit/test_pipeline.py::test_png_modes_are_processed[RGB] PASSED [ 76%] +student-tests/unit/test_pipeline.py::test_16_bit_grey_is_scaled_not_clipped PASSED [ 77%] +student-tests/unit/test_pipeline.py::test_pipeline_versions_differ_in_bytes_not_in_geometry PASSED [ 77%] +student-tests/unit/test_pipeline.py::test_pixel_limit_is_checked_before_decoding PASSED [ 77%] +student-tests/unit/test_pipeline.py::test_child_process_protocol_round_trip PASSED [ 77%] +student-tests/unit/test_pipeline.py::test_encoded_message_matches_jobs_schema_and_carries_ids_only PASSED [ 78%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change0] PASSED [ 78%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change1] PASSED [ 78%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change2] PASSED [ 79%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change3] PASSED [ 79%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change4] PASSED [ 79%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change5] PASSED [ 79%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change6] PASSED [ 80%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change7] PASSED [ 80%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change8] PASSED [ 80%] +student-tests/unit/test_pipeline.py::test_invalid_messages_are_rejected[change9] PASSED [ 80%] +student-tests/unit/test_pipeline.py::test_garbage_is_rejected PASSED [ 81%] +student-tests/unit/test_pipeline.py::test_backoff_is_full_jitter_and_capped PASSED [ 81%] +student-tests/unit/test_pipeline.py::test_pipeline_settings PASSED [ 81%] +student-tests/unit/test_unit.py::test_valid_passwords[Twelve-chars] PASSED [ 81%] +student-tests/unit/test_unit.py::test_valid_passwords[\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f] PASSED [ 82%] +student-tests/unit/test_unit.py::test_valid_passwords[\u043f\u0430\u0440\u043e\u043b\u044c:\u0441:\u0434\u0432\u043e\u0435\u0442\u043e\u0447\u0438\u044f\u043c\u0438] PASSED [ 82%] +student-tests/unit/test_unit.py::test_valid_passwords[\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600] PASSED [ 82%] +student-tests/unit/test_unit.py::test_valid_passwords[a b c d e f g] PASSED [ 82%] +student-tests/unit/test_unit.py::test_invalid_passwords[short-11ch] PASSED [ 83%] +student-tests/unit/test_unit.py::test_invalid_passwords[\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f] PASSED [ 83%] +student-tests/unit/test_unit.py::test_invalid_passwords[ ] PASSED [ 83%] +student-tests/unit/test_unit.py::test_invalid_passwords[\t\n\u3000\t\n\u3000\t\n\u3000\t\n\u3000\t\n\u3000] PASSED [ 83%] +student-tests/unit/test_unit.py::test_invalid_passwords[\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800] PASSED [ 84%] +student-tests/unit/test_unit.py::test_invalid_passwords[None] PASSED [ 84%] +student-tests/unit/test_unit.py::test_invalid_passwords[123456789012] PASSED [ 84%] +student-tests/unit/test_unit.py::test_text_may_contain_nul_but_not_lone_surrogates PASSED [ 85%] +student-tests/unit/test_unit.py::test_cursor_roundtrip_and_binding PASSED [ 85%] +student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|253402300800000000|11111111-1111-4111-8111-111111111111] PASSED [ 85%] +student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|9999999999999999999|11111111-1111-4111-8111-111111111111] PASSED [ 85%] +student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|-1|11111111-1111-4111-8111-111111111111] PASSED [ 86%] +student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|1|not-a-uuid] PASSED [ 86%] +student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|1|11111111-1111-4111-8111-11111111111G] PASSED [ 86%] +student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v2|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|1|11111111-1111-4111-8111-111111111111] PASSED [ 86%] +student-tests/unit/test_unit.py::test_time_helpers_are_exact PASSED [ 87%] +student-tests/unit/test_unit.py::test_basic_is_parsed_and_password_may_contain_colons PASSED [ 87%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[None] PASSED [ 87%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[] PASSED [ 87%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic] PASSED [ 88%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic \xe9] PASSED [ 88%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic !!!!] PASSED [ 88%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic bm8tY29sb24=] PASSED [ 88%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic //46cHc=] PASSED [ 89%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic OnBhc3N3b3Jk] PASSED [ 89%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic YWxpY2U6] PASSED [ 89%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic 0JDQu9C40YHQsDpwdw==] PASSED [ 90%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Bearer aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 90%] +student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA] PASSED [ 90%] +student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[None] PASSED [ 90%] +student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[] PASSED [ 91%] +student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Bearer] PASSED [ 91%] +student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Bearer short] PASSED [ 91%] +student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Bearer aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!] PASSED [ 91%] +student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Basic YWxpY2U6cGFzc3dvcmQtMTIz] PASSED [ 92%] +student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Token aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 92%] +student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Bearer \xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9] PASSED [ 92%] +student-tests/unit/test_unit.py::test_bearer_ok PASSED [ 92%] +student-tests/unit/test_unit.py::test_argon2id_hash_and_verify PASSED [ 93%] +student-tests/unit/test_unit.py::test_s3_settings_are_validated_and_contract_limits_cannot_be_raised PASSED [ 93%] +student-tests/unit/test_unit.py::test_settings_read_secret_files_and_contract_names PASSED [ 93%] +student-tests/unit/test_unit.py::test_valid_idempotency_keys[a] PASSED [ 93%] +student-tests/unit/test_unit.py::test_valid_idempotency_keys[kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk] PASSED [ 94%] +student-tests/unit/test_unit.py::test_valid_idempotency_keys[A-Za-z0-9._:-] PASSED [ 94%] +student-tests/unit/test_unit.py::test_valid_idempotency_keys[2026-10-16T10:00:00.1] PASSED [ 94%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[None] PASSED [ 95%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[] PASSED [ 95%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk] PASSED [ 95%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[\u043a\u043b\u044e\u0447] PASSED [ 95%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a b] PASSED [ 96%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a/b] PASSED [ 96%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a,b] PASSED [ 96%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a\tb] PASSED [ 96%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a\x00b] PASSED [ 97%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[*] PASSED [ 97%] +student-tests/unit/test_unit.py::test_invalid_idempotency_keys[\xe9] PASSED [ 97%] +student-tests/unit/test_unit.py::test_duplicate_keys_and_garbage_json_are_rejected[{"text":"a","text":"b"}] PASSED [ 97%] +student-tests/unit/test_unit.py::test_duplicate_keys_and_garbage_json_are_rejected[{"a":{"x":1,"x":2}}] PASSED [ 98%] +student-tests/unit/test_unit.py::test_duplicate_keys_and_garbage_json_are_rejected[\xff] PASSED [ 98%] +student-tests/unit/test_unit.py::test_duplicate_keys_and_garbage_json_are_rejected[{"text":"x"} {}] PASSED [ 98%] +student-tests/unit/test_unit.py::test_attachment_ids_absent_equals_empty_and_is_validated PASSED [ 98%] +student-tests/unit/test_unit.py::test_request_fingerprint_ignores_key_order_and_default_attachments PASSED [ 99%] +student-tests/unit/test_unit.py::test_instance_id_defaults_to_hostname_and_is_validated PASSED [ 99%] +student-tests/unit/test_unit.py::test_idempotency_ttl_cannot_be_below_24_hours PASSED [ 99%] +student-tests/unit/test_unit.py::test_message_write_path_setting PASSED [100%] + +================= 377 passed, 4 skipped in 1515.54s (0:25:15) ================= +[exit 0, 1516.9 s] diff --git a/evidence/test-failover.txt b/evidence/test-failover.txt new file mode 100644 index 0000000..e68a67f --- /dev/null +++ b/evidence/test-failover.txt @@ -0,0 +1,74 @@ +# make test-failover with scripts/stop-one: which container is really stopped, restore, repeat +# 2026-10-31T14:21:25Z, commit cf6e929 docs: PIPELINE, DEFENSE, DATA_MODEL, WINDOWS для lab5; черновик REPORT + +--- run 1; running API replicas BEFORE: +mtusi-lab5-api-1 host=d08185c0e3f3 running exit=0 pid=91728 started=2026-10-31T14:21:03.235 +mtusi-lab5-api-2 host=85c7cde066aa running exit=0 pid=91671 started=2026-10-31T14:21:02.858 + +$ make test-failover ACTION_SCRIPT=scripts/stop-one.cmd +python tests/resilience.py --mode failover --action-script "scripts/stop-one.cmd" --base-url "http://localhost:8080" +Executing explicit failover hook; hook output suppressed to avoid leaking secrets. +{"observed_instances_before": 2, "successful_writes": 126, "failed_attempts": 0, "first_recovery_seconds": 6.326} +PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence. +[exit 0, 19.4 s] + +AFTER (the stopped replica, left stopped by the hook): +mtusi-lab5-api-1 host=d08185c0e3f3 exited exit=0 pid=0 started=2026-10-31T14:21:03.235 + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py status +api2 172.28.0.2 UP mtusi-lab5-api-2 X-Instance-Id=85c7cde066aa +(other slots: MAINT, no container) +[exit 0, 0.9 s] + +$ sh scripts/restore-api.sh +mtusi-lab5-api-1 Up 6 seconds (healthy) +mtusi-lab5-api-2 Up 50 seconds (healthy) + Container mtusi-lab5-api-2 Running + Container mtusi-lab5-api-1 Starting + Container mtusi-lab5-api-1 Started + Container mtusi-lab5-api-2 Waiting + Container mtusi-lab5-api-1 Waiting + Container mtusi-lab5-api-2 Healthy + Container mtusi-lab5-api-1 Healthy +[exit 0, 6.9 s] + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py wait-up 2 +2 API replica(s) UP in HAProxy after 0.6 s: ['85c7cde066aa', 'd08185c0e3f3'] +[exit 0, 1.4 s] + +--- run 2; running API replicas BEFORE: +mtusi-lab5-api-1 host=d08185c0e3f3 running exit=0 pid=94963 started=2026-10-31T14:21:47.426 +mtusi-lab5-api-2 host=85c7cde066aa running exit=0 pid=91671 started=2026-10-31T14:21:02.858 + +$ make test-failover ACTION_SCRIPT=scripts/stop-one.cmd +python tests/resilience.py --mode failover --action-script "scripts/stop-one.cmd" --base-url "http://localhost:8080" +Executing explicit failover hook; hook output suppressed to avoid leaking secrets. +{"observed_instances_before": 2, "successful_writes": 129, "failed_attempts": 0, "first_recovery_seconds": 6.074} +PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence. +[exit 0, 18.0 s] + +AFTER (the stopped replica, left stopped by the hook): +mtusi-lab5-api-2 host=85c7cde066aa exited exit=0 pid=0 started=2026-10-31T14:21:02.858 + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py status +api1 172.28.0.3 UP mtusi-lab5-api-1 X-Instance-Id=d08185c0e3f3 +(other slots: MAINT, no container) +[exit 0, 0.7 s] + +$ sh scripts/restore-api.sh +mtusi-lab5-api-1 Up 34 seconds (healthy) +mtusi-lab5-api-2 Up 6 seconds (healthy) + Container mtusi-lab5-api-1 Running + Container mtusi-lab5-api-2 Starting + Container mtusi-lab5-api-2 Started + Container mtusi-lab5-api-2 Waiting + Container mtusi-lab5-api-1 Waiting + Container mtusi-lab5-api-2 Healthy + Container mtusi-lab5-api-1 Healthy +[exit 0, 7.4 s] + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py wait-up 2 +2 API replica(s) UP in HAProxy after 0.6 s: ['85c7cde066aa', 'd08185c0e3f3'] +[exit 0, 1.6 s] + +Replicas stopped by the public test across runs: ['mtusi-lab5-api-1', 'mtusi-lab5-api-2'] diff --git a/evidence/test-persistence.txt b/evidence/test-persistence.txt new file mode 100644 index 0000000..b16ba28 --- /dev/null +++ b/evidence/test-persistence.txt @@ -0,0 +1,38 @@ +# make test-persistence: restart of LB, API replicas, relay, worker, PostgreSQL, Valkey, S3, RabbitMQ +# 2026-10-31T14:20:06Z, commit cf6e929 docs: PIPELINE, DEFENSE, DATA_MODEL, WINDOWS для lab5; черновик REPORT + +Containers BEFORE: +mtusi-lab5-api-1 host=d08185c0e3f3 running exit=0 pid=83074 started=2026-10-31T14:18:45.487 +mtusi-lab5-api-2 host=85c7cde066aa running exit=0 pid=82998 started=2026-10-31T14:18:45.132 +mtusi-lab5-broker-1 host=broker running exit=0 pid=82136 started=2026-10-31T14:18:30.831 +mtusi-lab5-lb-1 host=a22e1c19fc90 running exit=0 pid=83486 started=2026-10-31T14:18:51.489 +mtusi-lab5-migrate-1 host=fa0a8a1c9191 exited exit=0 pid=0 started=2026-10-31T14:18:36.261 +mtusi-lab5-postgres-1 host=7961919f90c9 running exit=0 pid=82047 started=2026-10-31T14:18:30.501 +mtusi-lab5-relay-1 host=9b6249446ece running exit=0 pid=82799 started=2026-10-31T14:18:41.997 +mtusi-lab5-s3-1 host=bdee3ea030a4 running exit=0 pid=81901 started=2026-10-31T14:18:29.927 +mtusi-lab5-s3-init-1 host=2c85c6921c88 exited exit=0 pid=0 started=2026-10-31T14:18:38.517 +mtusi-lab5-valkey-1 host=d53b8a88e5b5 running exit=0 pid=81977 started=2026-10-31T14:18:30.236 +mtusi-lab5-worker-1 host=ee8c11f59aa8 running exit=0 pid=82939 started=2026-10-31T14:18:44.768 + +$ make test-persistence ACTION_SCRIPT=scripts/restart.cmd +python tests/resilience.py --mode persistence --action-script "scripts/restart.cmd" --base-url "http://localhost:8080" +Executing explicit persistence hook; hook output suppressed to avoid leaking secrets. +PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence. +[exit 0, 76.0 s] + +Containers AFTER (new pid/started for lb, api-*, relay, worker, postgres, valkey, s3, broker; volumes kept): +mtusi-lab5-api-1 host=d08185c0e3f3 running exit=0 pid=91728 started=2026-10-31T14:21:03.235 +mtusi-lab5-api-2 host=85c7cde066aa running exit=0 pid=91671 started=2026-10-31T14:21:02.858 +mtusi-lab5-broker-1 host=broker running exit=0 pid=89550 started=2026-10-31T14:20:17.480 +mtusi-lab5-lb-1 host=a22e1c19fc90 running exit=0 pid=92390 started=2026-10-31T14:21:12.759 +mtusi-lab5-migrate-1 host=fa0a8a1c9191 exited exit=0 pid=0 started=2026-10-31T14:21:14.217 +mtusi-lab5-postgres-1 host=7961919f90c9 running exit=0 pid=89433 started=2026-10-31T14:20:16.605 +mtusi-lab5-relay-1 host=9b6249446ece running exit=0 pid=92854 started=2026-10-31T14:21:16.976 +mtusi-lab5-s3-1 host=bdee3ea030a4 running exit=0 pid=90402 started=2026-10-31T14:20:36.421 +mtusi-lab5-s3-init-1 host=2c85c6921c88 exited exit=0 pid=0 started=2026-10-31T14:21:17.299 +mtusi-lab5-valkey-1 host=d53b8a88e5b5 running exit=0 pid=89376 started=2026-10-31T14:20:16.424 +mtusi-lab5-worker-1 host=ee8c11f59aa8 running exit=0 pid=93271 started=2026-10-31T14:21:22.639 + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py wait-up 2 +2 API replica(s) UP in HAProxy after 0.6 s: ['85c7cde066aa', 'd08185c0e3f3'] +[exit 0, 1.5 s] diff --git a/evidence/track2-runs.txt b/evidence/track2-runs.txt new file mode 100644 index 0000000..380a605 --- /dev/null +++ b/evidence/track2-runs.txt @@ -0,0 +1,112 @@ +# Track 2: bench/pipeline_faults.py (duplicate, reorder, crashes, stale versions, hung worker) + read-only audit of the invariants +# 2026-10-31T14:48:35Z, commit cf6e929 docs: PIPELINE, DEFENSE, DATA_MODEL, WINDOWS для lab5; черновик REPORT + +$ C:\Users\wfa22\Desktop\Высоконагруженные приложения\lab5-clean\.venv\Scripts\python.exe bench/pipeline_faults.py --output evidence/track2 --uploads 5 +[2026-10-31T14:48:37Z] baseline_no_faults +[2026-10-31T14:48:54Z] duplicate_publish +[2026-10-31T14:49:25Z] duplicate_and_reverse_order +[2026-10-31T14:49:56Z] relay_crash_after_confirm_before_mark +[2026-10-31T14:50:27Z] worker_crash_before_side_effect +[2026-10-31T14:51:19Z] worker_crash_after_s3_before_commit +[2026-10-31T14:52:05Z] worker_crash_after_commit_before_ack +[2026-10-31T14:53:10Z] transient_errors_then_success +[2026-10-31T14:53:42Z] two_workers_duplicate_publish +[2026-10-31T14:54:22Z] versions_scenario +[2026-10-31T14:55:08Z] queued_reorder_scenario +[2026-10-31T14:56:08Z] hung_worker_scenario +bcd68c4d813f0d14a490156098130cab5e63ae21352e9fc620b8ac85f48df995 +bcd68c4d813f0d14a490156098130cab5e63ae21352e9fc620b8ac85f48df995 +scenario uploads ready claims problems +baseline_no_faults 5 5 5 0 +duplicate_publish 5 5 5 0 +duplicate_and_reverse_order 5 5 5 0 +relay_crash_after_confirm_before_mark 5 5 5 0 +worker_crash_before_side_effect 5 5 14 0 +worker_crash_after_s3_before_commit 5 5 10 0 +worker_crash_after_commit_before_ack 5 5 5 0 +transient_errors_then_success 5 5 15 0 +two_workers_duplicate_publish 5 5 5 0 +stale_version_and_reorder 1 - - 0 +reordered_versions_in_queue 1 - 2 0 +hung_worker_docker_pause 1 - 2 0 +audit after all scenarios: ok=True violations=0 objects={'final': 523, 'staging': 2, 'published': 522, 'checksums_verified': True} +[exit 0, 515.1 s] + +$ docker compose run --rm auditor --verify-checksums +{ + "ok": true, + "violations": [], + "notes": [], + "attachments": { + "failed": 4, + "pending": 2, + "ready": 129 + }, + "jobs": { + "superseded": 2, + "succeeded": 131, + "failed": 6 + }, + "objects": { + "final": 523, + "staging": 2, + "published": 522, + "checksums_verified": true + } +} + Container mtusi-lab5-auditor-run-e2fdeabd6d0f Creating + Container mtusi-lab5-auditor-run-e2fdeabd6d0f Created +[exit 0, 7.2 s] + +$ docker compose run --rm jobctl stats +{ + "jobs": { + "failed": 6, + "succeeded": 131, + "superseded": 2 + }, + "outbox_unpublished": 0, + "outbox_due": 0 +} + Container mtusi-lab5-jobctl-run-6f68d2c5bb69 Creating + Container mtusi-lab5-jobctl-run-6f68d2c5bb69 Created +[exit 0, 5.0 s] + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py wait-up 2 +2 API replica(s) UP in HAProxy after 0.7 s: ['1fb1ea74a42b', '492b7b24bd1f'] +[exit 0, 1.7 s] + +$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe tests/load.py --duration 30 --concurrency 4 --output evidence/load.json +{ + "lab": 5, + "workload": "closed-loop POST messages; no automatic retries", + "requested_duration_seconds": 30.0, + "elapsed_seconds": 30.066, + "concurrency": 4, + "successful_requests": 5515, + "failed_requests": 0, + "error_ratio": 0.0, + "successful_rps": 183.429, + "successful_latency_ms": { + "p50": 17.96, + "p95": 43.491, + "p99": 58.653 + }, + "note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead." +} +[exit 0, 31.5 s] + +$ docker compose ps -a --format table {{.Service}} {{.Status}} +SERVICE STATUS +api Up 10 minutes (healthy) +api Up 11 minutes (healthy) +broker Up 23 minutes (healthy) +lb Up 30 minutes (healthy) +migrate Exited (0) About a minute ago +postgres Up 11 minutes (healthy) +relay Up 2 minutes (healthy) +s3 Up 9 minutes (healthy) +s3-init Exited (0) About a minute ago +valkey Up 13 minutes (healthy) +worker Up About a minute (healthy) +[exit 0, 0.4 s] diff --git a/evidence/track2/summary.json b/evidence/track2/summary.json new file mode 100644 index 0000000..9e3d1cb --- /dev/null +++ b/evidence/track2/summary.json @@ -0,0 +1,277 @@ +{ + "experiment": "track 2: idempotency of the versioned pipeline under injected faults", + "started": "2026-10-31T14:48:37Z", + "finished": "2026-10-31T14:57:10Z", + "host": { + "os": "Windows-10-10.0.19045-SP0", + "python": "3.14.6" + }, + "uploads_per_plain_scenario": 5, + "scenarios": [ + { + "scenario": "baseline_no_faults", + "faults": { + "worker": {} + }, + "uploads": 5, + "ready": 5, + "claims_total": 5, + "max_claims": 1, + "time_to_ready_s": { + "p50": 2.21, + "max": 3.15 + }, + "event_sequences": [ + "created > claimed > succeeded" + ], + "problems": {} + }, + { + "scenario": "duplicate_publish", + "faults": { + "relay": { + "RELAY_FAULT_POINTS": "duplicate_publish" + } + }, + "uploads": 5, + "ready": 5, + "claims_total": 5, + "max_claims": 1, + "time_to_ready_s": { + "p50": 1.89, + "max": 2.52 + }, + "event_sequences": [ + "created > claimed > succeeded" + ], + "problems": {} + }, + { + "scenario": "duplicate_and_reverse_order", + "faults": { + "relay": { + "RELAY_FAULT_POINTS": "duplicate_publish,reverse_order" + } + }, + "uploads": 5, + "ready": 5, + "claims_total": 5, + "max_claims": 1, + "time_to_ready_s": { + "p50": 1.89, + "max": 2.52 + }, + "event_sequences": [ + "created > claimed > succeeded" + ], + "problems": {} + }, + { + "scenario": "relay_crash_after_confirm_before_mark", + "faults": { + "relay": { + "RELAY_FAULT_POINTS": "crash_after_publish" + } + }, + "uploads": 5, + "ready": 5, + "claims_total": 5, + "max_claims": 1, + "time_to_ready_s": { + "p50": 1.89, + "max": 2.83 + }, + "event_sequences": [ + "created > claimed > fault_injected > succeeded", + "created > claimed > succeeded" + ], + "problems": {} + }, + { + "scenario": "worker_crash_before_side_effect", + "faults": { + "worker": { + "WORKER_FAULT_POINTS": "crash_after_claim", + "JOB_LEASE_SECONDS": "6", + "ORPHAN_GRACE_SECONDS": "3", + "CLEANUP_INTERVAL_SECONDS": "1" + } + }, + "uploads": 5, + "ready": 5, + "claims_total": 14, + "max_claims": 3, + "time_to_ready_s": { + "p50": 21.66, + "max": 21.68 + }, + "event_sequences": [ + "created > claimed > fault_injected > lease_expired > claimed > lease_expired > claimed > succeeded", + "created > claimed > fault_injected > lease_expired > claimed > succeeded", + "created > claimed > lease_expired > claimed > fault_injected > lease_expired > claimed > succeeded" + ], + "problems": {} + }, + { + "scenario": "worker_crash_after_s3_before_commit", + "faults": { + "worker": { + "WORKER_FAULT_POINTS": "crash_after_s3_write", + "JOB_LEASE_SECONDS": "6", + "ORPHAN_GRACE_SECONDS": "3", + "CLEANUP_INTERVAL_SECONDS": "1" + } + }, + "uploads": 5, + "ready": 5, + "claims_total": 10, + "max_claims": 2, + "time_to_ready_s": { + "p50": 15.49, + "max": 16.13 + }, + "event_sequences": [ + "created > claimed > fault_injected > lease_expired > claimed > succeeded" + ], + "problems": {} + }, + { + "scenario": "worker_crash_after_commit_before_ack", + "faults": { + "worker": { + "WORKER_FAULT_POINTS": "crash_after_commit", + "JOB_LEASE_SECONDS": "6", + "ORPHAN_GRACE_SECONDS": "3", + "CLEANUP_INTERVAL_SECONDS": "1" + } + }, + "uploads": 5, + "ready": 5, + "claims_total": 5, + "max_claims": 1, + "time_to_ready_s": { + "p50": 5.3, + "max": 8.4 + }, + "event_sequences": [ + "created > claimed > succeeded > fault_injected" + ], + "problems": {} + }, + { + "scenario": "transient_errors_then_success", + "faults": { + "worker": { + "WORKER_FAULT_POINTS": "transient_error=2", + "JOB_RETRY_BASE_SECONDS": "0.5", + "JOB_LEASE_SECONDS": "6", + "ORPHAN_GRACE_SECONDS": "3", + "CLEANUP_INTERVAL_SECONDS": "1" + } + }, + "uploads": 5, + "ready": 5, + "claims_total": 15, + "max_claims": 3, + "time_to_ready_s": { + "p50": 4.11, + "max": 4.13 + }, + "event_sequences": [ + "created > claimed > retry_scheduled > claimed > retry_scheduled > claimed > succeeded" + ], + "problems": {} + }, + { + "scenario": "two_workers_duplicate_publish", + "faults": { + "worker": {} + }, + "uploads": 5, + "ready": 5, + "claims_total": 5, + "max_claims": 1, + "time_to_ready_s": { + "p50": 1.58, + "max": 2.21 + }, + "event_sequences": [ + "created > claimed > succeeded" + ], + "problems": {} + }, + { + "scenario": "stale_version_and_reorder", + "uploads": 1, + "statuses_seen_during_reprocess": [ + "ready" + ], + "job_states": [ + "succeeded:v1", + "superseded:v2", + "succeeded:v3" + ], + "published_version": 3, + "variant_rows_by_version": 6, + "objects": 10, + "events": [ + "created", + "claimed", + "succeeded", + "created", + "claimed", + "created", + "claimed", + "succeeded", + "superseded" + ], + "problems": [] + }, + { + "scenario": "reordered_versions_in_queue", + "job_states": [ + "succeeded:v1", + "superseded:v2", + "succeeded:v3" + ], + "claims": 2, + "published_version": 3, + "events": [ + "created", + "claimed", + "succeeded", + "created", + "created", + "claimed", + "succeeded", + "superseded" + ], + "problems": [] + }, + { + "scenario": "hung_worker_docker_pause", + "frozen": "bcd68c4d813f", + "claims": 2, + "events": [ + "created", + "claimed", + "lease_expired", + "claimed", + "succeeded" + ], + "problems": [] + } + ], + "audit_after_all": { + "ok": true, + "violations": [], + "notes": 0, + "objects": { + "final": 523, + "staging": 2, + "published": 522, + "checksums_verified": true + } + }, + "violations": 0 +} \ No newline at end of file diff --git a/evidence/track2/summary.txt b/evidence/track2/summary.txt new file mode 100644 index 0000000..fc82a00 --- /dev/null +++ b/evidence/track2/summary.txt @@ -0,0 +1,14 @@ +scenario uploads ready claims problems +baseline_no_faults 5 5 5 0 +duplicate_publish 5 5 5 0 +duplicate_and_reverse_order 5 5 5 0 +relay_crash_after_confirm_before_mark 5 5 5 0 +worker_crash_before_side_effect 5 5 14 0 +worker_crash_after_s3_before_commit 5 5 10 0 +worker_crash_after_commit_before_ack 5 5 5 0 +transient_errors_then_success 5 5 15 0 +two_workers_duplicate_publish 5 5 5 0 +stale_version_and_reorder 1 - - 0 +reordered_versions_in_queue 1 - 2 0 +hung_worker_docker_pause 1 - 2 0 +audit after all scenarios: ok=True violations=0 objects={'final': 523, 'staging': 2, 'published': 522, 'checksums_verified': True}