docs: REPORT lab3 — доказательства со стенда из чистого clone, уровень 4, трек 1

Второй прогон scripts/collect_evidence.py на новом clone (коммит 2708732):
make check/validate/test, test-persistence (новые PID у lb, api x2,
postgres, valkey), test-failover для обеих реплик по очереди (0 неудачных
попыток), 201 свой тест, серия отказов stop/kill/pause x 2 реплики,
rolling update под нагрузкой (0 ошибок), tests/load.py. Таблица
критерий -> команда -> результат -> файл, раздел уровня 4 с периодами
проверок и временем обнаружения.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Alexey Bodanyuk
2026-10-19 18:28:25 +03:00
co-authored by Claude Opus 5.5
parent 27087326d0
commit c758e49cbb
19 changed files with 1678 additions and 3 deletions
+47 -3
View File
@@ -56,7 +56,7 @@ migrate (одноразовый) ──data──> postgres: миграции 00
- **Graceful shutdown.** SIGTERM → readiness 503 сразу → `DRAIN_SECONDS=3` реплика ещё обслуживает (LB успевает пометить её DOWN) → uvicorn закрывает сокет, запросы в полёте ≤ `SHUTDOWN_TIMEOUT_SECONDS=8` → exit 0. 3 + 8 < `stop_grace_period: 15s`, SIGKILL не нужен.
- **`X-Instance-Id`** = hostname контейнера (короткий container ID): стабилен для процесса, однозначно сопоставляется контейнеру (`docker inspect`), что использует `scripts/stop-one.sh`. Middleware ставит его и `X-Request-Id` на **каждый** ответ API, включая 404/405/413/500/503. Ошибки, созданные самим HAProxy, — без `X-Instance-Id` (их создал не API), но в формате контракта и с тем же `X-Request-Id`.
- **Общая идемпотентность** ([0002](migrations/0002_idempotency_keys.sql), [0003](migrations/0003_post_message_function.sql), [repo.py](src/messenger/repo.py), [app.py](src/messenger/app.py)). Таблица `idempotency_keys (user_id, chat_id, key)` → SHA-256 канонического тела → `message_id`. Ключ, часы чата, сообщение и «результат» — **одна транзакция**; отдельного pending-состояния нет, поэтому сбой не оставляет «висящий» ключ, а транзиентная 5xx не фиксируется. Конкурентный повтор ждёт на уникальном индексе завершения первой транзакции и затем видит её результат (READ COMMITTED). Тот же ключ и тело → тот же 201 и Message; другое тело → 409; постороннему — 404 всегда (членство проверяется до ключа); `attachment_ids: []` = отсутствие поля, порядок ключей JSON не важен. Ключи хранятся ≥ 24 ч, старше — очищаются пачками раз в 10 мин под advisory-замком.
- **Миграции** — только сервис `migrate`, до старта реплик; 0002 и 0003 только добавляют объекты и совместимы с работающей предыдущей версией (rolling update: сначала миграция новым образом, затем замена реплик).
- **Миграции** — только сервис `migrate`, до старта реплик; 0002 и 0003 только добавляют объекты и совместимы с работающей предыдущей версией (rolling update: сначала миграция новым образом, затем замена реплик). При `docker compose start api` (например, в `restart.sh`) Compose снова запускает `migrate` как зависимость — он берёт advisory-замок и отвечает `already applied, skipped`; реплики ждут его завершения ([test-persistence.txt](evidence/test-persistence.txt)).
- **Порты только на loopback.** LB опубликован на `127.0.0.1` и `[::1]`. На Windows `localhost` сначала разрешается в `::1`, и отказ соединения по нему стоил ~2 с на **каждый** запрос (измерено: 2,0 с против 0,007 с). Публичные тесты по умолчанию ходят на `http://localhost:8080`, и в первом прогоне из чистого clone `make test-failover` из-за этого успел сделать лишь 8 записей за 15 с. После исправления — 0,013 с.
- **Сети.** `edge` (lb, api) и internal `data` (api, migrate, postgres, valkey): LB не видит хранилища, у хранилищ нет выхода наружу. Secrets — каждому сервису только свои (проверяется тестом по `docker compose config`).
- **Дефект, найденный на lab3-сценарии рецензента.** Если одной реплике отключить сеть `data`, по дедлайну задача psycopg отменялась, psycopg слал cancel-запрос, а libpq `PQcancelStart` синхронно резолвит имя хоста **в потоке event loop** — вся реплика, включая `/health/live`, зависала примерно на минуту и не возвращалась сама. Теперь по дедлайну запрос сразу получает 503, а задача не отменяется и завершается сама; мёртвые соединения рвёт TCP keepalive + `tcp_user_timeout` = 3 с ([db.py](src/messenger/db.py)). Повторная проверка: live отвечает за 0,00 с, ready 503 за 1,0 с, после восстановления сети реплика сама UP за ~2 с (тест `test_lb_stops_routing_…[network]`).
@@ -83,7 +83,30 @@ Readiness-критичные зависимости: PostgreSQL (всё, кро
## Доказательства критериев
⟨EVIDENCE_TABLE⟩
Стенд: Windows 10 22H2, AMD Ryzen 5 3400G (4 ядра / 8 потоков), 30 ГБ RAM, Docker Desktop 29.8.0 (WSL2, 8 vCPU), Python 3.14 на хосте, GNU make 4.4.1. Все проверки уровней 3 и 4 ниже выполнены 2026-10-19 **на стенде из чистого clone** (новая папка, `git clone`, пустые volumes, secrets созданы заново) одной командой `python scripts/collect_evidence.py`, коммит `2708732`. Вывод сохранён целиком.
| Критерий README | Команда / сценарий | Наблюдаемый результат | Файл |
| --- | --- | --- | --- |
| Чистый clone, secrets, миграции с нуля, одна команда запуска | `init_secrets.py`, `compose config --quiet`, `compose up -d --build --wait`, `compose logs migrate`, `git check-ignore` | все 5 сервисов healthy, `migrate` Exited (0): `0001…0003: applied`; образы по digest; `secrets/` и `.env` игнорируются Git | [clean-clone.txt](evidence/clean-clone.txt) |
| Контракт | `make check`, `make validate` | PASS, PASS | [make-check-validate-test.txt](evidence/make-check-validate-test.txt) |
| **3:** публичный smoke | `make test` | `PASS: lab3 public API scenario` | там же |
| **3:** ≥ 2 реплики, одна точка входа, общая БД и сессии | `lbctl.py replicas/status`, `test_one_session_is_served_by_every_real_replica`, `test_write_on_one_replica_is_read_on_the_other` | 2 контейнера `api` UP в HAProxy; один токен — ответы от обоих, `X-Instance-Id` = их hostname; запись на одной сразу читается на другой; logout сразу действует на обеих | [clean-clone.txt](evidence/clean-clone.txt), [student-tests.txt](evidence/student-tests.txt) |
| **3:** остановка любой реплики, ≤ 10 с, подтверждённое на месте | `make test-failover ACTION_SCRIPT=scripts/stop-one.cmd` → `make restore` → снова (по очереди обе) | PASS для `api-1` и для `api-2`: 131 / 132 успешных записи, **0 неудачных попыток**, запись после хука — 5,9 / 5,7 с; остановленный контейнер `exited (0)`, вторая реплика UP | [test-failover.txt](evidence/test-failover.txt) |
| **3:** повтор POST без дубля, в т.ч. конкурентный и через другую реплику; неподтверждённую запись можно повторить | `test_idempotency.py` (21 тест) + публичный failover-тест (повторяет неопределённый POST с тем же ключом) | см. таблицу «ловушек» ниже; публичный тест: дублей нет | [student-tests.txt](evidence/student-tests.txt) |
| **3:** persistence | `make test-persistence ACTION_SCRIPT=scripts/restart.cmd` | PASS; у `lb`, `api-1`, `api-2`, `postgres`, `valkey` новые PID и `StartedAt`, volumes те же, токен тот же | [test-persistence.txt](evidence/test-persistence.txt) |
| **4:** readiness, таймауты, SIGTERM согласованы; LB исключает недоступный backend | `test_lb_stops_routing_…[pg_hba/network × 2]`, `test_graceful_stop_…[×2]`, `test_graceful_shutdown_order_in_logs` | живая, но неготовая реплика выходит из ротации за 3–4 с, live 200 / ready 503, сама возвращается ≤ 15 с; graceful stop под нагрузкой — 0 ошибок; в логах `drain_started` → `shutdown_requested` → `stopped`, exit 0, в логе HAProxy — `is DOWN` для этой реплики | [student-tests.txt](evidence/student-tests.txt) |
| **4:** период проверок и максимальное время обнаружения | [haproxy.cfg](infra/haproxy/haproxy.cfg) + серия отказов | проверка раз в 1 с, `fall 2`/`rise 2`, `timeout check 1s`; максимум ≈ 3,5 с (заморозка); подробно — в следующем разделе | [failover/series.md](evidence/failover/series.md) |
| **4:** поочерёдная остановка каждой реплики: RPS, ошибки, восстановление | `bench/failover_series.py` (stop / kill / pause × обе реплики) | stop: 0 ошибок; kill: 3 оборванных запроса за ≤ 0,25 с; pause: 4 неудачные попытки за ≤ 3,2 с; потерь и дублей нет | [failover/series.md](evidence/failover/series.md), [series.json](evidence/failover/series.json) |
| **4:** общая идемпотентность параллельными запросами | `test_concurrent_same_key_through_different_replicas_creates_exactly_one`, `…_different_bodies_one_wins` | 24 одновременных → 24 × 201, одно тело, ≥ 2 реплики, 1 строка ключа и 1 сообщение (×3 раунда); разные тела → один победитель, остальные 409 | [student-tests.txt](evidence/student-tests.txt) |
| **4:** лимиты CPU/RAM, нет портов в обход LB, secrets по месту, нет `container_name` | `test_compose_policy_limits_ports_secrets_images`, `test_running_containers_have_limits_and_no_bypass_ports` | лимиты у всех 5 сервисов (config и `docker inspect`); порты только у `lb` и только loopback; secrets: lb — нет, migrate — `database_url`, api — `database_url`+`cache_url`; сети edge/data | там же |
| **4:** rolling update под нагрузкой | `bench/rolling_under_load.py` → `scripts/rolling-update.sh` | 57 с, 11 848 запросов, **0 ошибок**; `api-1/2 revision=dev` → `api-3/4 revision=2708732-…`; 7119 подтверждённых сообщений, 0 потерь и дублей | [rolling-update/](evidence/rolling-update/) |
| **4:** миграции однократно и совместимы | `test_migrations_ran_once_by_the_migrate_service`, `test_migrations_rerun_is_a_noop` | применяет только `migrate` (в логах API миграций нет); повтор — `already applied, skipped`; 0002/0003 только добавляют объекты и применяются до замены реплик | [student-tests.txt](evidence/student-tests.txt), [rolling-update/script-output.txt](evidence/rolling-update/script-output.txt) |
| Публичный генератор нагрузки | `tests/load.py --duration 30 --concurrency 4` | 5247 успешных, 0 ошибок, 174 req/s (closed-loop через порт Windows), p50/p95/p99 = 19 / 50 / 65 мс | [load.json](evidence/load.json) |
| Свои тесты | `python -m pytest student-tests -v` | **201 passed** (69 unit + 132 против стенда), 8 мин; все отказы восстанавливаются в `finally` | [student-tests.txt](evidence/student-tests.txt) |
| Оба пути записи эквивалентны | тесты идемпотентности, контракта и БД с `MESSAGE_WRITE_PATH=multi` | 105 passed (режим `function` — весь набор выше) | раздел трека 1 |
| **5:** трек 1 | `bench/track1/series.sh baseline|improved|ceiling`, `profile.py`, `analyze.py` | см. раздел «Трек 1» | [evidence/track1/](evidence/track1/) |
Первый прогон из чистого clone (тот же сценарий, коммит `2846cae`) нашёл проблему: через `localhost` каждое соединение на Windows стоило 2 с (LB был опубликован только на `127.0.0.1`), и `make test-failover` успел сделать 8 записей вместо ~130. Исправлено публикацией и на `[::1]` (коммит `2708732`). Таблица выше — второй, полный прогон.
## Проверки, на которых ловили другие работы
@@ -115,7 +138,28 @@ Readiness-критичные зависимости: PostgreSQL (всё, кро
## Эксперименты и отказы (уровень 4)
⟨LEVEL4⟩
**Поочерёдная остановка каждой реплики** ([bench/failover_series.py](bench/failover_series.py), [series.md](evidence/failover/series.md)). Нагрузка — 4 closed-loop клиента с хоста через LB: POST сообщения (неопределённый результат → повтор тем же ключом) + GET последней страницы, таймаут клиента 3 с. Раунд: 6 с прогрева → остановка → 12 с без реплики → старт → ожидание UP в HAProxy → 6 с. Перед каждым раундом предыдущая реплика уже восстановлена.
| Воздействие | Что ожидаем | Что наблюдали (api-1 / api-2) | Восстановление / потери |
| --- | --- | --- | --- |
| `docker stop` (SIGTERM, drain) | 0 ошибок: реплика сама уходит из ротации до закрытия сокета | 3568 / 3565 запросов во время отказа, **0 ошибок**; цель последний раз ответила через 1,28 / 1,24 с после SIGTERM (drain) | возврат: UP через 3,3 / 1,4 с, снова обслуживает через 3,4 / 3,1 с; 0 потерь, 0 дублей |
| `docker stop -t 0` (SIGKILL) | обрываются только запросы «в полёте» на убитой реплике | 3 неудачных попытки (обрыв соединения) за ≤ 0,25 с; новые запросы сразу идут на живую (`retry-on conn-failure`) | возврат 2,8 / 3,6 с; неопределённые POST повторены тем же ключом, 0 дублей |
| `docker pause` (заморожен процесс — худший случай) | запросы на замороженную реплику ждут до таймаута, пока проверки её не выключат | 4 неудачные попытки, последняя через 3,2 / 3,1 с; ok-RPS во время отказа падает ~вдвое (часть клиентов ждёт) | после `unpause` UP через 2,2 / 1,2 с; 0 потерь, 0 дублей |
| неготовая, но живая реплика (`pg_hba reject` / `network disconnect` сети `data`) | LB исключает её по readiness, бизнес-запросы к ней не идут | DOWN через 3,2–4,1 с, затем 40 из 40 чтений и 20 из 20 регистраций — через другую реплику; live 200 / ready 503, процесс отвечает | сама UP ≤ 15 с после восстановления доступа (тест) |
| rolling update | ≥ 2 реплики в каждый момент | 57 с, 11 848 запросов, 0 ошибок, p95 53 мс | обе реплики на новой ревизии; 0 потерь, 0 дублей |
| restart всего стенда (`restart.sh`) | данные и токен на месте, LB находит новые IP сам | PASS публичного теста; новые PID у всех 5 процессов | LB видит обе реплики без своего рестарта (тест) |
**Период проверок и максимальное время обнаружения.** HAProxy проверяет `GET /health/ready` каждую 1 с, после первой неудачи — каждые 0,5 с (`fastinter`), DOWN после 2 неудач подряд, UP после 2 удач, таймаут проверки 1 с.
| Ситуация | Максимум до DOWN |
| --- | --- |
| реплика отвечает 503 (drain, нет БД) | ≈ 1 + 0,5 + 1 = 2,5 с |
| процесс заморожен (каждая проверка ждёт таймаут) | ≈ 1 + 1 + 0,5 + 1 = 3,5 с |
| процесс убит или порт закрыт | новые запросы получают отказ соединения, HAProxy сразу отправляет их на другую реплику (`retry-on conn-failure` + `redispatch`); проверка лишь фиксирует это |
Измерения выше в эти пределы укладываются: 3,2–4,1 с в тесте включают 0,1–0,5 с опроса через `docker exec`.
**Ограничение измерения.** Нагрузка closed-loop с хоста Windows через проброс порта Docker Desktop (~200–290 ok-RPS). Это проверка доступности и корректности, а не ёмкости — ёмкость меряет трек 1 с k6 внутри сети стенда. Неудачной попыткой считается любой не-2xx ответ или ошибка соединения/таймаут клиента.
## Трек 1. Цена масштабирования
+266
View File
@@ -0,0 +1,266 @@
# Clean clone: secrets, config, build, migrations from zero, start, versions
# 2026-10-19T15:10:17Z, commit 2708732 fix: LB и на [::1] — localhost на Windows стоил 2 с на запрос; сдвиг цели failover
$ git status --short --branch
## solution/bvt2302-bodanyuk...origin/solution/bvt2302-bodanyuk
[exit 0, 0.1 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe --version
Python 3.14.6
[exit 0, 0.0 s]
$ docker version --format client {{.Client.Version}} / server {{.Server.Version}}
client 29.8.0 / server 29.8.0
[exit 0, 0.1 s]
$ docker compose version
Docker Compose version v5.5.1
[exit 0, 0.1 s]
$ make --version
GNU Make 4.4.1
Built for Windows32
Copyright (C) 1988-2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
[exit 0, 0.0 s]
$ python scripts/init_secrets.py
created secrets/postgres_password
created secrets/valkey_password
created secrets/database_url
created secrets/cache_url
[exit 0, 0.1 s]
$ cp .env.example .env
$ docker compose --env-file .env config --quiet
[exit 0, 0.2 s]
$ docker compose --env-file .env up -d --build --wait
#1 [internal] load local bake definitions
#1 reading from stdin 1.36kB 0.1s done
#1 DONE 0.1s
#2 [migrate internal] load build definition from Dockerfile
#2 transferring dockerfile: 2.52kB 0.0s done
#2 DONE 0.0s
#3 [api] resolve image config for docker-image://docker.io/docker/dockerfile:1
#3 DONE 4.0s
#4 [api] docker-image://docker.io/docker/dockerfile:1@sha256:4edf897a3ffa55b89f906fc8cc78afdb3f1834cc9c7083565e611a8a7d5fe99e
#4 resolve docker.io/docker/dockerfile:1@sha256:4edf897a3ffa55b89f906fc8cc78afdb3f1834cc9c7083565e611a8a7d5fe99e 0.1s done
#4 CACHED
#5 [migrate internal] load metadata for docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641
#5 DONE 0.1s
#6 [api internal] load .dockerignore
#6 transferring context: 196B done
#6 DONE 0.0s
#7 [migrate internal] load build context
#7 transferring context: 219.60kB 0.0s done
#7 DONE 0.1s
#8 [api 1/8] FROM docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641
#8 resolve docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641 0.1s done
#8 DONE 0.1s
#9 [migrate 7/8] COPY migrations/ ./migrations/
#9 CACHED
#10 [migrate 2/8] WORKDIR /app
#10 CACHED
#11 [migrate 3/8] COPY requirements.txt ./
#11 CACHED
#12 [migrate 5/8] RUN groupadd --system --gid 10001 app && useradd --system --uid 10001 --gid app --no-create-home --shell /usr/sbin/nologin app
#12 CACHED
#13 [migrate 4/8] RUN pip install --root-user-action=ignore --require-hashes --no-deps -r requirements.txt
#13 CACHED
#14 [migrate 6/8] COPY src/ ./src/
#14 CACHED
#15 [migrate 8/8] COPY contracts/openapi.json ./contracts/openapi.json
#15 CACHED
#12 [api 5/8] RUN groupadd --system --gid 10001 app && useradd --system --uid 10001 --gid app --no-create-home --shell /usr/sbin/nologin app
#12 CACHED
#14 [api 6/8] COPY src/ ./src/
#14 CACHED
#10 [api 2/8] WORKDIR /app
#10 CACHED
#11 [api 3/8] COPY requirements.txt ./
#11 CACHED
#9 [api 7/8] COPY migrations/ ./migrations/
#9 CACHED
#13 [api 4/8] RUN pip install --root-user-action=ignore --require-hashes --no-deps -r requirements.txt
#13 CACHED
#16 [api] exporting to image
#16 exporting layers 0.0s done
#16 exporting manifest sha256:2e7919b4c60cf9f2135cb5523cf47bc6456f92675261377cb3a312868c357738 0.0s done
#16 exporting config sha256:986458b901447acf8fa6913d7b9548d1fb0b4335f4169158ad6eb2c5e050c1fc 0.0s done
#16 exporting attestation manifest sha256:712b9fac08d59b57dc8931284b52c54ddf116bc5ad39d17e789a98a04499f68d
#16 exporting attestation manifest sha256:712b9fac08d59b57dc8931284b52c54ddf116bc5ad39d17e789a98a04499f68d 0.1s done
#16 exporting manifest list sha256:331feed9c7da600658f524637b0ed8aa919c68e0372b0191d89b3d45e37122e0
#16 exporting manifest list sha256:331feed9c7da600658f524637b0ed8aa919c68e0372b0191d89b3d45e37122e0 0.1s done
#16 naming to docker.io/library/mtusi-messenger:lab3 0.0s done
#16 unpacking to docker.io/library/mtusi-messenger:lab3 0.0s done
#16 DONE 0.4s
#17 [migrate] exporting to image
#17 exporting layers 0.0s done
#17 exporting manifest sha256:06ff1601ca5b94a5b839cf1b1db979c3b1be6f371a81a349c64a49eba4aa718d 0.0s done
#17 exporting config sha256:e415695b978557fe5a23bf1d0a4a95e3e24651ab739b95088dfa124dd115d26a 0.0s done
#17 exporting attestation manifest sha256:3d9641bd01df9b87cdebd4f0b07e8f90504f14b9d3a090c6d8794e121b69c3d7 0.1s done
#17 exporting manifest list sha256:e7940cf7f96b1b5f7c63645c54e1774d32d1408edcfced01dd5b57e93bcb64c4 0.1s done
#17 naming to docker.io/library/mtusi-messenger:lab3 0.0s done
#17 unpacking to docker.io/library/mtusi-messenger:lab3 0.0s done
#17 DONE 0.3s
#18 [migrate] resolving provenance for metadata file
#18 DONE 0.0s
#19 [api] resolving provenance for metadata file
#19 DONE 0.0s
Image mtusi-messenger:lab3 Building
Image mtusi-messenger:lab3 Building
Image mtusi-messenger:lab3 Built
Image mtusi-messenger:lab3 Built
Network mtusi-lab3_edge Creating
Network mtusi-lab3_edge Creating
Volume mtusi-lab3_pgdata Creating
Volume mtusi-lab3_pgdata Creating
Volume mtusi-lab3_valkeydata Creating
Volume mtusi-lab3_valkeydata Creating
Network mtusi-lab3_data Creating
Network mtusi-lab3_data Creating
Volume mtusi-lab3_valkeydata Created
Volume mtusi-lab3_valkeydata Created
Volume mtusi-lab3_pgdata Created
Volume mtusi-lab3_pgdata Created
Network mtusi-lab3_edge Created
Network mtusi-lab3_edge Created
Network mtusi-lab3_data Created
Network mtusi-lab3_data Created
Container mtusi-lab3-postgres-1 Creating
Container mtusi-lab3-valkey-1 Creating
Container mtusi-lab3-valkey-1 Created
Container mtusi-lab3-postgres-1 Created
Container mtusi-lab3-migrate-1 Creating
Container mtusi-lab3-migrate-1 Created
Container mtusi-lab3-api-1 Creating
Container mtusi-lab3-api-2 Creating
Container mtusi-lab3-api-1 Created
Container mtusi-lab3-api-2 Created
Container mtusi-lab3-lb-1 Creating
Container mtusi-lab3-lb-1 Created
Container mtusi-lab3-postgres-1 Starting
Container mtusi-lab3-valkey-1 Starting
Container mtusi-lab3-postgres-1 Started
Container mtusi-lab3-postgres-1 Waiting
Container mtusi-lab3-valkey-1 Started
Container mtusi-lab3-postgres-1 Healthy
Container mtusi-lab3-migrate-1 Starting
Container mtusi-lab3-migrate-1 Started
Container mtusi-lab3-migrate-1 Waiting
Container mtusi-lab3-postgres-1 Waiting
Container mtusi-lab3-valkey-1 Waiting
Container mtusi-lab3-valkey-1 Healthy
Container mtusi-lab3-postgres-1 Healthy
Container mtusi-lab3-migrate-1 Exited
Container mtusi-lab3-api-2 Starting
Container mtusi-lab3-api-2 Started
Container mtusi-lab3-api-1 Starting
Container mtusi-lab3-api-1 Started
Container mtusi-lab3-api-2 Waiting
Container mtusi-lab3-api-1 Waiting
Container mtusi-lab3-api-2 Healthy
Container mtusi-lab3-api-1 Healthy
Container mtusi-lab3-lb-1 Starting
Container mtusi-lab3-lb-1 Started
Container mtusi-lab3-valkey-1 Waiting
Container mtusi-lab3-migrate-1 Waiting
Container mtusi-lab3-api-2 Waiting
Container mtusi-lab3-api-1 Waiting
Container mtusi-lab3-lb-1 Waiting
Container mtusi-lab3-postgres-1 Waiting
Container mtusi-lab3-postgres-1 Healthy
Container mtusi-lab3-valkey-1 Healthy
Container mtusi-lab3-migrate-1 Exited
Container mtusi-lab3-api-2 Healthy
Container mtusi-lab3-api-1 Healthy
Container mtusi-lab3-lb-1 Healthy
[exit 0, 28.2 s]
$ docker compose ps -a --format table {{.Service}} {{.Image}} {{.Status}}
SERVICE IMAGE STATUS
api mtusi-messenger:lab3 Up 11 seconds (healthy)
api mtusi-messenger:lab3 Up 12 seconds (healthy)
lb haproxy:3.2.26-alpine@sha256:254e6370f01cd654e0744edf607c67f33777c1b6a9dc0d4b28bcba7171460122 Up 6 seconds (healthy)
migrate mtusi-messenger:lab3 Exited (0) 12 seconds ago
postgres postgres:18.6-bookworm@sha256:afc7e2d441324c0388fa80c3d24f733b4194a4eb7f47dd8ee2b08eb1a24a647c Up 20 seconds (healthy)
valkey valkey/valkey:8.1.3-bookworm@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571 Up 19 seconds (healthy)
[exit 0, 0.3 s]
$ docker compose logs --no-log-prefix migrate
migrate: 0001_init: applied
migrate: 0002_idempotency_keys: applied
migrate: 0003_post_message_function: applied
migrate: done, 3 migration file(s) checked
[exit 0, 0.3 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py replicas
mtusi-lab3-api-1 X-Instance-Id=94b38b941a8d running health=healthy pid=23944 started=2026-10-19T15:10:34.257 ip=172.23.0.3,172.24.0.5
mtusi-lab3-api-2 X-Instance-Id=a8d9cffd7a6e running health=healthy pid=23887 started=2026-10-19T15:10:33.970 ip=172.23.0.2,172.24.0.4
[exit 0, 0.7 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py status
api1 172.23.0.3 UP mtusi-lab3-api-1 X-Instance-Id=94b38b941a8d
api2 172.23.0.2 UP mtusi-lab3-api-2 X-Instance-Id=a8d9cffd7a6e
(other slots: MAINT, no container)
[exit 0, 0.9 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe -m venv .venv
[exit 0, 6.9 s]
$ C:\Users\wfa22\Desktop\Высоконагруженные приложения\mtusi2026lab3-clean\.venv\Scripts\python.exe -m pip install -q --require-hashes --no-deps -r requirements.txt
[notice] A new release of pip is available: 26.1.2 -> 26.2.1
[notice] To update, run: C:\Users\wfa22\Desktop\Высоконагруженные приложения\mtusi2026lab3-clean\.venv\Scripts\python.exe -m pip install --upgrade pip
[exit 0, 15.8 s]
$ C:\Users\wfa22\Desktop\Высоконагруженные приложения\mtusi2026lab3-clean\.venv\Scripts\python.exe -m pip install -q -r requirements-test.txt -r requirements-dev.txt
[notice] A new release of pip is available: 26.1.2 -> 26.2.1
[notice] To update, run: C:\Users\wfa22\Desktop\Высоконагруженные приложения\mtusi2026lab3-clean\.venv\Scripts\python.exe -m pip install --upgrade pip
[exit 0, 22.3 s]
$ git check-ignore -v secrets/database_url .env
.gitignore:5:secrets/ secrets/database_url
.gitignore:2:.env .env
$ git status --short # secrets/ и .env не попадают в Git
?? evidence/clean-clone.txt
?? evidence/failover/
?? evidence/level4-runs.txt
?? evidence/load.json
?? evidence/make-check-validate-test.txt
?? evidence/rolling-update/
?? evidence/student-tests.txt
?? evidence/test-failover.txt
?? evidence/test-persistence.txt
[exit 0]
+504
View File
@@ -0,0 +1,504 @@
{
"meta": {
"date": "2026-10-19T15:23:54+00:00",
"commit": "2708732",
"host": "Windows 10, docker 29.8.0",
"workload": "4 closed-loop workers from the host via 127.0.0.1:8080: POST message (Idempotency-Key, uncertain -> same key) + GET last page; client timeout 3 s",
"lb": "HAProxy: check GET /health/ready inter 1s fastinter 500ms fall 2 rise 2, timeout check 1s; retry-on conn-failure + redispatch",
"api": "DRAIN_SECONDS=3, SHUTDOWN_TIMEOUT_SECONDS=8, stop_grace_period 15s"
},
"rounds": [
{
"mode": "stop",
"replica": "mtusi-lab3-api-1",
"instance_id": "907f96037a59",
"stop_command_seconds": 4.03,
"before": {
"seconds": 5.5,
"requests": 1621,
"ok": 1621,
"failed": 0,
"ok_rps": 294.6,
"error_ratio": 0.0,
"statuses": {
"201": 812,
"200": 809
},
"instances": {
"c3f64421973d": 810,
"907f96037a59": 811
},
"p50_ms": 10.88,
"p95_ms": 31.93,
"p99_ms": 60.82
},
"during": {
"seconds": 16.03,
"requests": 3568,
"ok": 3568,
"failed": 0,
"ok_rps": 222.5,
"error_ratio": 0.0,
"statuses": {
"200": 1784,
"201": 1784
},
"instances": {
"c3f64421973d": 3434,
"907f96037a59": 134
},
"p50_ms": 13.82,
"p95_ms": 61.1,
"p99_ms": 71.93
},
"after": {
"seconds": 6.02,
"requests": 1503,
"ok": 1503,
"failed": 0,
"ok_rps": 249.7,
"error_ratio": 0.0,
"statuses": {
"201": 750,
"200": 753
},
"instances": {
"c3f64421973d": 753,
"907f96037a59": 750
},
"p50_ms": 11.19,
"p95_ms": 55.98,
"p99_ms": 63.38
},
"failed_after_stop": 0,
"failed_statuses": [],
"last_failure_after_stop_s": 0.0,
"target_last_served_after_stop_s": 1.283,
"rejoin_up_in_lb_s": 3.25,
"rejoin_serving_s": 3.35,
"client_retries_same_key": 0,
"history": {
"history_messages": 3692,
"confirmed_messages": 3692,
"lost_confirmed": 0,
"altered_confirmed": 0,
"duplicate_ids": 0,
"duplicate_texts": 0,
"unconfirmed_but_stored": 0
},
"restarted_state": "running health=healthy"
},
{
"mode": "stop",
"replica": "mtusi-lab3-api-2",
"instance_id": "c3f64421973d",
"stop_command_seconds": 3.79,
"before": {
"seconds": 5.5,
"requests": 1426,
"ok": 1426,
"failed": 0,
"ok_rps": 259.2,
"error_ratio": 0.0,
"statuses": {
"201": 714,
"200": 712
},
"instances": {
"c3f64421973d": 713,
"907f96037a59": 713
},
"p50_ms": 11.18,
"p95_ms": 53.47,
"p99_ms": 64.41
},
"during": {
"seconds": 15.79,
"requests": 3565,
"ok": 3565,
"failed": 0,
"ok_rps": 225.7,
"error_ratio": 0.0,
"statuses": {
"200": 1782,
"201": 1783
},
"instances": {
"c3f64421973d": 222,
"907f96037a59": 3343
},
"p50_ms": 13.63,
"p95_ms": 63.19,
"p99_ms": 71.39
},
"after": {
"seconds": 6.02,
"requests": 1656,
"ok": 1656,
"failed": 0,
"ok_rps": 275.1,
"error_ratio": 0.0,
"statuses": {
"201": 827,
"200": 829
},
"instances": {
"c3f64421973d": 828,
"907f96037a59": 828
},
"p50_ms": 11.19,
"p95_ms": 41.65,
"p99_ms": 60.73
},
"failed_after_stop": 0,
"failed_statuses": [],
"last_failure_after_stop_s": 0.0,
"target_last_served_after_stop_s": 1.235,
"rejoin_up_in_lb_s": 1.35,
"rejoin_serving_s": 3.07,
"client_retries_same_key": 0,
"history": {
"history_messages": 3690,
"confirmed_messages": 3690,
"lost_confirmed": 0,
"altered_confirmed": 0,
"duplicate_ids": 0,
"duplicate_texts": 0,
"unconfirmed_but_stored": 0
},
"restarted_state": "running health=healthy"
},
{
"mode": "kill",
"replica": "mtusi-lab3-api-1",
"instance_id": "907f96037a59",
"stop_command_seconds": 0.72,
"before": {
"seconds": 5.5,
"requests": 1491,
"ok": 1491,
"failed": 0,
"ok_rps": 270.9,
"error_ratio": 0.0,
"statuses": {
"201": 745,
"200": 746
},
"instances": {
"907f96037a59": 746,
"c3f64421973d": 745
},
"p50_ms": 11.18,
"p95_ms": 41.28,
"p99_ms": 56.2
},
"during": {
"seconds": 12.72,
"requests": 2734,
"ok": 2731,
"failed": 3,
"ok_rps": 214.7,
"error_ratio": 0.0011,
"statuses": {
"201": 1366,
"200": 1365,
"0": 3
},
"instances": {
"907f96037a59": 22,
"c3f64421973d": 2709
},
"p50_ms": 13.72,
"p95_ms": 59.09,
"p99_ms": 69.77
},
"after": {
"seconds": 6.02,
"requests": 1477,
"ok": 1477,
"failed": 0,
"ok_rps": 245.5,
"error_ratio": 0.0,
"statuses": {
"200": 740,
"201": 737
},
"instances": {
"c3f64421973d": 739,
"907f96037a59": 738
},
"p50_ms": 11.54,
"p95_ms": 54.73,
"p99_ms": 62.68
},
"failed_after_stop": 3,
"failed_statuses": [
0
],
"last_failure_after_stop_s": 0.247,
"target_last_served_after_stop_s": 0.22,
"rejoin_up_in_lb_s": 1.17,
"rejoin_serving_s": 2.8,
"client_retries_same_key": 2,
"history": {
"history_messages": 3200,
"confirmed_messages": 3200,
"lost_confirmed": 0,
"altered_confirmed": 0,
"duplicate_ids": 0,
"duplicate_texts": 0,
"unconfirmed_but_stored": 0
},
"restarted_state": "running health=healthy"
},
{
"mode": "kill",
"replica": "mtusi-lab3-api-2",
"instance_id": "c3f64421973d",
"stop_command_seconds": 0.77,
"before": {
"seconds": 5.5,
"requests": 1241,
"ok": 1241,
"failed": 0,
"ok_rps": 225.6,
"error_ratio": 0.0,
"statuses": {
"201": 620,
"200": 621
},
"instances": {
"c3f64421973d": 621,
"907f96037a59": 620
},
"p50_ms": 11.41,
"p95_ms": 57.52,
"p99_ms": 66.91
},
"during": {
"seconds": 12.77,
"requests": 2297,
"ok": 2294,
"failed": 3,
"ok_rps": 179.6,
"error_ratio": 0.00131,
"statuses": {
"200": 1147,
"201": 1147,
"0": 3
},
"instances": {
"907f96037a59": 2276,
"c3f64421973d": 18
},
"p50_ms": 15.06,
"p95_ms": 66.96,
"p99_ms": 77.44
},
"after": {
"seconds": 6.01,
"requests": 1442,
"ok": 1442,
"failed": 0,
"ok_rps": 240.0,
"error_ratio": 0.0,
"statuses": {
"200": 722,
"201": 720
},
"instances": {
"c3f64421973d": 721,
"907f96037a59": 721
},
"p50_ms": 11.64,
"p95_ms": 53.4,
"p99_ms": 65.32
},
"failed_after_stop": 3,
"failed_statuses": [
0
],
"last_failure_after_stop_s": 0.214,
"target_last_served_after_stop_s": 0.174,
"rejoin_up_in_lb_s": 2.32,
"rejoin_serving_s": 3.64,
"client_retries_same_key": 2,
"history": {
"history_messages": 2885,
"confirmed_messages": 2885,
"lost_confirmed": 0,
"altered_confirmed": 0,
"duplicate_ids": 0,
"duplicate_texts": 0,
"unconfirmed_but_stored": 0
},
"restarted_state": "running health=healthy"
},
{
"mode": "pause",
"replica": "mtusi-lab3-api-1",
"instance_id": "907f96037a59",
"stop_command_seconds": 0.2,
"before": {
"seconds": 5.5,
"requests": 1493,
"ok": 1493,
"failed": 0,
"ok_rps": 271.3,
"error_ratio": 0.0,
"statuses": {
"200": 747,
"201": 746
},
"instances": {
"907f96037a59": 747,
"c3f64421973d": 746
},
"p50_ms": 11.06,
"p95_ms": 49.85,
"p99_ms": 61.7
},
"during": {
"seconds": 12.2,
"requests": 1653,
"ok": 1649,
"failed": 4,
"ok_rps": 135.2,
"error_ratio": 0.00242,
"statuses": {
"201": 824,
"200": 825,
"0": 4
},
"instances": {
"c3f64421973d": 1637,
"907f96037a59": 12
},
"p50_ms": 15.99,
"p95_ms": 68.99,
"p99_ms": 79.54
},
"after": {
"seconds": 6.03,
"requests": 1474,
"ok": 1474,
"failed": 0,
"ok_rps": 244.6,
"error_ratio": 0.0,
"statuses": {
"201": 736,
"200": 738
},
"instances": {
"907f96037a59": 737,
"c3f64421973d": 737
},
"p50_ms": 12.41,
"p95_ms": 49.6,
"p99_ms": 64.12
},
"failed_after_stop": 4,
"failed_statuses": [
0
],
"last_failure_after_stop_s": 3.204,
"target_last_served_after_stop_s": 0.148,
"rejoin_up_in_lb_s": 2.21,
"rejoin_serving_s": 2.31,
"client_retries_same_key": 3,
"history": {
"history_messages": 2610,
"confirmed_messages": 2610,
"lost_confirmed": 0,
"altered_confirmed": 0,
"duplicate_ids": 0,
"duplicate_texts": 0,
"unconfirmed_but_stored": 0
},
"restarted_state": "running health=healthy"
},
{
"mode": "pause",
"replica": "mtusi-lab3-api-2",
"instance_id": "c3f64421973d",
"stop_command_seconds": 0.13,
"before": {
"seconds": 5.5,
"requests": 1395,
"ok": 1395,
"failed": 0,
"ok_rps": 253.6,
"error_ratio": 0.0,
"statuses": {
"200": 697,
"201": 698
},
"instances": {
"c3f64421973d": 698,
"907f96037a59": 697
},
"p50_ms": 12.39,
"p95_ms": 44.63,
"p99_ms": 60.81
},
"during": {
"seconds": 12.13,
"requests": 1787,
"ok": 1783,
"failed": 4,
"ok_rps": 147.0,
"error_ratio": 0.00224,
"statuses": {
"200": 890,
"201": 893,
"0": 4
},
"instances": {
"907f96037a59": 1768,
"c3f64421973d": 15
},
"p50_ms": 15.86,
"p95_ms": 65.74,
"p99_ms": 73.59
},
"after": {
"seconds": 6.02,
"requests": 1522,
"ok": 1522,
"failed": 0,
"ok_rps": 253.0,
"error_ratio": 0.0,
"statuses": {
"200": 762,
"201": 760
},
"instances": {
"c3f64421973d": 761,
"907f96037a59": 761
},
"p50_ms": 12.2,
"p95_ms": 44.57,
"p99_ms": 61.5
},
"failed_after_stop": 4,
"failed_statuses": [
0
],
"last_failure_after_stop_s": 3.12,
"target_last_served_after_stop_s": 0.098,
"rejoin_up_in_lb_s": 1.16,
"rejoin_serving_s": 1.27,
"client_retries_same_key": 1,
"history": {
"history_messages": 2558,
"confirmed_messages": 2558,
"lost_confirmed": 0,
"altered_confirmed": 0,
"duplicate_ids": 0,
"duplicate_texts": 0,
"unconfirmed_but_stored": 0
},
"restarted_state": "running health=healthy"
}
]
}
+10
View File
@@ -0,0 +1,10 @@
Дата 2026-10-19T15:23:54+00:00, commit 2708732. 4 closed-loop workers from the host via 127.0.0.1:8080: POST message (Idempotency-Key, uncertain -> same key) + GET last page; client timeout 3 s.
| Режим | Реплика | Запросов во время отказа | Ошибок после stop | ok-RPS до / во время / после | Последняя ошибка, с | Цель ушла из ротации, с | Возврат в LB / обслуживает, с | Потери / дубли |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| stop | mtusi-lab3-api-1 | 3568 | 0 | 294.6 / 222.5 / 249.7 | 0.0 | 1.283 | 3.25 / 3.35 | 0 / 0 |
| stop | mtusi-lab3-api-2 | 3565 | 0 | 259.2 / 225.7 / 275.1 | 0.0 | 1.235 | 1.35 / 3.07 | 0 / 0 |
| kill | mtusi-lab3-api-1 | 2734 | 3 [0] | 270.9 / 214.7 / 245.5 | 0.247 | 0.22 | 1.17 / 2.8 | 0 / 0 |
| kill | mtusi-lab3-api-2 | 2297 | 3 [0] | 225.6 / 179.6 / 240.0 | 0.214 | 0.174 | 2.32 / 3.64 | 0 / 0 |
| pause | mtusi-lab3-api-1 | 1653 | 4 [0] | 271.3 / 135.2 / 244.6 | 3.204 | 0.148 | 2.21 / 2.31 | 0 / 0 |
| pause | mtusi-lab3-api-2 | 1787 | 4 [0] | 253.6 / 147.0 / 253.0 | 3.12 | 0.098 | 1.16 / 1.27 | 0 / 0 |
@@ -0,0 +1,29 @@
second,ok,failed,instances
0,341,0,TARGET:170|c3f64421973d:171
1,195,0,TARGET:98|c3f64421973d:97
2,348,0,TARGET:174|c3f64421973d:174
3,324,0,TARGET:162|c3f64421973d:162
4,173,0,TARGET:86|c3f64421973d:87
5,283,0,TARGET:142|c3f64421973d:141
6,79,3,-:3|TARGET:22|c3f64421973d:57
7,142,0,c3f64421973d:142
8,304,0,c3f64421973d:304
9,226,0,c3f64421973d:226
10,164,0,c3f64421973d:164
11,281,0,c3f64421973d:281
12,261,0,c3f64421973d:261
13,113,0,c3f64421973d:113
14,285,0,c3f64421973d:285
15,260,0,c3f64421973d:260
16,140,0,c3f64421973d:140
17,251,0,c3f64421973d:251
18,290,0,c3f64421973d:290
19,127,0,c3f64421973d:127
20,181,0,c3f64421973d:181
21,315,0,TARGET:88|c3f64421973d:227
22,300,0,TARGET:152|c3f64421973d:148
23,196,0,TARGET:95|c3f64421973d:101
24,194,0,TARGET:97|c3f64421973d:97
25,308,0,TARGET:157|c3f64421973d:151
26,194,0,TARGET:97|c3f64421973d:97
27,126,0,TARGET:61|c3f64421973d:65
1 second ok failed instances
2 0 341 0 TARGET:170|c3f64421973d:171
3 1 195 0 TARGET:98|c3f64421973d:97
4 2 348 0 TARGET:174|c3f64421973d:174
5 3 324 0 TARGET:162|c3f64421973d:162
6 4 173 0 TARGET:86|c3f64421973d:87
7 5 283 0 TARGET:142|c3f64421973d:141
8 6 79 3 -:3|TARGET:22|c3f64421973d:57
9 7 142 0 c3f64421973d:142
10 8 304 0 c3f64421973d:304
11 9 226 0 c3f64421973d:226
12 10 164 0 c3f64421973d:164
13 11 281 0 c3f64421973d:281
14 12 261 0 c3f64421973d:261
15 13 113 0 c3f64421973d:113
16 14 285 0 c3f64421973d:285
17 15 260 0 c3f64421973d:260
18 16 140 0 c3f64421973d:140
19 17 251 0 c3f64421973d:251
20 18 290 0 c3f64421973d:290
21 19 127 0 c3f64421973d:127
22 20 181 0 c3f64421973d:181
23 21 315 0 TARGET:88|c3f64421973d:227
24 22 300 0 TARGET:152|c3f64421973d:148
25 23 196 0 TARGET:95|c3f64421973d:101
26 24 194 0 TARGET:97|c3f64421973d:97
27 25 308 0 TARGET:157|c3f64421973d:151
28 26 194 0 TARGET:97|c3f64421973d:97
29 27 126 0 TARGET:61|c3f64421973d:65
@@ -0,0 +1,30 @@
second,ok,failed,instances
0,299,0,907f96037a59:151|TARGET:148
1,191,0,907f96037a59:92|TARGET:99
2,274,0,907f96037a59:139|TARGET:135
3,335,0,907f96037a59:169|TARGET:166
4,191,0,907f96037a59:92|TARGET:99
5,129,0,907f96037a59:66|TARGET:63
6,61,3,-:3|907f96037a59:43|TARGET:18
7,98,0,907f96037a59:98
8,94,0,907f96037a59:94
9,225,0,907f96037a59:225
10,288,0,907f96037a59:288
11,174,0,907f96037a59:174
12,212,0,907f96037a59:212
13,274,0,907f96037a59:274
14,199,0,907f96037a59:199
15,118,0,907f96037a59:118
16,240,0,907f96037a59:240
17,220,0,907f96037a59:220
18,119,0,907f96037a59:119
19,202,0,907f96037a59:202
20,198,0,907f96037a59:198
21,114,0,907f96037a59:114
22,288,0,907f96037a59:166|TARGET:122
23,323,0,907f96037a59:163|TARGET:160
24,176,0,907f96037a59:85|TARGET:91
25,179,0,907f96037a59:91|TARGET:88
26,284,0,907f96037a59:143|TARGET:141
27,179,0,907f96037a59:87|TARGET:92
28,87,0,907f96037a59:45|TARGET:42
1 second ok failed instances
2 0 299 0 907f96037a59:151|TARGET:148
3 1 191 0 907f96037a59:92|TARGET:99
4 2 274 0 907f96037a59:139|TARGET:135
5 3 335 0 907f96037a59:169|TARGET:166
6 4 191 0 907f96037a59:92|TARGET:99
7 5 129 0 907f96037a59:66|TARGET:63
8 6 61 3 -:3|907f96037a59:43|TARGET:18
9 7 98 0 907f96037a59:98
10 8 94 0 907f96037a59:94
11 9 225 0 907f96037a59:225
12 10 288 0 907f96037a59:288
13 11 174 0 907f96037a59:174
14 12 212 0 907f96037a59:212
15 13 274 0 907f96037a59:274
16 14 199 0 907f96037a59:199
17 15 118 0 907f96037a59:118
18 16 240 0 907f96037a59:240
19 17 220 0 907f96037a59:220
20 18 119 0 907f96037a59:119
21 19 202 0 907f96037a59:202
22 20 198 0 907f96037a59:198
23 21 114 0 907f96037a59:114
24 22 288 0 907f96037a59:166|TARGET:122
25 23 323 0 907f96037a59:163|TARGET:160
26 24 176 0 907f96037a59:85|TARGET:91
27 25 179 0 907f96037a59:91|TARGET:88
28 26 284 0 907f96037a59:143|TARGET:141
29 27 179 0 907f96037a59:87|TARGET:92
30 28 87 0 907f96037a59:45|TARGET:42
@@ -0,0 +1,26 @@
second,ok,failed,instances
0,326,0,TARGET:165|c3f64421973d:161
1,175,0,TARGET:84|c3f64421973d:91
2,329,0,TARGET:166|c3f64421973d:163
3,352,0,TARGET:178|c3f64421973d:174
4,179,0,TARGET:86|c3f64421973d:93
5,294,0,TARGET:149|c3f64421973d:145
6,30,0,TARGET:13|c3f64421973d:17
9,123,4,-:4|c3f64421973d:123
10,148,0,c3f64421973d:148
11,254,0,c3f64421973d:254
12,246,0,c3f64421973d:246
13,118,0,c3f64421973d:118
14,253,0,c3f64421973d:253
15,223,0,c3f64421973d:223
16,67,0,c3f64421973d:67
17,139,0,c3f64421973d:139
18,189,0,TARGET:15|c3f64421973d:174
19,227,0,TARGET:115|c3f64421973d:112
20,160,0,TARGET:77|c3f64421973d:83
21,258,0,TARGET:131|c3f64421973d:127
22,326,0,TARGET:163|c3f64421973d:163
23,167,0,TARGET:82|c3f64421973d:85
24,254,0,TARGET:128|c3f64421973d:126
25,301,0,TARGET:151|c3f64421973d:150
26,84,0,TARGET:42|c3f64421973d:42
1 second ok failed instances
2 0 326 0 TARGET:165|c3f64421973d:161
3 1 175 0 TARGET:84|c3f64421973d:91
4 2 329 0 TARGET:166|c3f64421973d:163
5 3 352 0 TARGET:178|c3f64421973d:174
6 4 179 0 TARGET:86|c3f64421973d:93
7 5 294 0 TARGET:149|c3f64421973d:145
8 6 30 0 TARGET:13|c3f64421973d:17
9 9 123 4 -:4|c3f64421973d:123
10 10 148 0 c3f64421973d:148
11 11 254 0 c3f64421973d:254
12 12 246 0 c3f64421973d:246
13 13 118 0 c3f64421973d:118
14 14 253 0 c3f64421973d:253
15 15 223 0 c3f64421973d:223
16 16 67 0 c3f64421973d:67
17 17 139 0 c3f64421973d:139
18 18 189 0 TARGET:15|c3f64421973d:174
19 19 227 0 TARGET:115|c3f64421973d:112
20 20 160 0 TARGET:77|c3f64421973d:83
21 21 258 0 TARGET:131|c3f64421973d:127
22 22 326 0 TARGET:163|c3f64421973d:163
23 23 167 0 TARGET:82|c3f64421973d:85
24 24 254 0 TARGET:128|c3f64421973d:126
25 25 301 0 TARGET:151|c3f64421973d:150
26 26 84 0 TARGET:42|c3f64421973d:42
@@ -0,0 +1,25 @@
second,ok,failed,instances
0,304,0,907f96037a59:152|TARGET:152
1,248,0,907f96037a59:125|TARGET:123
2,192,0,907f96037a59:95|TARGET:97
3,334,0,907f96037a59:167|TARGET:167
4,280,0,907f96037a59:141|TARGET:139
5,197,0,907f96037a59:97|TARGET:100
6,34,0,907f96037a59:19|TARGET:15
9,181,4,-:4|907f96037a59:181
10,173,0,907f96037a59:173
11,169,0,907f96037a59:169
12,268,0,907f96037a59:268
13,191,0,907f96037a59:191
14,133,0,907f96037a59:133
15,250,0,907f96037a59:250
16,241,0,907f96037a59:241
17,113,0,907f96037a59:113
18,166,0,907f96037a59:145|TARGET:21
19,308,0,907f96037a59:154|TARGET:154
20,167,0,907f96037a59:85|TARGET:82
21,260,0,907f96037a59:129|TARGET:131
22,303,0,907f96037a59:151|TARGET:152
23,211,0,907f96037a59:107|TARGET:104
24,261,0,907f96037a59:129|TARGET:132
25,130,0,907f96037a59:65|TARGET:65
1 second ok failed instances
2 0 304 0 907f96037a59:152|TARGET:152
3 1 248 0 907f96037a59:125|TARGET:123
4 2 192 0 907f96037a59:95|TARGET:97
5 3 334 0 907f96037a59:167|TARGET:167
6 4 280 0 907f96037a59:141|TARGET:139
7 5 197 0 907f96037a59:97|TARGET:100
8 6 34 0 907f96037a59:19|TARGET:15
9 9 181 4 -:4|907f96037a59:181
10 10 173 0 907f96037a59:173
11 11 169 0 907f96037a59:169
12 12 268 0 907f96037a59:268
13 13 191 0 907f96037a59:191
14 14 133 0 907f96037a59:133
15 15 250 0 907f96037a59:250
16 16 241 0 907f96037a59:241
17 17 113 0 907f96037a59:113
18 18 166 0 907f96037a59:145|TARGET:21
19 19 308 0 907f96037a59:154|TARGET:154
20 20 167 0 907f96037a59:85|TARGET:82
21 21 260 0 907f96037a59:129|TARGET:131
22 22 303 0 907f96037a59:151|TARGET:152
23 23 211 0 907f96037a59:107|TARGET:104
24 24 261 0 907f96037a59:129|TARGET:132
25 25 130 0 907f96037a59:65|TARGET:65
@@ -0,0 +1,33 @@
second,ok,failed,instances
0,171,0,TARGET:85|c3f64421973d:86
1,368,0,TARGET:185|c3f64421973d:183
2,272,0,TARGET:137|c3f64421973d:135
3,187,0,TARGET:92|c3f64421973d:95
4,370,0,TARGET:185|c3f64421973d:185
5,330,0,TARGET:166|c3f64421973d:164
6,180,0,TARGET:89|c3f64421973d:91
7,294,0,TARGET:46|c3f64421973d:248
8,227,0,c3f64421973d:227
9,109,0,c3f64421973d:109
10,218,0,c3f64421973d:218
11,277,0,c3f64421973d:277
12,164,0,c3f64421973d:164
13,243,0,c3f64421973d:243
14,298,0,c3f64421973d:298
15,181,0,c3f64421973d:181
16,223,0,c3f64421973d:223
17,292,0,c3f64421973d:292
18,156,0,c3f64421973d:156
19,197,0,c3f64421973d:197
20,277,0,c3f64421973d:277
21,224,0,c3f64421973d:224
22,108,0,c3f64421973d:108
23,207,0,c3f64421973d:207
24,229,0,TARGET:15|c3f64421973d:214
25,209,0,TARGET:105|c3f64421973d:104
26,343,0,TARGET:171|c3f64421973d:172
27,217,0,TARGET:107|c3f64421973d:110
28,168,0,TARGET:86|c3f64421973d:82
29,286,0,TARGET:142|c3f64421973d:144
30,273,0,TARGET:136|c3f64421973d:137
31,86,0,TARGET:44|c3f64421973d:42
1 second ok failed instances
2 0 171 0 TARGET:85|c3f64421973d:86
3 1 368 0 TARGET:185|c3f64421973d:183
4 2 272 0 TARGET:137|c3f64421973d:135
5 3 187 0 TARGET:92|c3f64421973d:95
6 4 370 0 TARGET:185|c3f64421973d:185
7 5 330 0 TARGET:166|c3f64421973d:164
8 6 180 0 TARGET:89|c3f64421973d:91
9 7 294 0 TARGET:46|c3f64421973d:248
10 8 227 0 c3f64421973d:227
11 9 109 0 c3f64421973d:109
12 10 218 0 c3f64421973d:218
13 11 277 0 c3f64421973d:277
14 12 164 0 c3f64421973d:164
15 13 243 0 c3f64421973d:243
16 14 298 0 c3f64421973d:298
17 15 181 0 c3f64421973d:181
18 16 223 0 c3f64421973d:223
19 17 292 0 c3f64421973d:292
20 18 156 0 c3f64421973d:156
21 19 197 0 c3f64421973d:197
22 20 277 0 c3f64421973d:277
23 21 224 0 c3f64421973d:224
24 22 108 0 c3f64421973d:108
25 23 207 0 c3f64421973d:207
26 24 229 0 TARGET:15|c3f64421973d:214
27 25 209 0 TARGET:105|c3f64421973d:104
28 26 343 0 TARGET:171|c3f64421973d:172
29 27 217 0 TARGET:107|c3f64421973d:110
30 28 168 0 TARGET:86|c3f64421973d:82
31 29 286 0 TARGET:142|c3f64421973d:144
32 30 273 0 TARGET:136|c3f64421973d:137
33 31 86 0 TARGET:44|c3f64421973d:42
@@ -0,0 +1,32 @@
second,ok,failed,instances
0,360,0,907f96037a59:180|TARGET:180
1,187,0,907f96037a59:93|TARGET:94
2,271,0,907f96037a59:136|TARGET:135
3,368,0,907f96037a59:184|TARGET:184
4,240,0,907f96037a59:120|TARGET:120
5,182,0,907f96037a59:91|TARGET:91
6,358,0,907f96037a59:179|TARGET:179
7,199,0,907f96037a59:156|TARGET:43
8,158,0,907f96037a59:158
9,249,0,907f96037a59:249
10,245,0,907f96037a59:245
11,129,0,907f96037a59:129
12,291,0,907f96037a59:291
13,291,0,907f96037a59:291
14,127,0,907f96037a59:127
15,224,0,907f96037a59:224
16,295,0,907f96037a59:295
17,139,0,907f96037a59:139
18,238,0,907f96037a59:238
19,299,0,907f96037a59:299
20,165,0,907f96037a59:165
21,207,0,907f96037a59:207
22,193,0,907f96037a59:193
23,170,0,907f96037a59:170
24,189,0,907f96037a59:152|TARGET:37
25,314,0,907f96037a59:157|TARGET:157
26,281,0,907f96037a59:139|TARGET:142
27,185,0,907f96037a59:94|TARGET:91
28,318,0,907f96037a59:159|TARGET:159
29,338,0,907f96037a59:167|TARGET:171
30,170,0,907f96037a59:87|TARGET:83
1 second ok failed instances
2 0 360 0 907f96037a59:180|TARGET:180
3 1 187 0 907f96037a59:93|TARGET:94
4 2 271 0 907f96037a59:136|TARGET:135
5 3 368 0 907f96037a59:184|TARGET:184
6 4 240 0 907f96037a59:120|TARGET:120
7 5 182 0 907f96037a59:91|TARGET:91
8 6 358 0 907f96037a59:179|TARGET:179
9 7 199 0 907f96037a59:156|TARGET:43
10 8 158 0 907f96037a59:158
11 9 249 0 907f96037a59:249
12 10 245 0 907f96037a59:245
13 11 129 0 907f96037a59:129
14 12 291 0 907f96037a59:291
15 13 291 0 907f96037a59:291
16 14 127 0 907f96037a59:127
17 15 224 0 907f96037a59:224
18 16 295 0 907f96037a59:295
19 17 139 0 907f96037a59:139
20 18 238 0 907f96037a59:238
21 19 299 0 907f96037a59:299
22 20 165 0 907f96037a59:165
23 21 207 0 907f96037a59:207
24 22 193 0 907f96037a59:193
25 23 170 0 907f96037a59:170
26 24 189 0 907f96037a59:152|TARGET:37
27 25 314 0 907f96037a59:157|TARGET:157
28 26 281 0 907f96037a59:139|TARGET:142
29 27 185 0 907f96037a59:94|TARGET:91
30 28 318 0 907f96037a59:159|TARGET:159
31 29 338 0 907f96037a59:167|TARGET:171
32 30 170 0 907f96037a59:87|TARGET:83
+84
View File
@@ -0,0 +1,84 @@
# Level 4: failover series, rolling update under load, public load.py
# 2026-10-19T15:20:55Z, commit 2708732 fix: LB и на [::1] — localhost на Windows стоил 2 с на запрос; сдвиг цели failover
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe bench/failover_series.py --output evidence/failover
== stop mtusi-lab3-api-1 ...
{"failed_after_stop": 0, "last_failure_after_stop_s": 0.0, "target_last_served_after_stop_s": 1.283, "rejoin_serving_s": 3.35}
== stop mtusi-lab3-api-2 ...
{"failed_after_stop": 0, "last_failure_after_stop_s": 0.0, "target_last_served_after_stop_s": 1.235, "rejoin_serving_s": 3.07}
== kill mtusi-lab3-api-1 ...
{"failed_after_stop": 3, "last_failure_after_stop_s": 0.247, "target_last_served_after_stop_s": 0.22, "rejoin_serving_s": 2.8}
== kill mtusi-lab3-api-2 ...
{"failed_after_stop": 3, "last_failure_after_stop_s": 0.214, "target_last_served_after_stop_s": 0.174, "rejoin_serving_s": 3.64}
== pause mtusi-lab3-api-1 ...
{"failed_after_stop": 4, "last_failure_after_stop_s": 3.204, "target_last_served_after_stop_s": 0.148, "rejoin_serving_s": 2.31}
== pause mtusi-lab3-api-2 ...
{"failed_after_stop": 4, "last_failure_after_stop_s": 3.12, "target_last_served_after_stop_s": 0.098, "rejoin_serving_s": 1.27}
| Режим | Реплика | Запросов во время отказа | Ошибок после stop | ok-RPS до / во время / после | Последняя ошибка, с | Цель ушла из ротации, с | Возврат в LB / обслуживает, с | Потери / дубли |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| stop | mtusi-lab3-api-1 | 3568 | 0 | 294.6 / 222.5 / 249.7 | 0.0 | 1.283 | 3.25 / 3.35 | 0 / 0 |
| stop | mtusi-lab3-api-2 | 3565 | 0 | 259.2 / 225.7 / 275.1 | 0.0 | 1.235 | 1.35 / 3.07 | 0 / 0 |
| kill | mtusi-lab3-api-1 | 2734 | 3 [0] | 270.9 / 214.7 / 245.5 | 0.247 | 0.22 | 1.17 / 2.8 | 0 / 0 |
| kill | mtusi-lab3-api-2 | 2297 | 3 [0] | 225.6 / 179.6 / 240.0 | 0.214 | 0.174 | 2.32 / 3.64 | 0 / 0 |
| pause | mtusi-lab3-api-1 | 1653 | 4 [0] | 271.3 / 135.2 / 244.6 | 3.204 | 0.148 | 2.21 / 2.31 | 0 / 0 |
| pause | mtusi-lab3-api-2 | 1787 | 4 [0] | 253.6 / 147.0 / 253.0 | 3.12 | 0.098 | 1.16 / 1.27 | 0 / 0 |
[exit 0, 179.0 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe bench/rolling_under_load.py --output evidence/rolling-update
{
"script_exit_code": 0,
"update_seconds": 57.1,
"before": {
"mtusi-lab3-api-1": "907f96037a59 revision=dev",
"mtusi-lab3-api-2": "c3f64421973d revision=dev"
},
"after": {
"mtusi-lab3-api-3": "97c3063d8125 revision=2708732-20261019T152401Z",
"mtusi-lab3-api-4": "7a0d24d4d31c revision=2708732-20261019T152401Z"
},
"history": {
"history_messages": 7119,
"confirmed_messages": 7119,
"lost_confirmed": 0,
"altered_confirmed": 0,
"duplicate_ids": 0,
"duplicate_texts": 0,
"unconfirmed_but_stored": 0
}
}
{"seconds": 57.09, "requests": 11848, "ok": 11848, "failed": 0, "ok_rps": 207.5, "error_ratio": 0.0, "statuses": {"201": 5924, "200": 5924}, "instances": {"907f96037a59": 3080, "c3f64421973d": 4767, "97c3063d8125": 2891, "7a0d24d4d31c": 1110}, "p50_ms": 14.32, "p95_ms": 53.01, "p99_ms": 68.28}
[exit 0, 74.9 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py wait-up 2
2 API replica(s) UP in HAProxy after 0.7 s: ['7a0d24d4d31c', '97c3063d8125']
[exit 0, 1.7 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe tests/load.py --duration 30 --concurrency 4 --output evidence/load.json
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.093,
"concurrency": 4,
"successful_requests": 5247,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 174.357,
"successful_latency_ms": {
"p50": 18.617,
"p95": 50.067,
"p99": 65.226
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
[exit 0, 30.8 s]
$ docker compose ps -a --format table {{.Service}} {{.Status}}
SERVICE STATUS
api Up About a minute (healthy)
api Up About a minute (healthy)
lb Up 9 minutes (healthy)
migrate Exited (0) 4 minutes ago
postgres Up 5 minutes (healthy)
valkey Up 6 minutes (healthy)
[exit 0, 0.3 s]
+17
View File
@@ -0,0 +1,17 @@
{
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.093,
"concurrency": 4,
"successful_requests": 5247,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 174.357,
"successful_latency_ms": {
"p50": 18.617,
"p95": 50.067,
"p99": 65.226
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+17
View File
@@ -0,0 +1,17 @@
# make check, make validate (venv), make test
# 2026-10-19T15:11:33Z, commit 2708732 fix: LB и на [::1] — localhost на Windows стоил 2 с на запрос; сдвиг цели failover
$ make check
python scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
[exit 0, 0.6 s]
$ make validate PYTHON=.venv/Scripts/python.exe
.venv/Scripts/python.exe scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
[exit 0, 1.9 s]
$ make test
python tests/smoke.py --base-url "http://localhost:8080"
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
[exit 0, 1.2 s]
+108
View File
@@ -0,0 +1,108 @@
$ sh scripts/rolling-update.sh (exit 0)
rolling-update: 2 replica(s) to replace, new revision 2708732-20261019T152401Z
#1 [internal] load local bake definitions
#1 reading from stdin 1.38kB 0.1s done
#1 DONE 0.1s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 2.52kB 0.0s done
#2 DONE 0.0s
#3 resolve image config for docker-image://docker.io/docker/dockerfile:1
#3 DONE 4.9s
#4 docker-image://docker.io/docker/dockerfile:1@sha256:4edf897a3ffa55b89f906fc8cc78afdb3f1834cc9c7083565e611a8a7d5fe99e
#4 resolve docker.io/docker/dockerfile:1@sha256:4edf897a3ffa55b89f906fc8cc78afdb3f1834cc9c7083565e611a8a7d5fe99e 0.1s done
#4 CACHED
#5 [internal] load metadata for docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641
#5 DONE 3.3s
#6 [internal] load .dockerignore
#6 transferring context: 196B done
#6 DONE 0.0s
#7 [internal] load build context
#7 transferring context: 1.11kB 0.0s done
#7 DONE 0.1s
#8 [1/8] FROM docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641
#8 resolve docker.io/library/python:3.13.16-slim-bookworm@sha256:a1165e272e578941b84abc79e4ab38a0305cd12803a5c4247979ac7655f4d641 0.1s done
#8 DONE 0.1s
#9 [6/8] COPY src/ ./src/
#9 CACHED
#10 [7/8] COPY migrations/ ./migrations/
#10 CACHED
#11 [2/8] WORKDIR /app
#11 CACHED
#12 [4/8] RUN pip install --root-user-action=ignore --require-hashes --no-deps -r requirements.txt
#12 CACHED
#13 [3/8] COPY requirements.txt ./
#13 CACHED
#14 [5/8] RUN groupadd --system --gid 10001 app && useradd --system --uid 10001 --gid app --no-create-home --shell /usr/sbin/nologin app
#14 CACHED
#15 [8/8] COPY contracts/openapi.json ./contracts/openapi.json
#15 CACHED
#16 exporting to image
#16 exporting layers 0.0s done
#16 exporting manifest sha256:42a5367c0c23b0291d73d5a6e1b863c547c88f1acc80d90c707850b79e894d33
#16 exporting manifest sha256:42a5367c0c23b0291d73d5a6e1b863c547c88f1acc80d90c707850b79e894d33 0.0s done
#16 exporting config sha256:45ff09fc4f0a55a942e45f56fcdf19071c8c76fd5a4a0243912407c23439bbe7 0.1s done
#16 exporting attestation manifest sha256:92b52b37c33ce3d4c52d0aef43d53bae0fe620968c5277e0a0905e6f1443822c 0.1s done
#16 exporting manifest list sha256:ccb3adf721f08abed4b0678ad0c49e587a0871f560ad4dfc686957846825cf9a
#16 exporting manifest list sha256:ccb3adf721f08abed4b0678ad0c49e587a0871f560ad4dfc686957846825cf9a 0.1s done
#16 naming to docker.io/library/mtusi-messenger:lab3 0.0s done
#16 unpacking to docker.io/library/mtusi-messenger:lab3
#16 unpacking to docker.io/library/mtusi-messenger:lab3 0.1s done
#16 DONE 0.6s
#17 resolving provenance for metadata file
#17 DONE 0.0s
migrate: 0001_init: already applied, skipped
migrate: 0002_idempotency_keys: already applied, skipped
migrate: 0003_post_message_function: already applied, skipped
migrate: done, 3 migration file(s) checked
3 API replica(s) UP in HAProxy after 1.0 s: ['907f96037a59', '97c3063d8125', 'c3f64421973d']
2 API replica(s) UP in HAProxy after 1.0 s: ['97c3063d8125', 'c3f64421973d']
rolling-update: replaced mtusi-lab3-api-1
3 API replica(s) UP in HAProxy after 1.4 s: ['7a0d24d4d31c', '97c3063d8125', 'c3f64421973d']
2 API replica(s) UP in HAProxy after 0.8 s: ['7a0d24d4d31c', '97c3063d8125']
rolling-update: replaced mtusi-lab3-api-2
/mtusi-lab3-api-3 revision=2708732-20261019T152401Z running
/mtusi-lab3-api-4 revision=2708732-20261019T152401Z running
Image mtusi-messenger:lab3 Building
Image mtusi-messenger:lab3 Built
Container mtusi-lab3-migrate-run-5a1b1047bd22 Creating
Container mtusi-lab3-migrate-run-5a1b1047bd22 Created
Container mtusi-lab3-api-1 Running
Container mtusi-lab3-api-2 Running
Container mtusi-lab3-api-3 Creating
Container mtusi-lab3-api-3 Created
Container mtusi-lab3-api-3 Starting
Container mtusi-lab3-api-3 Started
Container mtusi-lab3-api-3 Waiting
Container mtusi-lab3-api-2 Waiting
Container mtusi-lab3-api-1 Waiting
Container mtusi-lab3-api-3 Healthy
Container mtusi-lab3-api-2 Healthy
Container mtusi-lab3-api-1 Healthy
Container mtusi-lab3-api-2 Running
Container mtusi-lab3-api-3 Running
Container mtusi-lab3-api-4 Creating
Container mtusi-lab3-api-4 Created
Container mtusi-lab3-api-4 Starting
Container mtusi-lab3-api-4 Started
Container mtusi-lab3-api-4 Waiting
Container mtusi-lab3-api-3 Waiting
Container mtusi-lab3-api-2 Waiting
Container mtusi-lab3-api-4 Healthy
Container mtusi-lab3-api-3 Healthy
Container mtusi-lab3-api-2 Healthy
+65
View File
@@ -0,0 +1,65 @@
{
"script_exit_code": 0,
"update_seconds": 57.1,
"before": {
"mtusi-lab3-api-1": "907f96037a59 revision=dev",
"mtusi-lab3-api-2": "c3f64421973d revision=dev"
},
"after": {
"mtusi-lab3-api-3": "97c3063d8125 revision=2708732-20261019T152401Z",
"mtusi-lab3-api-4": "7a0d24d4d31c revision=2708732-20261019T152401Z"
},
"during_update": {
"seconds": 57.09,
"requests": 11848,
"ok": 11848,
"failed": 0,
"ok_rps": 207.5,
"error_ratio": 0.0,
"statuses": {
"201": 5924,
"200": 5924
},
"instances": {
"907f96037a59": 3080,
"c3f64421973d": 4767,
"97c3063d8125": 2891,
"7a0d24d4d31c": 1110
},
"p50_ms": 14.32,
"p95_ms": 53.01,
"p99_ms": 68.28
},
"whole_run": {
"seconds": 67.79,
"requests": 14238,
"ok": 14238,
"failed": 0,
"ok_rps": 210.0,
"error_ratio": 0.0,
"statuses": {
"201": 7119,
"200": 7119
},
"instances": {
"907f96037a59": 3754,
"c3f64421973d": 5445,
"97c3063d8125": 3410,
"7a0d24d4d31c": 1629
},
"p50_ms": 13.97,
"p95_ms": 52.62,
"p99_ms": 67.65
},
"client_retries_same_key": 0,
"history": {
"history_messages": 7119,
"confirmed_messages": 7119,
"lost_confirmed": 0,
"altered_confirmed": 0,
"duplicate_ids": 0,
"duplicate_texts": 0,
"unconfirmed_but_stored": 0
},
"workload": "4 closed-loop workers: POST (Idempotency-Key) + GET page via 127.0.0.1:8080"
}
+69
View File
@@ -0,0 +1,69 @@
second,ok,failed,instances
0,324,0,907f96037a59:162|c3f64421973d:162
1,218,0,907f96037a59:108|c3f64421973d:110
2,188,0,907f96037a59:95|c3f64421973d:93
3,313,0,907f96037a59:156|c3f64421973d:157
4,309,0,907f96037a59:153|c3f64421973d:156
5,132,0,907f96037a59:67|c3f64421973d:65
6,224,0,907f96037a59:113|c3f64421973d:111
7,250,0,907f96037a59:125|c3f64421973d:125
8,176,0,907f96037a59:86|c3f64421973d:90
9,232,0,907f96037a59:118|c3f64421973d:114
10,337,0,907f96037a59:168|c3f64421973d:169
11,181,0,907f96037a59:89|c3f64421973d:92
12,227,0,907f96037a59:115|c3f64421973d:112
13,290,0,907f96037a59:145|c3f64421973d:145
14,277,0,907f96037a59:138|c3f64421973d:139
15,204,0,907f96037a59:103|c3f64421973d:101
16,302,0,907f96037a59:151|c3f64421973d:151
17,203,0,907f96037a59:100|c3f64421973d:103
18,108,0,907f96037a59:54|c3f64421973d:54
19,216,0,907f96037a59:110|c3f64421973d:106
20,252,0,907f96037a59:125|c3f64421973d:127
21,145,0,907f96037a59:72|c3f64421973d:73
22,214,0,907f96037a59:108|c3f64421973d:106
23,231,0,907f96037a59:115|c3f64421973d:116
24,187,0,907f96037a59:92|c3f64421973d:95
25,138,0,907f96037a59:69|c3f64421973d:69
26,237,0,907f96037a59:120|c3f64421973d:117
27,249,0,907f96037a59:125|c3f64421973d:124
28,188,0,907f96037a59:77|97c3063d8125:34|c3f64421973d:77
29,334,0,907f96037a59:111|97c3063d8125:111|c3f64421973d:112
30,289,0,907f96037a59:96|97c3063d8125:97|c3f64421973d:96
31,139,0,907f96037a59:45|97c3063d8125:47|c3f64421973d:47
32,232,0,907f96037a59:79|97c3063d8125:77|c3f64421973d:76
33,240,0,907f96037a59:80|97c3063d8125:79|c3f64421973d:81
34,170,0,907f96037a59:55|97c3063d8125:58|c3f64421973d:57
35,260,0,907f96037a59:29|97c3063d8125:116|c3f64421973d:115
36,334,0,97c3063d8125:166|c3f64421973d:168
37,203,0,97c3063d8125:103|c3f64421973d:100
38,176,0,97c3063d8125:87|c3f64421973d:89
39,232,0,97c3063d8125:116|c3f64421973d:116
40,182,0,97c3063d8125:92|c3f64421973d:90
41,88,0,97c3063d8125:44|c3f64421973d:44
42,202,0,97c3063d8125:100|c3f64421973d:102
43,225,0,97c3063d8125:112|c3f64421973d:113
44,171,0,97c3063d8125:87|c3f64421973d:84
45,197,0,97c3063d8125:97|c3f64421973d:100
46,342,0,7a0d24d4d31c:9|97c3063d8125:167|c3f64421973d:166
47,195,0,7a0d24d4d31c:65|97c3063d8125:65|c3f64421973d:65
48,134,0,7a0d24d4d31c:45|97c3063d8125:45|c3f64421973d:44
49,168,0,7a0d24d4d31c:54|97c3063d8125:56|c3f64421973d:58
50,179,0,7a0d24d4d31c:61|97c3063d8125:59|c3f64421973d:59
51,148,0,7a0d24d4d31c:50|97c3063d8125:50|c3f64421973d:48
52,128,0,7a0d24d4d31c:41|97c3063d8125:43|c3f64421973d:44
53,124,0,7a0d24d4d31c:42|97c3063d8125:41|c3f64421973d:41
54,247,0,7a0d24d4d31c:89|97c3063d8125:87|c3f64421973d:71
55,156,0,7a0d24d4d31c:77|97c3063d8125:79
56,159,0,7a0d24d4d31c:78|97c3063d8125:81
57,251,0,7a0d24d4d31c:128|97c3063d8125:123
58,179,0,7a0d24d4d31c:89|97c3063d8125:90
59,136,0,7a0d24d4d31c:67|97c3063d8125:69
60,215,0,7a0d24d4d31c:108|97c3063d8125:107
61,197,0,7a0d24d4d31c:99|97c3063d8125:98
62,183,0,7a0d24d4d31c:91|97c3063d8125:92
63,177,0,7a0d24d4d31c:89|97c3063d8125:88
64,320,0,7a0d24d4d31c:160|97c3063d8125:160
65,209,0,7a0d24d4d31c:104|97c3063d8125:105
66,140,0,7a0d24d4d31c:69|97c3063d8125:71
67,25,0,7a0d24d4d31c:14|97c3063d8125:11
1 second ok failed instances
2 0 324 0 907f96037a59:162|c3f64421973d:162
3 1 218 0 907f96037a59:108|c3f64421973d:110
4 2 188 0 907f96037a59:95|c3f64421973d:93
5 3 313 0 907f96037a59:156|c3f64421973d:157
6 4 309 0 907f96037a59:153|c3f64421973d:156
7 5 132 0 907f96037a59:67|c3f64421973d:65
8 6 224 0 907f96037a59:113|c3f64421973d:111
9 7 250 0 907f96037a59:125|c3f64421973d:125
10 8 176 0 907f96037a59:86|c3f64421973d:90
11 9 232 0 907f96037a59:118|c3f64421973d:114
12 10 337 0 907f96037a59:168|c3f64421973d:169
13 11 181 0 907f96037a59:89|c3f64421973d:92
14 12 227 0 907f96037a59:115|c3f64421973d:112
15 13 290 0 907f96037a59:145|c3f64421973d:145
16 14 277 0 907f96037a59:138|c3f64421973d:139
17 15 204 0 907f96037a59:103|c3f64421973d:101
18 16 302 0 907f96037a59:151|c3f64421973d:151
19 17 203 0 907f96037a59:100|c3f64421973d:103
20 18 108 0 907f96037a59:54|c3f64421973d:54
21 19 216 0 907f96037a59:110|c3f64421973d:106
22 20 252 0 907f96037a59:125|c3f64421973d:127
23 21 145 0 907f96037a59:72|c3f64421973d:73
24 22 214 0 907f96037a59:108|c3f64421973d:106
25 23 231 0 907f96037a59:115|c3f64421973d:116
26 24 187 0 907f96037a59:92|c3f64421973d:95
27 25 138 0 907f96037a59:69|c3f64421973d:69
28 26 237 0 907f96037a59:120|c3f64421973d:117
29 27 249 0 907f96037a59:125|c3f64421973d:124
30 28 188 0 907f96037a59:77|97c3063d8125:34|c3f64421973d:77
31 29 334 0 907f96037a59:111|97c3063d8125:111|c3f64421973d:112
32 30 289 0 907f96037a59:96|97c3063d8125:97|c3f64421973d:96
33 31 139 0 907f96037a59:45|97c3063d8125:47|c3f64421973d:47
34 32 232 0 907f96037a59:79|97c3063d8125:77|c3f64421973d:76
35 33 240 0 907f96037a59:80|97c3063d8125:79|c3f64421973d:81
36 34 170 0 907f96037a59:55|97c3063d8125:58|c3f64421973d:57
37 35 260 0 907f96037a59:29|97c3063d8125:116|c3f64421973d:115
38 36 334 0 97c3063d8125:166|c3f64421973d:168
39 37 203 0 97c3063d8125:103|c3f64421973d:100
40 38 176 0 97c3063d8125:87|c3f64421973d:89
41 39 232 0 97c3063d8125:116|c3f64421973d:116
42 40 182 0 97c3063d8125:92|c3f64421973d:90
43 41 88 0 97c3063d8125:44|c3f64421973d:44
44 42 202 0 97c3063d8125:100|c3f64421973d:102
45 43 225 0 97c3063d8125:112|c3f64421973d:113
46 44 171 0 97c3063d8125:87|c3f64421973d:84
47 45 197 0 97c3063d8125:97|c3f64421973d:100
48 46 342 0 7a0d24d4d31c:9|97c3063d8125:167|c3f64421973d:166
49 47 195 0 7a0d24d4d31c:65|97c3063d8125:65|c3f64421973d:65
50 48 134 0 7a0d24d4d31c:45|97c3063d8125:45|c3f64421973d:44
51 49 168 0 7a0d24d4d31c:54|97c3063d8125:56|c3f64421973d:58
52 50 179 0 7a0d24d4d31c:61|97c3063d8125:59|c3f64421973d:59
53 51 148 0 7a0d24d4d31c:50|97c3063d8125:50|c3f64421973d:48
54 52 128 0 7a0d24d4d31c:41|97c3063d8125:43|c3f64421973d:44
55 53 124 0 7a0d24d4d31c:42|97c3063d8125:41|c3f64421973d:41
56 54 247 0 7a0d24d4d31c:89|97c3063d8125:87|c3f64421973d:71
57 55 156 0 7a0d24d4d31c:77|97c3063d8125:79
58 56 159 0 7a0d24d4d31c:78|97c3063d8125:81
59 57 251 0 7a0d24d4d31c:128|97c3063d8125:123
60 58 179 0 7a0d24d4d31c:89|97c3063d8125:90
61 59 136 0 7a0d24d4d31c:67|97c3063d8125:69
62 60 215 0 7a0d24d4d31c:108|97c3063d8125:107
63 61 197 0 7a0d24d4d31c:99|97c3063d8125:98
64 62 183 0 7a0d24d4d31c:91|97c3063d8125:92
65 63 177 0 7a0d24d4d31c:89|97c3063d8125:88
66 64 320 0 7a0d24d4d31c:160|97c3063d8125:160
67 65 209 0 7a0d24d4d31c:104|97c3063d8125:105
68 66 140 0 7a0d24d4d31c:69|97c3063d8125:71
69 67 25 0 7a0d24d4d31c:14|97c3063d8125:11
+214
View File
@@ -0,0 +1,214 @@
# Own tests: unit + stand (pytest -v); failures restored in finally
# 2026-10-19T15:12:54Z, commit 2708732 fix: LB и на [::1] — localhost на Windows стоил 2 с на запрос; сдвиг цели failover
$ python -m pytest student-tests -v
============================= test session starts =============================
platform win32 -- Python 3.14.6, pytest-9.1.1, pluggy-1.7.0 -- C:\Users\wfa22\Desktop\Высоконагруженные приложения\mtusi2026lab3-clean\.venv\Scripts\python.exe
rootdir: C:\Users\wfa22\Desktop\Высоконагруженные приложения\mtusi2026lab3-clean
plugins: anyio-4.15.1
collecting ... collected 201 items
student-tests/test_api.py::test_register_never_returns_password_and_login_returns_bearer PASSED [ 0%]
student-tests/test_api.py::test_duplicate_username_is_409 PASSED [ 0%]
student-tests/test_api.py::test_bad_password_on_registration_is_400[short-11ch] PASSED [ 1%]
student-tests/test_api.py::test_bad_password_on_registration_is_400[ ] PASSED [ 1%]
student-tests/test_api.py::test_bad_password_on_registration_is_400[ \t \n \t \n \t \n ] PASSED [ 2%]
student-tests/test_api.py::test_bad_password_on_registration_is_400[\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f] PASSED [ 2%]
student-tests/test_api.py::test_edge_passwords_register_and_login[\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f] PASSED [ 3%]
student-tests/test_api.py::test_edge_passwords_register_and_login[with:colons:inside] PASSED [ 3%]
student-tests/test_api.py::test_edge_passwords_register_and_login[Twelve chars] PASSED [ 4%]
student-tests/test_api.py::test_wrong_password_and_unknown_user_give_the_same_401 PASSED [ 4%]
student-tests/test_api.py::test_malformed_basic_is_401_not_500[Basic \xe9] PASSED [ 5%]
student-tests/test_api.py::test_malformed_basic_is_401_not_500[Basic !!!] PASSED [ 5%]
student-tests/test_api.py::test_malformed_basic_is_401_not_500[Basic bm9fY29sb24=] PASSED [ 6%]
student-tests/test_api.py::test_malformed_basic_is_401_not_500[Basic //46eA==] PASSED [ 6%]
student-tests/test_api.py::test_malformed_basic_is_401_not_500[Bearer aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 7%]
student-tests/test_api.py::test_malformed_basic_is_401_not_500[] PASSED [ 7%]
student-tests/test_api.py::test_logout_revokes_only_this_token PASSED [ 8%]
student-tests/test_api.py::test_x_user_id_basic_and_garbage_do_not_authenticate PASSED [ 8%]
student-tests/test_api.py::test_malformed_json_is_400[{] PASSED [ 9%]
student-tests/test_api.py::test_malformed_json_is_400[] PASSED [ 9%]
student-tests/test_api.py::test_malformed_json_is_400[[]] PASSED [ 10%]
student-tests/test_api.py::test_malformed_json_is_400[null] PASSED [ 10%]
student-tests/test_api.py::test_malformed_json_is_400[{"username": "abc_def", "display_name": NaN, "password": "Twelve-chars"}] PASSED [ 11%]
student-tests/test_api.py::test_malformed_json_is_400[{"username": "abc_def", "display_name": "x", "password": "Twelve-chars"}{"a": 1}] PASSED [ 11%]
student-tests/test_api.py::test_malformed_json_is_400[{"username": "abc_def", "display_name": "\ud800", "password": "Twelve-chars"}] PASSED [ 12%]
student-tests/test_api.py::test_invalid_registration_fields_are_400[body0] PASSED [ 12%]
student-tests/test_api.py::test_invalid_registration_fields_are_400[body1] PASSED [ 13%]
student-tests/test_api.py::test_invalid_registration_fields_are_400[body2] PASSED [ 13%]
student-tests/test_api.py::test_invalid_registration_fields_are_400[body3] PASSED [ 14%]
student-tests/test_api.py::test_invalid_registration_fields_are_400[body4] PASSED [ 14%]
student-tests/test_api.py::test_invalid_registration_fields_are_400[body5] PASSED [ 15%]
student-tests/test_api.py::test_unicode_limits_are_code_points_and_nul_is_stored PASSED [ 15%]
student-tests/test_api.py::test_oversized_body_and_wrong_content_type PASSED [ 16%]
student-tests/test_api.py::test_chat_validation PASSED [ 16%]
student-tests/test_api.py::test_foreign_chat_is_indistinguishable_from_missing PASSED [ 17%]
student-tests/test_api.py::test_chat_list_order_pagination_and_isolation PASSED [ 17%]
student-tests/test_api.py::test_history_pages_without_loss_or_duplicates PASSED [ 18%]
student-tests/test_api.py::test_bad_limit_or_cursor_is_400[limit=0] PASSED [ 18%]
student-tests/test_api.py::test_bad_limit_or_cursor_is_400[limit=101] PASSED [ 19%]
student-tests/test_api.py::test_bad_limit_or_cursor_is_400[limit=abc] PASSED [ 19%]
student-tests/test_api.py::test_bad_limit_or_cursor_is_400[limit=1&limit=2] PASSED [ 20%]
student-tests/test_api.py::test_bad_limit_or_cursor_is_400[cursor=] PASSED [ 20%]
student-tests/test_api.py::test_bad_limit_or_cursor_is_400[cursor=garbage] PASSED [ 21%]
student-tests/test_api.py::test_bad_limit_or_cursor_is_400[cursor=djF8bXx4fDF8Mg] PASSED [ 21%]
student-tests/test_api.py::test_bad_limit_or_cursor_is_400[cursor=djF8bXx4fDk5OTk5OTk5OTk5OTk5OTk5OTk5fHk=] PASSED [ 22%]
student-tests/test_api.py::test_cursor_bound_to_chat PASSED [ 22%]
student-tests/test_api.py::test_read_after_write PASSED [ 23%]
student-tests/test_api.py::test_malformed_cursor_values_are_400_on_both_lists PASSED [ 23%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[\xff] PASSED [ 24%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"first","TEXT":"second"}] PASSED [ 24%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"a","text":"b"}] PASSED [ 25%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":123}] PASSED [ 25%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"valid"}}] PASSED [ 26%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"valid"}]] PASSED [ 26%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"x"} {}] PASSED [ 27%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"x","sender_id":"11111111-1111-4111-8111-111111111111"}] PASSED [ 27%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"x","attachment_ids":null}] PASSED [ 28%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":"x","attachment_ids":["nope"]}] PASSED [ 28%]
student-tests/test_api.py::test_bad_message_body_is_400_and_nothing_is_stored[{"text":" \n\t "}] PASSED [ 29%]
student-tests/test_api.py::test_ff_byte_body_on_registration_is_contract_error PASSED [ 29%]
student-tests/test_api.py::test_text_is_stored_verbatim_and_length_counts_raw_text PASSED [ 30%]
student-tests/test_api.py::test_unknown_attachment_ids_are_404 PASSED [ 30%]
student-tests/test_api.py::test_member_ids_are_compared_as_uuids_not_strings PASSED [ 31%]
student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[Bearer \xe9] PASSED [ 31%]
student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[Bearer .] PASSED [ 32%]
student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[Bearer a b] PASSED [ 32%]
student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[Bearer \xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9] PASSED [ 33%]
student-tests/test_api.py::test_non_ascii_or_garbage_bearer_is_401_not_500[bearer\taaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 33%]
student-tests/test_api.py::test_login_with_a_body_is_rejected PASSED [ 34%]
student-tests/test_api.py::test_password_range_is_12_to_128_code_points PASSED [ 34%]
student-tests/test_api.py::test_every_api_response_carries_instance_and_request_id PASSED [ 35%]
student-tests/test_api.py::test_unicode_whitespace_only_strings_are_400[\xa0] PASSED [ 35%]
student-tests/test_api.py::test_unicode_whitespace_only_strings_are_400[\u2003\u2003] PASSED [ 36%]
student-tests/test_api.py::test_unicode_whitespace_only_strings_are_400[ \xa0\t\u3000] PASSED [ 36%]
student-tests/test_api.py::test_odd_registration_bodies_are_400_and_the_api_survives[{"username":"num_probe","display_name":"x","password":1e1000}] PASSED [ 37%]
student-tests/test_api.py::test_odd_registration_bodies_are_400_and_the_api_survives[{"username":"ff_probe","display_name":"x","password":"12345678901\xff"}] PASSED [ 37%]
student-tests/test_api.py::test_odd_registration_bodies_are_400_and_the_api_survives[{"username":"num_probe","display_name":1,"password":"Twelve-chars-ok"}] PASSED [ 38%]
student-tests/test_api.py::test_more_than_100_members_is_400 PASSED [ 38%]
student-tests/test_api.py::test_cursor_of_one_list_is_rejected_by_the_other PASSED [ 39%]
student-tests/test_balancing.py::test_one_session_is_served_by_every_real_replica PASSED [ 39%]
student-tests/test_balancing.py::test_write_on_one_replica_is_read_on_the_other PASSED [ 40%]
student-tests/test_balancing.py::test_lb_stops_routing_to_a_replica_that_is_alive_but_not_ready[0-pg_hba] PASSED [ 40%]
student-tests/test_balancing.py::test_lb_stops_routing_to_a_replica_that_is_alive_but_not_ready[0-network] PASSED [ 41%]
student-tests/test_balancing.py::test_lb_stops_routing_to_a_replica_that_is_alive_but_not_ready[1-pg_hba] PASSED [ 41%]
student-tests/test_balancing.py::test_lb_stops_routing_to_a_replica_that_is_alive_but_not_ready[1-network] PASSED [ 42%]
student-tests/test_balancing.py::test_graceful_stop_of_each_replica_loses_no_request[0] PASSED [ 42%]
student-tests/test_balancing.py::test_graceful_stop_of_each_replica_loses_no_request[1] PASSED [ 43%]
student-tests/test_balancing.py::test_killed_replica_bounded_errors_recovery_and_no_duplicates[0] PASSED [ 43%]
student-tests/test_balancing.py::test_killed_replica_bounded_errors_recovery_and_no_duplicates[1] PASSED [ 44%]
student-tests/test_balancing.py::test_graceful_shutdown_order_in_logs PASSED [ 44%]
student-tests/test_balancing.py::test_full_restart_lb_rediscovers_all_replicas PASSED [ 45%]
student-tests/test_balancing.py::test_scale_up_and_down_is_picked_up_without_errors PASSED [ 45%]
student-tests/test_balancing.py::test_compose_policy_limits_ports_secrets_images PASSED [ 46%]
student-tests/test_balancing.py::test_running_containers_have_limits_and_no_bypass_ports PASSED [ 46%]
student-tests/test_balancing.py::test_migrations_ran_once_by_the_migrate_service PASSED [ 47%]
student-tests/test_db.py::test_registration_race_exactly_one_201_and_seven_409 PASSED [ 47%]
student-tests/test_db.py::test_create_chat_rolls_back_after_the_chat_row_was_inserted PASSED [ 48%]
student-tests/test_db.py::test_migrations_rerun_is_a_noop PASSED [ 48%]
student-tests/test_db.py::test_100_messages_from_10_clients_keep_order_and_cursor_walk PASSED [ 49%]
student-tests/test_db.py::test_secrets_are_not_stored_in_plain_text PASSED [ 49%]
student-tests/test_explain.py::test_late_page_uses_index_range_not_filter PASSED [ 50%]
student-tests/test_failures.py::test_valkey_failure[stop] PASSED [ 50%]
student-tests/test_failures.py::test_valkey_failure[pause] PASSED [ 51%]
student-tests/test_failures.py::test_postgres_failure[stop] PASSED [ 51%]
student-tests/test_failures.py::test_postgres_failure[pause] PASSED [ 52%]
student-tests/test_failures.py::test_locked_table_gives_bounded_503 PASSED [ 52%]
student-tests/test_failures.py::test_logout_during_valkey_outage_is_never_lost PASSED [ 53%]
student-tests/test_idempotency.py::test_replay_returns_the_same_201_and_message_from_every_replica PASSED [ 53%]
student-tests/test_idempotency.py::test_other_body_with_the_same_key_is_409_and_keeps_the_original PASSED [ 54%]
student-tests/test_idempotency.py::test_key_order_and_missing_attachment_ids_are_the_same_body PASSED [ 54%]
student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[] PASSED [ 55%]
student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 55%]
student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[\u043a\u043b\u044e\u0447] PASSED [ 56%]
student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[a b] PASSED [ 56%]
student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[a/b] PASSED [ 57%]
student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[a,b] PASSED [ 57%]
student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[a\tb] PASSED [ 58%]
student-tests/test_idempotency.py::test_missing_or_invalid_key_is_400[*] PASSED [ 58%]
student-tests/test_idempotency.py::test_repeated_key_header_is_400 PASSED [ 59%]
student-tests/test_idempotency.py::test_longest_key_and_every_allowed_character PASSED [ 59%]
student-tests/test_idempotency.py::test_scope_is_user_plus_chat_plus_key PASSED [ 60%]
student-tests/test_idempotency.py::test_outsider_and_revoked_session_never_see_the_stored_result PASSED [ 60%]
student-tests/test_idempotency.py::test_concurrent_same_key_through_different_replicas_creates_exactly_one PASSED [ 61%]
student-tests/test_idempotency.py::test_concurrent_same_key_different_bodies_one_wins PASSED [ 61%]
student-tests/test_idempotency.py::test_failure_after_the_key_insert_rolls_everything_back PASSED [ 62%]
student-tests/test_idempotency.py::test_client_timeout_then_retry_with_the_same_key_gives_the_committed_message PASSED [ 62%]
student-tests/test_idempotency.py::test_lock_timeout_is_a_transient_503_that_is_not_recorded PASSED [ 63%]
student-tests/test_idempotency.py::test_replay_after_the_serving_replica_is_stopped PASSED [ 63%]
student-tests/test_sessions.py::test_short_ttl_expires[valkey] PASSED [ 64%]
student-tests/test_sessions.py::test_short_ttl_expires[postgres] PASSED [ 64%]
student-tests/test_sessions.py::test_restart_does_not_extend_ttl PASSED [ 65%]
student-tests/test_sessions.py::test_postgres_session_store PASSED [ 65%]
student-tests/unit/test_unit.py::test_valid_passwords[Twelve-chars] PASSED [ 66%]
student-tests/unit/test_unit.py::test_valid_passwords[\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f] PASSED [ 66%]
student-tests/unit/test_unit.py::test_valid_passwords[\u043f\u0430\u0440\u043e\u043b\u044c:\u0441:\u0434\u0432\u043e\u0435\u0442\u043e\u0447\u0438\u044f\u043c\u0438] PASSED [ 67%]
student-tests/unit/test_unit.py::test_valid_passwords[\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600\U0001f600] PASSED [ 67%]
student-tests/unit/test_unit.py::test_valid_passwords[a b c d e f g] PASSED [ 68%]
student-tests/unit/test_unit.py::test_invalid_passwords[short-11ch] PASSED [ 68%]
student-tests/unit/test_unit.py::test_invalid_passwords[\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f\u044f] PASSED [ 69%]
student-tests/unit/test_unit.py::test_invalid_passwords[ ] PASSED [ 69%]
student-tests/unit/test_unit.py::test_invalid_passwords[\t\n\u3000\t\n\u3000\t\n\u3000\t\n\u3000\t\n\u3000] PASSED [ 70%]
student-tests/unit/test_unit.py::test_invalid_passwords[\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800\ud800] PASSED [ 70%]
student-tests/unit/test_unit.py::test_invalid_passwords[None] PASSED [ 71%]
student-tests/unit/test_unit.py::test_invalid_passwords[123456789012] PASSED [ 71%]
student-tests/unit/test_unit.py::test_text_may_contain_nul_but_not_lone_surrogates PASSED [ 72%]
student-tests/unit/test_unit.py::test_cursor_roundtrip_and_binding PASSED [ 72%]
student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|253402300800000000|11111111-1111-4111-8111-111111111111] PASSED [ 73%]
student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|9999999999999999999|11111111-1111-4111-8111-111111111111] PASSED [ 73%]
student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|-1|11111111-1111-4111-8111-111111111111] PASSED [ 74%]
student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|1|not-a-uuid] PASSED [ 74%]
student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v1|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|1|11111111-1111-4111-8111-11111111111G] PASSED [ 75%]
student-tests/unit/test_unit.py::test_broken_cursors_are_400_not_500[v2|m|3f2c7a5e-0b7d-4a35-9a43-8d1f0f3b2c11|1|11111111-1111-4111-8111-111111111111] PASSED [ 75%]
student-tests/unit/test_unit.py::test_time_helpers_are_exact PASSED [ 76%]
student-tests/unit/test_unit.py::test_basic_is_parsed_and_password_may_contain_colons PASSED [ 76%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[None] PASSED [ 77%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[] PASSED [ 77%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic] PASSED [ 78%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic \xe9] PASSED [ 78%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic !!!!] PASSED [ 79%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic bm8tY29sb24=] PASSED [ 79%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic //46cHc=] PASSED [ 80%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic OnBhc3N3b3Jk] PASSED [ 80%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic YWxpY2U6] PASSED [ 81%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic 0JDQu9C40YHQsDpwdw==] PASSED [ 81%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Bearer aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 82%]
student-tests/unit/test_unit.py::test_bad_basic_is_401_with_basic_challenge[Basic AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA] PASSED [ 82%]
student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[None] PASSED [ 83%]
student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[] PASSED [ 83%]
student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Bearer] PASSED [ 84%]
student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Bearer short] PASSED [ 84%]
student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Bearer aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!] PASSED [ 85%]
student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Basic YWxpY2U6cGFzc3dvcmQtMTIz] PASSED [ 85%]
student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Token aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] PASSED [ 86%]
student-tests/unit/test_unit.py::test_bad_bearer_is_401_with_bearer_challenge[Bearer \xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9\xe9] PASSED [ 86%]
student-tests/unit/test_unit.py::test_bearer_ok PASSED [ 87%]
student-tests/unit/test_unit.py::test_argon2id_hash_and_verify PASSED [ 87%]
student-tests/unit/test_unit.py::test_settings_read_secret_files_and_contract_names PASSED [ 88%]
student-tests/unit/test_unit.py::test_valid_idempotency_keys[a] PASSED [ 88%]
student-tests/unit/test_unit.py::test_valid_idempotency_keys[kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk] PASSED [ 89%]
student-tests/unit/test_unit.py::test_valid_idempotency_keys[A-Za-z0-9._:-] PASSED [ 89%]
student-tests/unit/test_unit.py::test_valid_idempotency_keys[2026-10-16T10:00:00.1] PASSED [ 90%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[None] PASSED [ 90%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[] PASSED [ 91%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk] PASSED [ 91%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[\u043a\u043b\u044e\u0447] PASSED [ 92%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a b] PASSED [ 92%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a/b] PASSED [ 93%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a,b] PASSED [ 93%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a\tb] PASSED [ 94%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[a\x00b] PASSED [ 94%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[*] PASSED [ 95%]
student-tests/unit/test_unit.py::test_invalid_idempotency_keys[\xe9] PASSED [ 95%]
student-tests/unit/test_unit.py::test_duplicate_keys_and_garbage_json_are_rejected[{"text":"a","text":"b"}] PASSED [ 96%]
student-tests/unit/test_unit.py::test_duplicate_keys_and_garbage_json_are_rejected[{"a":{"x":1,"x":2}}] PASSED [ 96%]
student-tests/unit/test_unit.py::test_duplicate_keys_and_garbage_json_are_rejected[\xff] PASSED [ 97%]
student-tests/unit/test_unit.py::test_duplicate_keys_and_garbage_json_are_rejected[{"text":"x"} {}] PASSED [ 97%]
student-tests/unit/test_unit.py::test_attachment_ids_absent_equals_empty_and_is_validated PASSED [ 98%]
student-tests/unit/test_unit.py::test_request_fingerprint_ignores_key_order_and_default_attachments PASSED [ 98%]
student-tests/unit/test_unit.py::test_instance_id_defaults_to_hostname_and_is_validated PASSED [ 99%]
student-tests/unit/test_unit.py::test_idempotency_ttl_cannot_be_below_24_hours PASSED [ 99%]
student-tests/unit/test_unit.py::test_message_write_path_setting PASSED [100%]
======================= 201 passed in 479.84s (0:07:59) =======================
[exit 0, 481.1 s]
+74
View File
@@ -0,0 +1,74 @@
# make test-failover with scripts/stop-one: which container is really stopped, restore, repeat
# 2026-10-19T15:12:01Z, commit 2708732 fix: LB и на [::1] — localhost на Windows стоил 2 с на запрос; сдвиг цели failover
--- run 1; running API replicas BEFORE:
mtusi-lab3-api-1 host=94b38b941a8d running exit=0 pid=27148 started=2026-10-19T15:11:53.132
mtusi-lab3-api-2 host=a8d9cffd7a6e running exit=0 pid=27091 started=2026-10-19T15:11:52.824
$ make test-failover ACTION_SCRIPT=scripts/stop-one.cmd
python tests/resilience.py --mode failover --action-script "scripts/stop-one.cmd" --base-url "http://localhost:8080"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 131, "failed_attempts": 0, "first_recovery_seconds": 5.902}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
[exit 0, 16.4 s]
AFTER (the stopped replica, left stopped by the hook):
mtusi-lab3-api-1 host=94b38b941a8d exited exit=0 pid=0 started=2026-10-19T15:11:53.132
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py status
api2 172.23.0.2 UP mtusi-lab3-api-2 X-Instance-Id=a8d9cffd7a6e
(other slots: MAINT, no container)
[exit 0, 0.7 s]
$ sh scripts/restore-api.sh
mtusi-lab3-api-1 Up 6 seconds (healthy)
mtusi-lab3-api-2 Up 33 seconds (healthy)
Container mtusi-lab3-api-2 Running
Container mtusi-lab3-api-1 Starting
Container mtusi-lab3-api-1 Started
Container mtusi-lab3-api-2 Waiting
Container mtusi-lab3-api-1 Waiting
Container mtusi-lab3-api-2 Healthy
Container mtusi-lab3-api-1 Healthy
[exit 0, 6.9 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py wait-up 2
2 API replica(s) UP in HAProxy after 0.7 s: ['94b38b941a8d', 'a8d9cffd7a6e']
[exit 0, 1.4 s]
--- run 2; running API replicas BEFORE:
mtusi-lab3-api-1 host=94b38b941a8d running exit=0 pid=28373 started=2026-10-19T15:12:20.518
mtusi-lab3-api-2 host=a8d9cffd7a6e running exit=0 pid=27091 started=2026-10-19T15:11:52.824
$ make test-failover ACTION_SCRIPT=scripts/stop-one.cmd
python tests/resilience.py --mode failover --action-script "scripts/stop-one.cmd" --base-url "http://localhost:8080"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 132, "failed_attempts": 0, "first_recovery_seconds": 5.667}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
[exit 0, 16.3 s]
AFTER (the stopped replica, left stopped by the hook):
mtusi-lab3-api-2 host=a8d9cffd7a6e exited exit=0 pid=0 started=2026-10-19T15:11:52.824
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py status
api1 172.23.0.3 UP mtusi-lab3-api-1 X-Instance-Id=94b38b941a8d
(other slots: MAINT, no container)
[exit 0, 0.9 s]
$ sh scripts/restore-api.sh
mtusi-lab3-api-1 Up 32 seconds (healthy)
mtusi-lab3-api-2 Up 6 seconds (healthy)
Container mtusi-lab3-api-1 Running
Container mtusi-lab3-api-2 Starting
Container mtusi-lab3-api-2 Started
Container mtusi-lab3-api-2 Waiting
Container mtusi-lab3-api-1 Waiting
Container mtusi-lab3-api-2 Healthy
Container mtusi-lab3-api-1 Healthy
[exit 0, 7.0 s]
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py wait-up 2
2 API replica(s) UP in HAProxy after 0.7 s: ['94b38b941a8d', 'a8d9cffd7a6e']
[exit 0, 1.6 s]
Replicas stopped by the public test across runs: ['mtusi-lab3-api-1', 'mtusi-lab3-api-2']
+28
View File
@@ -0,0 +1,28 @@
# make test-persistence: restart of LB, all API replicas, PostgreSQL, Valkey
# 2026-10-19T15:11:37Z, commit 2708732 fix: LB и на [::1] — localhost на Windows стоил 2 с на запрос; сдвиг цели failover
Containers BEFORE:
mtusi-lab3-api-1 host=94b38b941a8d running exit=0 pid=23944 started=2026-10-19T15:10:34.257
mtusi-lab3-api-2 host=a8d9cffd7a6e running exit=0 pid=23887 started=2026-10-19T15:10:33.970
mtusi-lab3-lb-1 host=d903237b9e99 running exit=0 pid=24172 started=2026-10-19T15:10:40.108
mtusi-lab3-migrate-1 host=d39cf5a2265b exited exit=0 pid=0 started=2026-10-19T15:10:31.699
mtusi-lab3-postgres-1 host=3ee559fb5f7f running exit=0 pid=23533 started=2026-10-19T15:10:25.911
mtusi-lab3-valkey-1 host=8594c96d36f9 running exit=0 pid=23621 started=2026-10-19T15:10:26.220
$ make test-persistence ACTION_SCRIPT=scripts/restart.cmd
python tests/resilience.py --mode persistence --action-script "scripts/restart.cmd" --base-url "http://localhost:8080"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
[exit 0, 22.6 s]
Containers AFTER (new pid/started for lb, api-*, postgres, valkey; volumes kept):
mtusi-lab3-api-1 host=94b38b941a8d running exit=0 pid=27148 started=2026-10-19T15:11:53.132
mtusi-lab3-api-2 host=a8d9cffd7a6e running exit=0 pid=27091 started=2026-10-19T15:11:52.824
mtusi-lab3-lb-1 host=d903237b9e99 running exit=0 pid=27452 started=2026-10-19T15:11:59.277
mtusi-lab3-migrate-1 host=d39cf5a2265b exited exit=0 pid=0 started=2026-10-19T15:11:54.449
mtusi-lab3-postgres-1 host=3ee559fb5f7f running exit=0 pid=26819 started=2026-10-19T15:11:44.558
mtusi-lab3-valkey-1 host=8594c96d36f9 running exit=0 pid=26779 started=2026-10-19T15:11:44.461
$ C:\Users\wfa22\AppData\Local\Python\pythoncore-3.14-64\python.exe scripts/lbctl.py wait-up 2
2 API replica(s) UP in HAProxy after 0.6 s: ['94b38b941a8d', 'a8d9cffd7a6e']
[exit 0, 1.4 s]