diff --git a/REPORT.md b/REPORT.md index 55e1c56..a805906 100644 --- a/REPORT.md +++ b/REPORT.md @@ -134,7 +134,7 @@ Argon2id, Basic→Bearer-сессии в общем Redis, чаты, keyset-па | Health/readiness, LB исключает недоступный backend | `python3 scripts/lab3/lab3.py watch` во время failover | HAProxy: `apiN UP → UP 1/2 (L7STS) → DOWN → MAINT (resolution)`; период проверки 500 ms, `fall 2` ⇒ обнаружение ≈1 s ≪ дрейн 3 s | `evidence/level3/failover-*.txt` | | Последовательная остановка каждой реплики под нагрузкой: RPS, ошибки, восстановление | `python3 scripts/lab3/lab3.py failover --replicas all --rate 200 --duration 60` | по каждой реплике: 0 ошибок у клиента и у LB, 0 дублей и 0 потерь, реплика возвращается за ≈3 s; k6 держит 200 req/s без dropped | `evidence/level4/failover-sweep.json`, `.md` | | Лимиты CPU/RAM, отсутствие портов API, отсутствие `container_name`, секреты | `docker inspect`, `docker compose ps/config` | у api `NanoCpus`/`Memory` = лимитам конфигурации, `bindings=map[]`; суммарные лимиты стенда 2304 MiB; `container_name` отсутствует; пароль БД и токен метрик в трекаемых файлах не встречаются | `evidence/level4/limits-and-ports.txt` | -| Rolling update под нагрузкой, миграции однократно | `python3 scripts/lab3/lab3.py rolling-update --under-load --rate 200 --build-id …` | реплики заменяются по одной, `BUILD_ID` и image id меняются, 0 ответов 5xx из 880 попыток, `migrate` не перезапускался | `evidence/level4/rolling-update.json`, `.md` | +| Rolling update под нагрузкой, миграции однократно | `python3 scripts/lab3/lab3.py rolling-update --under-load --rate 200 --build-id …` | реплики заменяются по одной, `BUILD_ID` и image id меняются, 0 ответов 5xx из 884 попыток, `migrate` не перезапускался | `evidence/level4/rolling-update.json`, `.md` | | Отказы зависимостей | `make test-dependencies` | 2/2: bounded `503 unavailable` с `request_id` для Redis и PostgreSQL, запрос без сессии остаётся `401`, после возврата зависимости данные и сессия целы | `evidence/level4/dependency-faults.txt` | | Трек 1: цена масштабирования | `python3 scripts/lab3/lab3.py sweep/growth/ab/analyze` | серия 1/2/4 реплики, точки сатурации, серия роста ресурсов, подтверждённое улучшение | `evidence/track1/summary-*.md`, `evidence/track1/graphs/*.png`, `evidence/track1/raw/` | | Воспроизводимость с чистого clone | `git clone -b lab3 … /tmp/lab3-clean` → `cp .env.example .env` → `docker compose down -v` → `make stand-up && make test && make test-persistence && make test-failover` | `PASS`: миграции применяются с пустого тома, smoke, сохранность и отказ реплики проходят без правок репозитория | `evidence/level3/clean-clone.txt` | @@ -331,9 +331,9 @@ PostgreSQL 155 % (квота 150 %) и генератор, чей пик 144 % | Воздействие | Что ожидаем | Что наблюдали | Время восстановления / потеря данных | | --- | --- | --- | --- | -| `SIGTERM` одной API-реплике (`scripts/stop-one.sh`, `docker stop --time 10`) | LB исключает узел, клиент продолжает работать; подтверждённые записи целы | HAProxy: `UP → 1/2 → DOWN (L7STS) → MAINT (resolution)` ≈1 s после начала остановки; k6 держит 200 req/s без dropped и ошибок; проба клиента: 0 ошибок, 0 дублей, 0 потерь (289/290 и 290/290 подтверждены) | реплика возвращается за ≈3 s (`scripts/start-replica.sh`); потерь нет | +| `SIGTERM` одной API-реплике (`scripts/stop-one.sh`, `docker stop --time 10`) | LB исключает узел, клиент продолжает работать; подтверждённые записи целы | HAProxy: `UP → 1/2 → DOWN (L7STS) → MAINT (resolution)` ≈1 s после начала остановки; k6 держит 200 req/s без dropped и ошибок; проба клиента: 0 ошибок, 0 дублей, 0 потерь (295/295 и 293/293 подтверждены) | реплика возвращается за ≈3 s (`scripts/start-replica.sh`); потерь нет | | Рестарт всего стенда (`scripts/restart.sh`) | данные и сессии сохраняются, повторный вход не нужен | `make test-persistence` PASS; StartedAt всех контейнеровобновились, токен и сообщения живы | ≈8 s до `/health/ready`; потерь нет | -| Rolling update под нагрузкой (замена реплик по одной) | нет 5xx, миграции не перезапускаются | 0 ответов 5xx из 880 попыток при 200 req/s; `BUILD_ID` меняется по шагам; `migrate` StartedAt не изменился | реплика поднимается за ≈8 s, стенд не теряет запросы | +| Rolling update под нагрузкой (замена реплик по одной) | нет 5xx, миграции не перезапускаются | 0 ответов 5xx из 884 попыток при 200 req/s; `BUILD_ID` меняется по шагам; `migrate` StartedAt не изменился | реплика поднимается за ≈8 s, стенд не теряет запросы | | Отказ Redis (`docker compose stop redis`) | ограниченный 503, без 500 и без утечки деталей драйвера; анонимный запрос остаётся 401 | `make test-dependencies` PASS | зависимость возвращается за ≈2 s, сессия сохраняется (AOF) | | Отказ PostgreSQL (`docker compose stop postgres`) | то же; после возврата данные целы | PASS, подтверждённые сообщения и сессия на месте | ≈6 s до `/health/ready` | diff --git a/evidence/level4/dependency-faults.txt b/evidence/level4/dependency-faults.txt index e05fe1e..0f85906 100644 --- a/evidence/level4/dependency-faults.txt +++ b/evidence/level4/dependency-faults.txt @@ -1,6 +1,6 @@ === make test-dependencies (отказы Redis и PostgreSQL) -captured_at 2026-10-15T18:32:34Z -commit: 410e1ea3ad1b12b1bc650484c3531e6bf76479be +captured_at 2026-10-15T20:17:44Z +commit: eabb1d2909a02aa6020694769ca1304cd3de63a5 cd app && E2E_COMPOSE_DIR="/home/tomato/Projects/VP/mtusi2026lab3" E2E_COMPOSE_ENV_FILE=".env" npm run test:dependencies @@ -14,6 +14,6 @@ The plugin "vite-tsconfig-paths" is detected. Vite now supports tsconfig paths r Test Files 1 passed (1) Tests 2 passed (2) - Start at 21:32:35 - Duration 35.70s (transform 36ms, setup 0ms, import 52ms, tests 35.51s, environment 0ms) + Start at 23:17:44 + Duration 35.87s (transform 38ms, setup 0ms, import 51ms, tests 35.72s, environment 0ms) diff --git a/evidence/level4/failover-sweep.json b/evidence/level4/failover-sweep.json index d7157f9..0098d74 100644 --- a/evidence/level4/failover-sweep.json +++ b/evidence/level4/failover-sweep.json @@ -1,7 +1,7 @@ [ { "address": "172.30.0.5", - "container": "438cdb0e0de6", + "container": "3adeb4c637a0", "duplicates": [], "haproxy": { "BACKEND": { @@ -10,9 +10,9 @@ ], "errors": 0, "qcur_peak": 0, - "requests": 11962, - "rps": 202.75, - "scur_peak": 5, + "requests": 11960, + "rps": 199.33, + "scur_peak": 2, "statuses": [ "UP" ], @@ -20,7 +20,7 @@ { "check": "", "status": "UP", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] }, @@ -30,8 +30,8 @@ ], "errors": 0, "qcur_peak": 0, - "requests": 392, - "rps": 6.64, + "requests": 390, + "rps": 6.5, "scur_peak": 62, "statuses": [ "OPEN" @@ -40,7 +40,7 @@ { "check": "", "status": "OPEN", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] }, @@ -51,9 +51,9 @@ ], "errors": 0, "qcur_peak": 0, - "requests": 3377, - "rps": 57.24, - "scur_peak": 2, + "requests": 3385, + "rps": 56.42, + "scur_peak": 1, "statuses": [ "DOWN", "MAINT (resolution)", @@ -64,32 +64,32 @@ { "check": "L7OK", "status": "UP", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" + }, + { + "check": "L7STS", + "status": "UP 1/2", + "t": "2026-10-15T20:18:39Z" }, { "check": "L7STS", "status": "DOWN", - "t": "2026-10-15T18:37:03Z" + "t": "2026-10-15T20:18:39Z" }, { "check": "", "status": "MAINT (resolution)", - "t": "2026-10-15T18:37:06Z" - }, - { - "check": "L4TOUT", - "status": "UP 1/2", - "t": "2026-10-15T18:37:26Z" + "t": "2026-10-15T20:18:42Z" }, { "check": "L4CON", "status": "DOWN", - "t": "2026-10-15T18:37:26Z" + "t": "2026-10-15T20:19:02Z" }, { "check": "L7OK", "status": "UP", - "t": "2026-10-15T18:37:28Z" + "t": "2026-10-15T20:19:04Z" } ] }, @@ -99,9 +99,9 @@ ], "errors": 0, "qcur_peak": 0, - "requests": 8639, - "rps": 146.42, - "scur_peak": 3, + "requests": 8586, + "rps": 143.1, + "scur_peak": 2, "statuses": [ "UP" ], @@ -109,7 +109,7 @@ { "check": "L7OK", "status": "UP", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] }, @@ -129,7 +129,7 @@ { "check": "", "status": "MAINT (resolution)", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] }, @@ -149,7 +149,7 @@ { "check": "", "status": "MAINT (resolution)", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] }, @@ -169,7 +169,7 @@ { "check": "", "status": "MAINT (resolution)", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] }, @@ -189,7 +189,7 @@ { "check": "", "status": "MAINT (resolution)", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] }, @@ -209,7 +209,7 @@ { "check": "", "status": "MAINT (resolution)", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] }, @@ -229,12 +229,12 @@ { "check": "", "status": "MAINT (resolution)", - "t": "2026-10-15T18:36:54Z" + "t": "2026-10-15T20:18:30Z" } ] } }, - "instance": "438cdb0e0de6", + "instance": "3adeb4c637a0", "k6": { "achieved_rps": 200.02, "dropped": 0, @@ -243,1979 +243,547 @@ "missing_confirmed": [], "phases": { "first_success_after_exit_s": 0.2, - "lb_down_at": "2026-10-15T18:37:03Z", + "lb_down_at": "2026-10-15T20:18:39Z", "probe_offsets": { - "container_exit": 11.25, - "last_sample": 59.74, - "restore_command": 31.45, - "stop_command": 8.01 - }, - "replica_rejoined_s": 3.34, - "restore_at": "2026-10-15T18:37:25Z", - "stop_command_at": "2026-10-15T18:37:05Z", - "stop_command_took_s": 3.23, - "stop_exit_code": 0 - }, - "probe": { - "attempts": 289, - "confirmed": 289, - "errors_while_stopped": 0, - "failures": 0, - "rps_after": 4.83, - "rps_before": 4.87, - "rps_while_stopped": 0.0, - "successes": 289, - "timeline": [ - [ - 0.002, - 1 - ], - [ - 0.218, - 1 - ], - [ - 0.429, - 1 - ], - [ - 0.635, - 1 - ], - [ - 0.84, - 1 - ], - [ - 1.052, - 1 - ], - [ - 1.264, - 1 - ], - [ - 1.472, - 1 - ], - [ - 1.677, - 1 - ], - [ - 1.883, - 1 - ], - [ - 2.088, - 1 - ], - [ - 2.293, - 1 - ], - [ - 2.499, - 1 - ], - [ - 2.706, - 1 - ], - [ - 2.912, - 1 - ], - [ - 3.124, - 1 - ], - [ - 3.332, - 1 - ], - [ - 3.537, - 1 - ], - [ - 3.744, - 1 - ], - [ - 3.955, - 1 - ], - [ - 4.165, - 1 - ], - [ - 4.377, - 1 - ], - [ - 4.583, - 1 - ], - [ - 4.792, - 1 - ], - [ - 4.997, - 1 - ], - [ - 5.211, - 1 - ], - [ - 5.422, - 1 - ], - [ - 5.631, - 1 - ], - [ - 5.841, - 1 - ], - [ - 6.055, - 1 - ], - [ - 6.262, - 1 - ], - [ - 6.47, - 1 - ], - [ - 6.676, - 1 - ], - [ - 6.886, - 1 - ], - [ - 7.092, - 1 - ], - [ - 7.297, - 1 - ], - [ - 7.505, - 1 - ], - [ - 7.716, - 1 - ], - [ - 7.925, - 1 - ], - [ - 8.135, - 1 - ], - [ - 8.343, - 1 - ], - [ - 8.553, - 1 - ], - [ - 8.76, - 1 - ], - [ - 8.966, - 1 - ], - [ - 9.177, - 1 - ], - [ - 9.383, - 1 - ], - [ - 9.589, - 1 - ], - [ - 9.793, - 1 - ], - [ - 9.997, - 1 - ], - [ - 10.204, - 1 - ], - [ - 10.408, - 1 - ], - [ - 10.613, - 1 - ], - [ - 10.82, - 1 - ], - [ - 11.026, - 1 - ], - [ - 11.231, - 1 - ], - [ - 11.436, - 1 - ], - [ - 11.641, - 1 - ], - [ - 11.845, - 1 - ], - [ - 12.05, - 1 - ], - [ - 12.256, - 1 - ], - [ - 12.46, - 1 - ], - [ - 12.665, - 1 - ], - [ - 12.871, - 1 - ], - [ - 13.075, - 1 - ], - [ - 13.28, - 1 - ], - [ - 13.485, - 1 - ], - [ - 13.69, - 1 - ], - [ - 13.898, - 1 - ], - [ - 14.112, - 1 - ], - [ - 14.323, - 1 - ], - [ - 14.532, - 1 - ], - [ - 14.742, - 1 - ], - [ - 14.95, - 1 - ], - [ - 15.161, - 1 - ], - [ - 15.372, - 1 - ], - [ - 15.585, - 1 - ], - [ - 15.799, - 1 - ], - [ - 16.004, - 1 - ], - [ - 16.209, - 1 - ], - [ - 16.414, - 1 - ], - [ - 16.619, - 1 - ], - [ - 16.825, - 1 - ], - [ - 17.031, - 1 - ], - [ - 17.235, - 1 - ], - [ - 17.44, - 1 - ], - [ - 17.646, - 1 - ], - [ - 17.851, - 1 - ], - [ - 18.055, - 1 - ], - [ - 18.263, - 1 - ], - [ - 18.47, - 1 - ], - [ - 18.679, - 1 - ], - [ - 18.885, - 1 - ], - [ - 19.091, - 1 - ], - [ - 19.296, - 1 - ], - [ - 19.501, - 1 - ], - [ - 19.706, - 1 - ], - [ - 19.912, - 1 - ], - [ - 20.117, - 1 - ], - [ - 20.322, - 1 - ], - [ - 20.529, - 1 - ], - [ - 20.734, - 1 - ], - [ - 20.94, - 1 - ], - [ - 21.147, - 1 - ], - [ - 21.353, - 1 - ], - [ - 21.558, - 1 - ], - [ - 21.765, - 1 - ], - [ - 21.969, - 1 - ], - [ - 22.174, - 1 - ], - [ - 22.379, - 1 - ], - [ - 22.583, - 1 - ], - [ - 22.788, - 1 - ], - [ - 22.993, - 1 - ], - [ - 23.198, - 1 - ], - [ - 23.405, - 1 - ], - [ - 23.61, - 1 - ], - [ - 23.818, - 1 - ], - [ - 24.027, - 1 - ], - [ - 24.239, - 1 - ], - [ - 24.448, - 1 - ], - [ - 24.656, - 1 - ], - [ - 24.866, - 1 - ], - [ - 25.078, - 1 - ], - [ - 25.286, - 1 - ], - [ - 25.492, - 1 - ], - [ - 25.697, - 1 - ], - [ - 25.902, - 1 - ], - [ - 26.107, - 1 - ], - [ - 26.311, - 1 - ], - [ - 26.517, - 1 - ], - [ - 26.722, - 1 - ], - [ - 26.928, - 1 - ], - [ - 27.133, - 1 - ], - [ - 27.338, - 1 - ], - [ - 27.542, - 1 - ], - [ - 27.747, - 1 - ], - [ - 27.952, - 1 - ], - [ - 28.156, - 1 - ], - [ - 28.36, - 1 - ], - [ - 28.566, - 1 - ], - [ - 28.771, - 1 - ], - [ - 28.976, - 1 - ], - [ - 29.181, - 1 - ], - [ - 29.387, - 1 - ], - [ - 29.592, - 1 - ], - [ - 29.796, - 1 - ], - [ - 30.0, - 1 - ], - [ - 30.207, - 1 - ], - [ - 30.411, - 1 - ], - [ - 30.615, - 1 - ], - [ - 30.821, - 1 - ], - [ - 31.026, - 1 - ], - [ - 31.23, - 1 - ], - [ - 31.436, - 1 - ], - [ - 31.641, - 1 - ], - [ - 31.847, - 1 - ], - [ - 32.057, - 1 - ], - [ - 32.262, - 1 - ], - [ - 32.469, - 1 - ], - [ - 32.674, - 1 - ], - [ - 32.881, - 1 - ], - [ - 33.112, - 1 - ], - [ - 33.317, - 1 - ], - [ - 33.523, - 1 - ], - [ - 33.728, - 1 - ], - [ - 33.934, - 1 - ], - [ - 34.14, - 1 - ], - [ - 34.344, - 1 - ], - [ - 34.549, - 1 - ], - [ - 34.753, - 1 - ], - [ - 34.958, - 1 - ], - [ - 35.164, - 1 - ], - [ - 35.37, - 1 - ], - [ - 35.575, - 1 - ], - [ - 35.781, - 1 - ], - [ - 35.987, - 1 - ], - [ - 36.193, - 1 - ], - [ - 36.398, - 1 - ], - [ - 36.603, - 1 - ], - [ - 36.809, - 1 - ], - [ - 37.014, - 1 - ], - [ - 37.22, - 1 - ], - [ - 37.426, - 1 - ], - [ - 37.631, - 1 - ], - [ - 37.836, - 1 - ], - [ - 38.046, - 1 - ], - [ - 38.253, - 1 - ], - [ - 38.508, - 1 - ], - [ - 38.722, - 1 - ], - [ - 38.933, - 1 - ], - [ - 39.141, - 1 - ], - [ - 39.349, - 1 - ], - [ - 39.566, - 1 - ], - [ - 39.774, - 1 - ], - [ - 39.98, - 1 - ], - [ - 40.186, - 1 - ], - [ - 40.397, - 1 - ], - [ - 40.606, - 1 - ], - [ - 40.817, - 1 - ], - [ - 41.025, - 1 - ], - [ - 41.233, - 1 - ], - [ - 41.44, - 1 - ], - [ - 41.647, - 1 - ], - [ - 41.856, - 1 - ], - [ - 42.061, - 1 - ], - [ - 42.268, - 1 - ], - [ - 42.477, - 1 - ], - [ - 42.688, - 1 - ], - [ - 42.908, - 1 - ], - [ - 43.117, - 1 - ], - [ - 43.325, - 1 - ], - [ - 43.534, - 1 - ], - [ - 43.742, - 1 - ], - [ - 43.954, - 1 - ], - [ - 44.168, - 1 - ], - [ - 44.377, - 1 - ], - [ - 44.583, - 1 - ], - [ - 44.79, - 1 - ], - [ - 44.995, - 1 - ], - [ - 45.202, - 1 - ], - [ - 45.406, - 1 - ], - [ - 45.611, - 1 - ], - [ - 45.817, - 1 - ], - [ - 46.021, - 1 - ], - [ - 46.226, - 1 - ], - [ - 46.432, - 1 - ], - [ - 46.638, - 1 - ], - [ - 46.842, - 1 - ], - [ - 47.047, - 1 - ], - [ - 47.253, - 1 - ], - [ - 47.458, - 1 - ], - [ - 47.664, - 1 - ], - [ - 47.869, - 1 - ], - [ - 48.075, - 1 - ], - [ - 48.28, - 1 - ], - [ - 48.486, - 1 - ], - [ - 48.693, - 1 - ], - [ - 48.898, - 1 - ], - [ - 49.105, - 1 - ], - [ - 49.314, - 1 - ], - [ - 49.52, - 1 - ], - [ - 49.725, - 1 - ], - [ - 49.931, - 1 - ], - [ - 50.135, - 1 - ], - [ - 50.342, - 1 - ], - [ - 50.55, - 1 - ], - [ - 50.759, - 1 - ], - [ - 50.968, - 1 - ], - [ - 51.173, - 1 - ], - [ - 51.379, - 1 - ], - [ - 51.586, - 1 - ], - [ - 51.795, - 1 - ], - [ - 52.006, - 1 - ], - [ - 52.215, - 1 - ], - [ - 52.425, - 1 - ], - [ - 52.632, - 1 - ], - [ - 52.837, - 1 - ], - [ - 53.047, - 1 - ], - [ - 53.258, - 1 - ], - [ - 53.463, - 1 - ], - [ - 53.672, - 1 - ], - [ - 53.88, - 1 - ], - [ - 54.089, - 1 - ], - [ - 54.296, - 1 - ], - [ - 54.503, - 1 - ], - [ - 54.709, - 1 - ], - [ - 54.916, - 1 - ], - [ - 55.123, - 1 - ], - [ - 55.336, - 1 - ], - [ - 55.547, - 1 - ], - [ - 55.752, - 1 - ], - [ - 55.957, - 1 - ], - [ - 56.162, - 1 - ], - [ - 56.368, - 1 - ], - [ - 56.582, - 1 - ], - [ - 56.8, - 1 - ], - [ - 57.006, - 1 - ], - [ - 57.212, - 1 - ], - [ - 57.417, - 1 - ], - [ - 57.621, - 1 - ], - [ - 57.826, - 1 - ], - [ - 58.032, - 1 - ], - [ - 58.24, - 1 - ], - [ - 58.45, - 1 - ], - [ - 58.66, - 1 - ], - [ - 58.872, - 1 - ], - [ - 59.118, - 1 - ], - [ - 59.323, - 1 - ], - [ - 59.529, - 1 - ], - [ - 59.739, - 1 - ] - ] - }, - "rate": 200 - }, - { - "address": "172.30.0.4", - "container": "63edcffba23f", - "duplicates": [], - "haproxy": { - "BACKEND": { - "addresses": [ - "" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 11957, - "rps": 199.28, - "scur_peak": 3, - "statuses": [ - "UP" - ], - "transitions": [ - { - "check": "", - "status": "UP", - "t": "2026-10-15T18:37:55Z" - } - ] - }, - "FRONTEND": { - "addresses": [ - "" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 387, - "rps": 6.45, - "scur_peak": 62, - "statuses": [ - "OPEN" - ], - "transitions": [ - { - "check": "", - "status": "OPEN", - "t": "2026-10-15T18:37:55Z" - } - ] - }, - "api1": { - "addresses": [ - "172.30.0.5:8080" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 8583, - "rps": 143.05, - "scur_peak": 3, - "statuses": [ - "UP" - ], - "transitions": [ - { - "check": "L7OK", - "status": "UP", - "t": "2026-10-15T18:37:55Z" - } - ] - }, - "api2": { - "addresses": [ - "", - "172.30.0.4:8080" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 3422, - "rps": 57.03, - "scur_peak": 2, - "statuses": [ - "DOWN", - "MAINT (resolution)", - "UP", - "UP 1/2" - ], - "transitions": [ - { - "check": "L7OK", - "status": "UP", - "t": "2026-10-15T18:37:55Z" - }, - { - "check": "L7STS", - "status": "UP 1/2", - "t": "2026-10-15T18:38:04Z" - }, - { - "check": "L7STS", - "status": "DOWN", - "t": "2026-10-15T18:38:04Z" - }, - { - "check": "", - "status": "MAINT (resolution)", - "t": "2026-10-15T18:38:07Z" - }, - { - "check": "L7STS", - "status": "UP 1/2", - "t": "2026-10-15T18:38:28Z" - }, - { - "check": "L4CON", - "status": "DOWN", - "t": "2026-10-15T18:38:28Z" - }, - { - "check": "L7OK", - "status": "UP", - "t": "2026-10-15T18:38:29Z" - } - ] - }, - "api3": { - "addresses": [ - "" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 0, - "rps": 0.0, - "scur_peak": 0, - "statuses": [ - "MAINT (resolution)" - ], - "transitions": [ - { - "check": "", - "status": "MAINT (resolution)", - "t": "2026-10-15T18:37:55Z" - } - ] - }, - "api4": { - "addresses": [ - "" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 0, - "rps": 0.0, - "scur_peak": 0, - "statuses": [ - "MAINT (resolution)" - ], - "transitions": [ - { - "check": "", - "status": "MAINT (resolution)", - "t": "2026-10-15T18:37:55Z" - } - ] - }, - "api5": { - "addresses": [ - "" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 0, - "rps": 0.0, - "scur_peak": 0, - "statuses": [ - "MAINT (resolution)" - ], - "transitions": [ - { - "check": "", - "status": "MAINT (resolution)", - "t": "2026-10-15T18:37:55Z" - } - ] - }, - "api6": { - "addresses": [ - "" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 0, - "rps": 0.0, - "scur_peak": 0, - "statuses": [ - "MAINT (resolution)" - ], - "transitions": [ - { - "check": "", - "status": "MAINT (resolution)", - "t": "2026-10-15T18:37:55Z" - } - ] - }, - "api7": { - "addresses": [ - "" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 0, - "rps": 0.0, - "scur_peak": 0, - "statuses": [ - "MAINT (resolution)" - ], - "transitions": [ - { - "check": "", - "status": "MAINT (resolution)", - "t": "2026-10-15T18:37:55Z" - } - ] - }, - "api8": { - "addresses": [ - "" - ], - "errors": 0, - "qcur_peak": 0, - "requests": 0, - "rps": 0.0, - "scur_peak": 0, - "statuses": [ - "MAINT (resolution)" - ], - "transitions": [ - { - "check": "", - "status": "MAINT (resolution)", - "t": "2026-10-15T18:37:55Z" - } - ] - } - }, - "instance": "63edcffba23f", - "k6": { - "achieved_rps": 200.02, - "dropped": 0, - "failed": 0 - }, - "missing_confirmed": [], - "phases": { - "first_success_after_exit_s": 0.2, - "lb_down_at": "2026-10-15T18:38:04Z", - "probe_offsets": { - "container_exit": 11.24, - "last_sample": 59.66, - "restore_command": 31.44, + "container_exit": 11.22, + "last_sample": 60.48, + "restore_command": 31.42, "stop_command": 8.0 }, "replica_rejoined_s": 2.84, - "restore_at": "2026-10-15T18:38:27Z", - "stop_command_at": "2026-10-15T18:38:07Z", - "stop_command_took_s": 3.24, + "restore_at": "2026-10-15T20:19:02Z", + "stop_command_at": "2026-10-15T20:18:41Z", + "stop_command_took_s": 3.22, "stop_exit_code": 0 }, "probe": { - "attempts": 290, - "confirmed": 290, + "attempts": 295, + "confirmed": 295, "errors_while_stopped": 0, "failures": 0, - "rps_after": 4.85, + "rps_after": 4.87, "rps_before": 4.87, "rps_while_stopped": 0.0, - "successes": 290, + "successes": 295, "timeline": [ [ 0.0, 1 ], [ - 0.216, + 0.224, 1 ], [ - 0.424, + 0.446, 1 ], [ - 0.628, + 0.652, 1 ], [ - 0.848, + 0.86, 1 ], [ - 1.064, + 1.082, 1 ], [ - 1.277, + 1.294, 1 ], [ - 1.488, + 1.513, 1 ], [ - 1.698, + 1.723, 1 ], [ - 1.907, + 1.932, 1 ], [ - 2.112, + 2.149, 1 ], [ - 2.321, + 2.356, 1 ], [ - 2.531, + 2.561, 1 ], [ - 2.74, + 2.767, 1 ], [ - 2.946, + 2.973, 1 ], [ - 3.151, + 3.184, 1 ], [ - 3.356, + 3.394, 1 ], [ - 3.56, + 3.612, 1 ], [ - 3.766, + 3.817, 1 ], [ - 3.971, + 4.026, 1 ], [ - 4.176, + 4.234, 1 ], [ - 4.38, + 4.442, 1 ], [ - 4.585, + 4.646, 1 ], [ - 4.791, + 4.85, 1 ], [ - 4.996, + 5.056, 1 ], [ - 5.201, + 5.263, 1 ], [ - 5.406, + 5.468, 1 ], [ - 5.61, + 5.672, 1 ], [ - 5.816, + 5.878, 1 ], [ - 6.022, + 6.083, 1 ], [ - 6.227, + 6.288, 1 ], [ - 6.431, + 6.492, 1 ], [ - 6.636, + 6.697, 1 ], [ - 6.842, + 6.901, 1 ], [ - 7.047, + 7.106, 1 ], [ - 7.253, + 7.311, 1 ], [ - 7.459, + 7.517, 1 ], [ - 7.666, + 7.721, 1 ], [ - 7.871, + 7.924, 1 ], [ - 8.079, + 8.129, 1 ], [ - 8.283, + 8.334, 1 ], [ - 8.488, + 8.538, 1 ], [ - 8.693, + 8.742, 1 ], [ - 8.898, + 8.948, 1 ], [ - 9.102, + 9.152, 1 ], [ - 9.308, + 9.356, 1 ], [ - 9.513, + 9.561, 1 ], [ - 9.718, + 9.766, 1 ], [ - 9.924, + 9.971, 1 ], [ - 10.129, + 10.175, 1 ], [ - 10.333, + 10.381, 1 ], [ - 10.538, + 10.585, 1 ], [ - 10.755, + 10.789, 1 ], [ - 10.96, + 10.995, 1 ], [ - 11.168, + 11.201, 1 ], [ - 11.374, + 11.405, 1 ], [ - 11.584, + 11.609, 1 ], [ - 11.794, + 11.82, 1 ], [ - 11.999, + 12.03, 1 ], [ - 12.205, + 12.239, 1 ], [ - 12.411, + 12.448, 1 ], [ - 12.617, + 12.658, 1 ], [ - 12.822, + 12.868, 1 ], [ - 13.027, + 13.076, 1 ], [ - 13.233, + 13.284, 1 ], [ - 13.438, + 13.489, 1 ], [ - 13.643, + 13.693, 1 ], [ - 13.848, + 13.897, 1 ], [ - 14.053, + 14.102, 1 ], [ - 14.258, + 14.306, 1 ], [ - 14.463, + 14.511, 1 ], [ - 14.668, + 14.715, 1 ], [ - 14.873, + 14.92, 1 ], [ - 15.079, + 15.125, 1 ], [ - 15.284, + 15.33, 1 ], [ - 15.488, + 15.534, 1 ], [ - 15.693, + 15.74, 1 ], [ - 15.898, + 15.943, 1 ], [ - 16.103, + 16.149, 1 ], [ - 16.308, + 16.355, 1 ], [ - 16.512, + 16.559, 1 ], [ - 16.718, + 16.764, 1 ], [ - 16.923, + 16.97, 1 ], [ - 17.128, + 17.174, 1 ], [ - 17.333, + 17.378, 1 ], [ - 17.538, + 17.583, 1 ], [ - 17.743, + 17.787, 1 ], [ - 17.948, + 17.991, 1 ], [ - 18.153, + 18.203, 1 ], [ - 18.358, + 18.411, 1 ], [ - 18.564, + 18.619, 1 ], [ - 18.768, + 18.83, 1 ], [ - 18.973, + 19.035, 1 ], [ - 19.177, + 19.241, 1 ], [ - 19.382, + 19.445, 1 ], [ - 19.589, + 19.65, 1 ], [ - 19.793, + 19.856, 1 ], [ - 19.998, + 20.06, 1 ], [ - 20.203, + 20.265, 1 ], [ - 20.408, + 20.468, 1 ], [ - 20.612, + 20.673, 1 ], [ - 20.818, + 20.877, 1 ], [ - 21.024, + 21.082, 1 ], [ - 21.228, + 21.285, 1 ], [ - 21.432, + 21.491, 1 ], [ - 21.637, + 21.695, 1 ], [ - 21.843, + 21.9, 1 ], [ - 22.048, + 22.105, 1 ], [ - 22.253, + 22.309, 1 ], [ - 22.459, + 22.513, 1 ], [ - 22.662, + 22.718, 1 ], [ - 22.869, + 22.922, 1 ], [ - 23.075, + 23.127, 1 ], [ - 23.279, + 23.331, 1 ], [ - 23.485, + 23.536, 1 ], [ - 23.691, + 23.741, 1 ], [ - 23.895, + 23.945, 1 ], [ - 24.1, + 24.149, 1 ], [ - 24.305, + 24.353, 1 ], [ - 24.513, + 24.557, 1 ], [ - 24.72, + 24.76, 1 ], [ - 24.932, + 24.967, 1 ], [ - 25.142, + 25.17, 1 ], [ - 25.347, + 25.376, 1 ], [ - 25.56, + 25.581, 1 ], [ - 25.768, + 25.785, 1 ], [ - 25.981, + 25.99, 1 ], [ - 26.192, + 26.195, 1 ], [ - 26.398, + 26.4, 1 ], [ - 26.603, + 26.604, 1 ], [ @@ -2223,15 +791,15 @@ 1 ], [ - 27.013, + 27.014, 1 ], [ - 27.217, + 27.218, 1 ], [ - 27.423, + 27.422, 1 ], [ @@ -2243,619 +811,2088 @@ 1 ], [ - 28.038, + 28.035, 1 ], [ - 28.244, + 28.24, 1 ], [ - 28.448, + 28.445, 1 ], [ - 28.652, + 28.651, 1 ], [ - 28.858, + 28.855, 1 ], [ - 29.063, + 29.061, 1 ], [ - 29.267, + 29.265, 1 ], [ - 29.473, + 29.469, 1 ], [ - 29.678, + 29.673, 1 ], [ - 29.883, + 29.877, 1 ], [ - 30.088, + 30.081, 1 ], [ - 30.294, + 30.285, 1 ], [ - 30.499, + 30.489, 1 ], [ - 30.704, + 30.694, 1 ], [ - 30.909, + 30.9, 1 ], [ - 31.114, + 31.105, 1 ], [ - 31.319, + 31.309, 1 ], [ - 31.524, + 31.516, 1 ], [ - 31.731, + 31.72, 1 ], [ - 31.936, + 31.924, 1 ], [ - 32.141, + 32.128, 1 ], [ - 32.35, + 32.336, 1 ], [ - 32.555, + 32.54, 1 ], [ - 32.762, + 32.743, 1 ], [ - 32.97, + 32.949, 1 ], [ - 33.175, + 33.155, 1 ], [ - 33.381, + 33.359, 1 ], [ - 33.586, + 33.564, 1 ], [ - 33.791, + 33.772, 1 ], [ - 34.002, + 33.979, 1 ], [ - 34.215, + 34.187, 1 ], [ - 34.422, + 34.394, 1 ], [ - 34.634, + 34.603, 1 ], [ - 34.848, + 34.812, 1 ], [ - 35.061, + 35.021, 1 ], [ - 35.267, + 35.227, 1 ], [ - 35.473, + 35.432, 1 ], [ - 35.681, + 35.638, 1 ], [ - 35.894, + 35.842, 1 ], [ - 36.108, + 36.051, 1 ], [ - 36.322, + 36.263, 1 ], [ - 36.528, + 36.473, 1 ], [ - 36.736, + 36.68, 1 ], [ - 36.941, + 36.888, 1 ], [ - 37.151, + 37.096, 1 ], [ - 37.361, + 37.302, 1 ], [ - 37.569, + 37.507, 1 ], [ - 37.775, + 37.714, 1 ], [ - 37.98, + 37.921, 1 ], [ - 38.191, + 38.13, 1 ], [ - 38.399, + 38.336, 1 ], [ - 38.612, + 38.544, 1 ], [ - 38.821, + 38.754, 1 ], [ - 39.03, + 38.964, 1 ], [ - 39.243, + 39.174, 1 ], [ - 39.454, + 39.382, 1 ], [ - 39.659, + 39.588, 1 ], [ - 39.864, + 39.793, 1 ], [ - 40.069, + 39.998, 1 ], [ - 40.279, + 40.203, 1 ], [ - 40.489, + 40.407, 1 ], [ - 40.695, + 40.611, 1 ], [ - 40.902, + 40.815, 1 ], [ - 41.106, + 41.02, 1 ], [ - 41.311, + 41.226, 1 ], [ - 41.515, + 41.431, 1 ], [ - 41.721, + 41.636, 1 ], [ - 41.926, + 41.841, 1 ], [ - 42.131, + 42.046, 1 ], [ - 42.337, + 42.25, 1 ], [ - 42.547, + 42.456, 1 ], [ - 42.76, + 42.66, 1 ], [ - 42.969, + 42.864, 1 ], [ - 43.175, + 43.068, 1 ], [ - 43.383, + 43.272, 1 ], [ - 43.594, + 43.477, 1 ], [ - 43.805, + 43.681, 1 ], [ - 44.014, + 43.884, 1 ], [ - 44.219, + 44.089, 1 ], [ - 44.425, + 44.295, 1 ], [ - 44.629, + 44.499, 1 ], [ - 44.834, + 44.703, 1 ], [ - 45.04, + 44.909, 1 ], [ - 45.245, + 45.114, 1 ], [ - 45.451, + 45.318, 1 ], [ - 45.656, + 45.523, 1 ], [ - 45.861, + 45.727, 1 ], [ - 46.067, + 45.932, 1 ], [ - 46.272, + 46.135, 1 ], [ - 46.478, + 46.339, 1 ], [ - 46.683, + 46.546, 1 ], [ - 46.89, + 46.75, 1 ], [ - 47.103, + 46.954, 1 ], [ - 47.31, + 47.158, 1 ], [ - 47.515, + 47.363, 1 ], [ - 47.72, + 47.568, 1 ], [ - 47.925, + 47.773, 1 ], [ - 48.13, + 47.977, 1 ], [ - 48.335, + 48.182, 1 ], [ - 48.541, + 48.386, 1 ], [ - 48.746, + 48.591, 1 ], [ - 48.952, + 48.795, 1 ], [ - 49.157, + 49.003, 1 ], [ - 49.363, + 49.212, 1 ], [ - 49.569, + 49.421, 1 ], [ - 49.773, + 49.626, 1 ], [ - 49.978, + 49.831, 1 ], [ - 50.183, + 50.035, 1 ], [ - 50.388, + 50.24, 1 ], [ - 50.594, + 50.444, 1 ], [ - 50.799, + 50.65, 1 ], [ - 51.005, + 50.855, 1 ], [ - 51.21, + 51.059, 1 ], [ - 51.416, + 51.266, 1 ], [ - 51.621, + 51.471, 1 ], [ - 51.829, + 51.675, 1 ], [ - 52.035, + 51.88, 1 ], [ - 52.24, + 52.086, 1 ], [ - 52.446, + 52.289, 1 ], [ - 52.651, + 52.494, 1 ], [ - 52.856, + 52.698, 1 ], [ - 53.062, + 52.902, 1 ], [ - 53.267, + 53.106, 1 ], [ - 53.473, + 53.311, 1 ], [ - 53.678, + 53.516, 1 ], [ - 53.884, + 53.721, 1 ], [ - 54.091, + 53.925, 1 ], [ - 54.295, + 54.131, 1 ], [ - 54.5, + 54.335, 1 ], [ - 54.705, + 54.539, 1 ], [ - 54.91, + 54.743, 1 ], [ - 55.115, + 54.949, 1 ], [ - 55.32, + 55.153, 1 ], [ - 55.524, + 55.357, 1 ], [ - 55.728, + 55.562, 1 ], [ - 55.937, + 55.767, 1 ], [ - 56.145, + 55.972, 1 ], [ - 56.357, + 56.176, 1 ], [ - 56.568, + 56.38, 1 ], [ - 56.775, + 56.584, 1 ], [ - 56.981, + 56.789, 1 ], [ - 57.19, + 56.994, 1 ], [ - 57.394, + 57.198, 1 ], [ - 57.606, + 57.404, 1 ], [ - 57.817, + 57.608, 1 ], [ - 58.021, + 57.812, 1 ], [ - 58.227, + 58.017, 1 ], [ - 58.433, + 58.222, 1 ], [ - 58.638, + 58.426, 1 ], [ - 58.843, + 58.631, 1 ], [ - 59.049, + 58.835, 1 ], [ - 59.254, + 59.041, 1 ], [ - 59.458, + 59.246, 1 ], [ - 59.665, + 59.45, + 1 + ], + [ + 59.655, + 1 + ], + [ + 59.859, + 1 + ], + [ + 60.063, + 1 + ], + [ + 60.268, + 1 + ], + [ + 60.475, + 1 + ] + ] + }, + "rate": 200 + }, + { + "address": "172.30.0.4", + "container": "e4b62a206858", + "duplicates": [], + "haproxy": { + "BACKEND": { + "addresses": [ + "" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 11970, + "rps": 202.88, + "scur_peak": 2, + "statuses": [ + "UP" + ], + "transitions": [ + { + "check": "", + "status": "UP", + "t": "2026-10-15T20:19:33Z" + } + ] + }, + "FRONTEND": { + "addresses": [ + "" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 400, + "rps": 6.78, + "scur_peak": 62, + "statuses": [ + "OPEN" + ], + "transitions": [ + { + "check": "", + "status": "OPEN", + "t": "2026-10-15T20:19:33Z" + } + ] + }, + "api1": { + "addresses": [ + "172.30.0.5:8080" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 8697, + "rps": 147.41, + "scur_peak": 2, + "statuses": [ + "UP" + ], + "transitions": [ + { + "check": "L7OK", + "status": "UP", + "t": "2026-10-15T20:19:33Z" + } + ] + }, + "api2": { + "addresses": [ + "", + "172.30.0.4:8080" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 3321, + "rps": 56.29, + "scur_peak": 1, + "statuses": [ + "DOWN", + "DOWN 1/2", + "MAINT (resolution)", + "UP", + "UP 1/2" + ], + "transitions": [ + { + "check": "L7OK", + "status": "UP", + "t": "2026-10-15T20:19:33Z" + }, + { + "check": "L7STS", + "status": "UP 1/2", + "t": "2026-10-15T20:19:41Z" + }, + { + "check": "L7STS", + "status": "DOWN", + "t": "2026-10-15T20:19:42Z" + }, + { + "check": "", + "status": "MAINT (resolution)", + "t": "2026-10-15T20:19:44Z" + }, + { + "check": "L4CON", + "status": "DOWN", + "t": "2026-10-15T20:20:05Z" + }, + { + "check": "L7OK", + "status": "DOWN 1/2", + "t": "2026-10-15T20:20:07Z" + }, + { + "check": "L7OK", + "status": "UP", + "t": "2026-10-15T20:20:08Z" + } + ] + }, + "api3": { + "addresses": [ + "" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 0, + "rps": 0.0, + "scur_peak": 0, + "statuses": [ + "MAINT (resolution)" + ], + "transitions": [ + { + "check": "", + "status": "MAINT (resolution)", + "t": "2026-10-15T20:19:33Z" + } + ] + }, + "api4": { + "addresses": [ + "" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 0, + "rps": 0.0, + "scur_peak": 0, + "statuses": [ + "MAINT (resolution)" + ], + "transitions": [ + { + "check": "", + "status": "MAINT (resolution)", + "t": "2026-10-15T20:19:33Z" + } + ] + }, + "api5": { + "addresses": [ + "" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 0, + "rps": 0.0, + "scur_peak": 0, + "statuses": [ + "MAINT (resolution)" + ], + "transitions": [ + { + "check": "", + "status": "MAINT (resolution)", + "t": "2026-10-15T20:19:33Z" + } + ] + }, + "api6": { + "addresses": [ + "" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 0, + "rps": 0.0, + "scur_peak": 0, + "statuses": [ + "MAINT (resolution)" + ], + "transitions": [ + { + "check": "", + "status": "MAINT (resolution)", + "t": "2026-10-15T20:19:33Z" + } + ] + }, + "api7": { + "addresses": [ + "" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 0, + "rps": 0.0, + "scur_peak": 0, + "statuses": [ + "MAINT (resolution)" + ], + "transitions": [ + { + "check": "", + "status": "MAINT (resolution)", + "t": "2026-10-15T20:19:33Z" + } + ] + }, + "api8": { + "addresses": [ + "" + ], + "errors": 0, + "qcur_peak": 0, + "requests": 0, + "rps": 0.0, + "scur_peak": 0, + "statuses": [ + "MAINT (resolution)" + ], + "transitions": [ + { + "check": "", + "status": "MAINT (resolution)", + "t": "2026-10-15T20:19:33Z" + } + ] + } + }, + "instance": "e4b62a206858", + "k6": { + "achieved_rps": 200.02, + "dropped": 0, + "failed": 0 + }, + "missing_confirmed": [], + "phases": { + "first_success_after_exit_s": 0.2, + "lb_down_at": "2026-10-15T20:19:41Z", + "probe_offsets": { + "container_exit": 11.2, + "last_sample": 60.0, + "restore_command": 31.4, + "stop_command": 8.0 + }, + "replica_rejoined_s": 3.86, + "restore_at": "2026-10-15T20:20:04Z", + "stop_command_at": "2026-10-15T20:19:44Z", + "stop_command_took_s": 3.2, + "stop_exit_code": 0 + }, + "probe": { + "attempts": 293, + "confirmed": 293, + "errors_while_stopped": 0, + "failures": 0, + "rps_after": 4.88, + "rps_before": 4.87, + "rps_while_stopped": 0.0, + "successes": 293, + "timeline": [ + [ + 0.0, + 1 + ], + [ + 0.207, + 1 + ], + [ + 0.423, + 1 + ], + [ + 0.628, + 1 + ], + [ + 0.834, + 1 + ], + [ + 1.044, + 1 + ], + [ + 1.254, + 1 + ], + [ + 1.465, + 1 + ], + [ + 1.673, + 1 + ], + [ + 1.881, + 1 + ], + [ + 2.089, + 1 + ], + [ + 2.3, + 1 + ], + [ + 2.505, + 1 + ], + [ + 2.709, + 1 + ], + [ + 2.915, + 1 + ], + [ + 3.12, + 1 + ], + [ + 3.325, + 1 + ], + [ + 3.529, + 1 + ], + [ + 3.735, + 1 + ], + [ + 3.939, + 1 + ], + [ + 4.144, + 1 + ], + [ + 4.348, + 1 + ], + [ + 4.553, + 1 + ], + [ + 4.757, + 1 + ], + [ + 4.961, + 1 + ], + [ + 5.166, + 1 + ], + [ + 5.372, + 1 + ], + [ + 5.577, + 1 + ], + [ + 5.782, + 1 + ], + [ + 5.986, + 1 + ], + [ + 6.19, + 1 + ], + [ + 6.394, + 1 + ], + [ + 6.6, + 1 + ], + [ + 6.805, + 1 + ], + [ + 7.008, + 1 + ], + [ + 7.212, + 1 + ], + [ + 7.417, + 1 + ], + [ + 7.623, + 1 + ], + [ + 7.827, + 1 + ], + [ + 8.032, + 1 + ], + [ + 8.237, + 1 + ], + [ + 8.441, + 1 + ], + [ + 8.646, + 1 + ], + [ + 8.862, + 1 + ], + [ + 9.066, + 1 + ], + [ + 9.271, + 1 + ], + [ + 9.477, + 1 + ], + [ + 9.682, + 1 + ], + [ + 9.888, + 1 + ], + [ + 10.092, + 1 + ], + [ + 10.297, + 1 + ], + [ + 10.501, + 1 + ], + [ + 10.707, + 1 + ], + [ + 10.911, + 1 + ], + [ + 11.117, + 1 + ], + [ + 11.325, + 1 + ], + [ + 11.53, + 1 + ], + [ + 11.736, + 1 + ], + [ + 11.94, + 1 + ], + [ + 12.144, + 1 + ], + [ + 12.348, + 1 + ], + [ + 12.552, + 1 + ], + [ + 12.757, + 1 + ], + [ + 12.962, + 1 + ], + [ + 13.167, + 1 + ], + [ + 13.372, + 1 + ], + [ + 13.577, + 1 + ], + [ + 13.781, + 1 + ], + [ + 13.986, + 1 + ], + [ + 14.194, + 1 + ], + [ + 14.399, + 1 + ], + [ + 14.603, + 1 + ], + [ + 14.808, + 1 + ], + [ + 15.012, + 1 + ], + [ + 15.216, + 1 + ], + [ + 15.421, + 1 + ], + [ + 15.627, + 1 + ], + [ + 15.832, + 1 + ], + [ + 16.037, + 1 + ], + [ + 16.242, + 1 + ], + [ + 16.446, + 1 + ], + [ + 16.652, + 1 + ], + [ + 16.858, + 1 + ], + [ + 17.062, + 1 + ], + [ + 17.267, + 1 + ], + [ + 17.472, + 1 + ], + [ + 17.676, + 1 + ], + [ + 17.881, + 1 + ], + [ + 18.087, + 1 + ], + [ + 18.292, + 1 + ], + [ + 18.497, + 1 + ], + [ + 18.702, + 1 + ], + [ + 18.907, + 1 + ], + [ + 19.111, + 1 + ], + [ + 19.316, + 1 + ], + [ + 19.521, + 1 + ], + [ + 19.726, + 1 + ], + [ + 19.931, + 1 + ], + [ + 20.137, + 1 + ], + [ + 20.342, + 1 + ], + [ + 20.547, + 1 + ], + [ + 20.752, + 1 + ], + [ + 20.957, + 1 + ], + [ + 21.161, + 1 + ], + [ + 21.365, + 1 + ], + [ + 21.571, + 1 + ], + [ + 21.777, + 1 + ], + [ + 21.982, + 1 + ], + [ + 22.187, + 1 + ], + [ + 22.391, + 1 + ], + [ + 22.598, + 1 + ], + [ + 22.803, + 1 + ], + [ + 23.007, + 1 + ], + [ + 23.211, + 1 + ], + [ + 23.417, + 1 + ], + [ + 23.623, + 1 + ], + [ + 23.827, + 1 + ], + [ + 24.031, + 1 + ], + [ + 24.236, + 1 + ], + [ + 24.44, + 1 + ], + [ + 24.644, + 1 + ], + [ + 24.848, + 1 + ], + [ + 25.054, + 1 + ], + [ + 25.257, + 1 + ], + [ + 25.461, + 1 + ], + [ + 25.666, + 1 + ], + [ + 25.873, + 1 + ], + [ + 26.077, + 1 + ], + [ + 26.281, + 1 + ], + [ + 26.487, + 1 + ], + [ + 26.691, + 1 + ], + [ + 26.894, + 1 + ], + [ + 27.097, + 1 + ], + [ + 27.303, + 1 + ], + [ + 27.508, + 1 + ], + [ + 27.712, + 1 + ], + [ + 27.917, + 1 + ], + [ + 28.123, + 1 + ], + [ + 28.327, + 1 + ], + [ + 28.532, + 1 + ], + [ + 28.739, + 1 + ], + [ + 28.944, + 1 + ], + [ + 29.147, + 1 + ], + [ + 29.352, + 1 + ], + [ + 29.556, + 1 + ], + [ + 29.76, + 1 + ], + [ + 29.965, + 1 + ], + [ + 30.168, + 1 + ], + [ + 30.373, + 1 + ], + [ + 30.577, + 1 + ], + [ + 30.781, + 1 + ], + [ + 30.987, + 1 + ], + [ + 31.192, + 1 + ], + [ + 31.396, + 1 + ], + [ + 31.6, + 1 + ], + [ + 31.811, + 1 + ], + [ + 32.017, + 1 + ], + [ + 32.223, + 1 + ], + [ + 32.43, + 1 + ], + [ + 32.635, + 1 + ], + [ + 32.842, + 1 + ], + [ + 33.048, + 1 + ], + [ + 33.252, + 1 + ], + [ + 33.457, + 1 + ], + [ + 33.663, + 1 + ], + [ + 33.867, + 1 + ], + [ + 34.075, + 1 + ], + [ + 34.279, + 1 + ], + [ + 34.483, + 1 + ], + [ + 34.688, + 1 + ], + [ + 34.893, + 1 + ], + [ + 35.115, + 1 + ], + [ + 35.32, + 1 + ], + [ + 35.527, + 1 + ], + [ + 35.733, + 1 + ], + [ + 35.941, + 1 + ], + [ + 36.151, + 1 + ], + [ + 36.358, + 1 + ], + [ + 36.568, + 1 + ], + [ + 36.776, + 1 + ], + [ + 36.981, + 1 + ], + [ + 37.189, + 1 + ], + [ + 37.395, + 1 + ], + [ + 37.601, + 1 + ], + [ + 37.806, + 1 + ], + [ + 38.011, + 1 + ], + [ + 38.216, + 1 + ], + [ + 38.422, + 1 + ], + [ + 38.626, + 1 + ], + [ + 38.831, + 1 + ], + [ + 39.036, + 1 + ], + [ + 39.24, + 1 + ], + [ + 39.445, + 1 + ], + [ + 39.651, + 1 + ], + [ + 39.856, + 1 + ], + [ + 40.067, + 1 + ], + [ + 40.275, + 1 + ], + [ + 40.483, + 1 + ], + [ + 40.691, + 1 + ], + [ + 40.898, + 1 + ], + [ + 41.103, + 1 + ], + [ + 41.307, + 1 + ], + [ + 41.511, + 1 + ], + [ + 41.718, + 1 + ], + [ + 41.922, + 1 + ], + [ + 42.127, + 1 + ], + [ + 42.333, + 1 + ], + [ + 42.54, + 1 + ], + [ + 42.751, + 1 + ], + [ + 42.961, + 1 + ], + [ + 43.168, + 1 + ], + [ + 43.372, + 1 + ], + [ + 43.577, + 1 + ], + [ + 43.781, + 1 + ], + [ + 43.986, + 1 + ], + [ + 44.19, + 1 + ], + [ + 44.395, + 1 + ], + [ + 44.6, + 1 + ], + [ + 44.806, + 1 + ], + [ + 45.01, + 1 + ], + [ + 45.214, + 1 + ], + [ + 45.419, + 1 + ], + [ + 45.623, + 1 + ], + [ + 45.827, + 1 + ], + [ + 46.032, + 1 + ], + [ + 46.238, + 1 + ], + [ + 46.442, + 1 + ], + [ + 46.646, + 1 + ], + [ + 46.855, + 1 + ], + [ + 47.061, + 1 + ], + [ + 47.269, + 1 + ], + [ + 47.473, + 1 + ], + [ + 47.678, + 1 + ], + [ + 47.883, + 1 + ], + [ + 48.087, + 1 + ], + [ + 48.291, + 1 + ], + [ + 48.496, + 1 + ], + [ + 48.7, + 1 + ], + [ + 48.904, + 1 + ], + [ + 49.111, + 1 + ], + [ + 49.314, + 1 + ], + [ + 49.519, + 1 + ], + [ + 49.724, + 1 + ], + [ + 49.928, + 1 + ], + [ + 50.132, + 1 + ], + [ + 50.338, + 1 + ], + [ + 50.542, + 1 + ], + [ + 50.746, + 1 + ], + [ + 50.951, + 1 + ], + [ + 51.155, + 1 + ], + [ + 51.359, + 1 + ], + [ + 51.564, + 1 + ], + [ + 51.768, + 1 + ], + [ + 51.973, + 1 + ], + [ + 52.177, + 1 + ], + [ + 52.38, + 1 + ], + [ + 52.585, + 1 + ], + [ + 52.79, + 1 + ], + [ + 52.995, + 1 + ], + [ + 53.199, + 1 + ], + [ + 53.403, + 1 + ], + [ + 53.608, + 1 + ], + [ + 53.812, + 1 + ], + [ + 54.02, + 1 + ], + [ + 54.23, + 1 + ], + [ + 54.436, + 1 + ], + [ + 54.645, + 1 + ], + [ + 54.855, + 1 + ], + [ + 55.065, + 1 + ], + [ + 55.273, + 1 + ], + [ + 55.477, + 1 + ], + [ + 55.684, + 1 + ], + [ + 55.891, + 1 + ], + [ + 56.095, + 1 + ], + [ + 56.301, + 1 + ], + [ + 56.505, + 1 + ], + [ + 56.71, + 1 + ], + [ + 56.915, + 1 + ], + [ + 57.12, + 1 + ], + [ + 57.324, + 1 + ], + [ + 57.529, + 1 + ], + [ + 57.733, + 1 + ], + [ + 57.938, + 1 + ], + [ + 58.144, + 1 + ], + [ + 58.347, + 1 + ], + [ + 58.552, + 1 + ], + [ + 58.759, + 1 + ], + [ + 58.966, + 1 + ], + [ + 59.175, + 1 + ], + [ + 59.386, + 1 + ], + [ + 59.592, + 1 + ], + [ + 59.799, + 1 + ], + [ + 60.004, 1 ] ] diff --git a/evidence/level4/failover-sweep.md b/evidence/level4/failover-sweep.md index 13f4725..0039b2b 100644 --- a/evidence/level4/failover-sweep.md +++ b/evidence/level4/failover-sweep.md @@ -2,8 +2,8 @@ | реплика | контейнер | попыток пробы | успешных | ошибок в отказе | остановка контейнера, s | первый успех после остановки, s | реплика вернулась, s | LB DOWN | RPS до | RPS в отказе | RPS после | k6 RPS | k6 ошибок | дубли | потери | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| 438cdb0e0de6 | 438cdb0e0de6 | 289 | 289 | 0 | 3.23 | 0.2 | 3.34 | 18:37:03 | 4.87 | 0.0 | 4.83 | 200.02 | 0 | 0 | 0 | -| 63edcffba23f | 63edcffba23f | 290 | 290 | 0 | 3.24 | 0.2 | 2.84 | 18:38:04 | 4.87 | 0.0 | 4.85 | 200.02 | 0 | 0 | 0 | +| 3adeb4c637a0 | 3adeb4c637a0 | 295 | 295 | 0 | 3.22 | 0.2 | 2.84 | 20:18:39 | 4.87 | 0.0 | 4.87 | 200.02 | 0 | 0 | 0 | +| e4b62a206858 | e4b62a206858 | 293 | 293 | 0 | 3.2 | 0.2 | 3.86 | 20:19:41 | 4.87 | 0.0 | 4.88 | 200.02 | 0 | 0 | 0 | Проба: последовательные POST с интервалом 0.2 s, при неопределённом результате повтор с тем же Idempotency-Key (как у реального клиента). Окно отсутствия реплики считается от выхода контейнера до первого успешного ответа после него; дрейн перед остановкой специально переживает обнаружение отказа LB, поэтому клиент не обязан видеть ошибки. Потери и дубли проверяются по истории чата. diff --git a/evidence/level4/rolling-update.json b/evidence/level4/rolling-update.json index 893cfa8..37183b4 100644 --- a/evidence/level4/rolling-update.json +++ b/evidence/level4/rolling-update.json @@ -1,46 +1,46 @@ { - "build_id": "20261015T184401Z", + "build_id": "final-20261015T202034Z", "build_stderr_tail": [ - "#15 naming to docker.io/library/mtusi-messenger:lab3 done", - "#15 unpacking to docker.io/library/mtusi-messenger:lab3 0.0s done", - "#15 DONE 0.1s" + "#15 unpacking to docker.io/library/mtusi-messenger:lab3", + "#15 unpacking to docker.io/library/mtusi-messenger:lab3 1.7s done", + "#15 DONE 4.8s" ], "image": "mtusi-messenger:lab3", - "image_id": "sha256:9a78a626f3be95235160d38b9389f0045fa4092731cc184605bc3d5ccd580a55", + "image_id": "sha256:8facbd10b145de8a8d0c5f2f03ef0843d5227e1c9b22177ad2039f3469d29016", "k6": { "achieved_rps": 200.01, "failed": 0 }, "migrate": { - "after": "2026-10-15T18:32:35.902006021Z", - "before": "2026-10-15T18:32:35.902006021Z", + "after": "2026-10-15T20:17:45.344168816Z", + "before": "2026-10-15T20:17:45.344168816Z", "restarted": false }, "probe": { - "attempts": 880, + "attempts": 884, "failures": 0, "server_5xx": 0 }, "replicas": 2, "steps": [ { - "migrate_started_at": "2026-10-15T18:32:35.902006021Z", - "new_build_id": "20261015T184401Z", - "new_container": "69f2e591c9a3", - "new_instance": "69f2e591c9a3", - "old_build_id": "20261015T184032Z", - "old_container": "f8ec14fe702a", - "old_instance": "f8ec14fe702a", + "migrate_started_at": "2026-10-15T20:17:45.344168816Z", + "new_build_id": "final-20261015T202034Z", + "new_container": "7dbde7f3e6c6", + "new_instance": "7dbde7f3e6c6", + "old_build_id": "unknown", + "old_container": "3adeb4c637a0", + "old_instance": "3adeb4c637a0", "step": 1 }, { - "migrate_started_at": "2026-10-15T18:32:35.902006021Z", - "new_build_id": "20261015T184401Z", - "new_container": "5d2fd511f464", - "new_instance": "5d2fd511f464", - "old_build_id": "20261015T184032Z", - "old_container": "61d3c1458343", - "old_instance": "61d3c1458343", + "migrate_started_at": "2026-10-15T20:17:45.344168816Z", + "new_build_id": "final-20261015T202034Z", + "new_container": "e4acf5ca4ae3", + "new_instance": "e4acf5ca4ae3", + "old_build_id": "unknown", + "old_container": "e4b62a206858", + "old_instance": "e4b62a206858", "step": 2 } ], diff --git a/evidence/level4/rolling-update.md b/evidence/level4/rolling-update.md index f4aedb2..0fade14 100644 --- a/evidence/level4/rolling-update.md +++ b/evidence/level4/rolling-update.md @@ -2,8 +2,8 @@ | шаг | было | BUILD_ID до | стало | BUILD_ID после | | --- | --- | --- | --- | --- | -| 1 | f8ec14fe702a | 20261015T184032Z | 69f2e591c9a3 | 20261015T184401Z | -| 2 | 61d3c1458343 | 20261015T184032Z | 5d2fd511f464 | 20261015T184401Z | +| 1 | 3adeb4c637a0 | unknown | 7dbde7f3e6c6 | final-20261015T202034Z | +| 2 | e4b62a206858 | unknown | e4acf5ca4ae3 | final-20261015T202034Z | -5xx от API во время rolling update: 0 из 880 попыток +5xx от API во время rolling update: 0 из 884 попыток Миграции перезапускались: False diff --git a/evidence/level4/spread.json b/evidence/level4/spread.json index 64948a6..fd2bc5a 100644 --- a/evidence/level4/spread.json +++ b/evidence/level4/spread.json @@ -1,26 +1,26 @@ { - "captured_at": "2026-10-15T18:30:44Z", + "captured_at": "2026-10-15T20:17:44Z", "distinct_instances": 2, "instances": { - "4579bce31e8d": 20, - "77b49a240814": 20 + "51d5b56a7909": 20, + "ef9aeeda3cf8": 20 }, "replicas": { - "4579bce31e8d": { - "container": "4579bce31e8d", + "51d5b56a7909": { + "container": "51d5b56a7909", "cpus": 1.5, - "image_id": "sha256:009e36e05eb149dc11222453ff740b9654aaaac40282037279b74eab8d9a7c86", + "image_id": "sha256:b57c58f89a8839b6803a4e815b505a5138a2390173a87ba597d4614e30245c5b", "memory_mb": 768, "name": "mtusi-lab3-api-1", - "started_at": "2026-10-15T18:29:41.666917735Z" + "started_at": "2026-10-15T20:17:14.787282931Z" }, - "77b49a240814": { - "container": "77b49a240814", + "ef9aeeda3cf8": { + "container": "ef9aeeda3cf8", "cpus": 1.5, - "image_id": "sha256:009e36e05eb149dc11222453ff740b9654aaaac40282037279b74eab8d9a7c86", + "image_id": "sha256:b57c58f89a8839b6803a4e815b505a5138a2390173a87ba597d4614e30245c5b", "memory_mb": 768, "name": "mtusi-lab3-api-2", - "started_at": "2026-10-15T18:26:23.220069678Z" + "started_at": "2026-10-15T20:17:14.888556357Z" } }, "requests": 40 diff --git a/evidence/track1/raw/failover#3adeb4c637a0.k6.json b/evidence/track1/raw/failover#3adeb4c637a0.k6.json new file mode 100644 index 0000000..145ccd1 --- /dev/null +++ b/evidence/track1/raw/failover#3adeb4c637a0.k6.json @@ -0,0 +1,35 @@ +{ + "base_url": "http://haproxy:8080", + "checks": { + "fails": 0, + "passes": 0 + }, + "dropped_iterations": 0, + "http_req_duration_p95": 6.74, + "http_reqs": 11631, + "iterations": 11601, + "kind": "steady", + "label": "failover#3adeb4c637a0", + "mode": "write", + "profile": "spread", + "steps": [ + { + "achieved_rps": 200.02, + "avg": 4, + "duration_s": 58, + "failed": 0, + "max": 69.68, + "p50": 3.48, + "p95": 6.71, + "p99": 8.95, + "rate": 200, + "requests": 11601, + "vus_active": 1 + } + ], + "vus_configured": { + "max": 200, + "preAllocated": 60 + }, + "vus_max": 60 +} diff --git a/evidence/track1/raw/failover#3adeb4c637a0.manifest.json b/evidence/track1/raw/failover#3adeb4c637a0.manifest.json new file mode 100644 index 0000000..58b86bb --- /dev/null +++ b/evidence/track1/raw/failover#3adeb4c637a0.manifest.json @@ -0,0 +1,101 @@ +{ + "commit": "eabb1d2909a02aa6020694769ca1304cd3de63a5", + "dataset": { + "chat_members": 84, + "chats": 33, + "idempotency_keys": 11906, + "messages": 11906, + "users": 18 + }, + "finished_at": "2026-10-15T20:19:31Z", + "generator": { + "cpus": 1.5, + "max_vus": "200", + "memory": "256m", + "pre_vus": "60" + }, + "host": { + "background": { + "load_average": [ + 4.19, + 4.91, + 6.42 + ], + "mem_available_mib": 4944, + "mem_total_mib": 15771, + "quiesced": false + }, + "cpus": 8 + }, + "kind": "steady", + "label": "failover#3adeb4c637a0", + "mode": "write", + "phases": { + "first_success_after_exit_s": 0.2, + "lb_down_at": "2026-10-15T20:18:39Z", + "probe_offsets": { + "container_exit": 11.22, + "last_sample": 60.48, + "restore_command": 31.42, + "stop_command": 8.0 + }, + "replica_rejoined_s": 2.84, + "restore_at": "2026-10-15T20:19:02Z", + "stop_command_at": "2026-10-15T20:18:41Z", + "stop_command_took_s": 3.22, + "stop_exit_code": 0 + }, + "profile": "spread", + "quiesce": null, + "replicas": { + "3adeb4c637a0": { + "container": "3adeb4c637a0", + "cpus": 1.5, + "image_id": "sha256:b57c58f89a8839b6803a4e815b505a5138a2390173a87ba597d4614e30245c5b", + "memory_mb": 768, + "name": "mtusi-lab3-api-2", + "started_at": "2026-10-15T20:19:02.244821917Z" + }, + "e4b62a206858": { + "container": "e4b62a206858", + "cpus": 1.5, + "image_id": "sha256:b57c58f89a8839b6803a4e815b505a5138a2390173a87ba597d4614e30245c5b", + "memory_mb": 768, + "name": "mtusi-lab3-api-1", + "started_at": "2026-10-15T20:18:23.977538795Z" + } + }, + "sample_count": 1503, + "sampler_errors": [], + "stand": { + "API_IMAGE": "mtusi-messenger:lab3", + "API_REPLICAS": 2, + "API_TOTAL_CPUS": 3.0, + "API_TOTAL_MEMORY_MB": 1536, + "BUILD_ID": "base", + "DATABASE_POOL_MAX": 10, + "LB_CPUS": 0.25, + "LB_MEMORY": "128m", + "LOADGEN_CPUS": 1.5, + "LOADGEN_MEMORY": "256m", + "PG_CPUS": 1.5, + "PG_MEMORY": "512m", + "REDIS_CPUS": 0.25, + "REDIS_MEMORY": "128m" + }, + "started_at": "2026-10-15T20:18:30Z", + "step_seconds": 58, + "steps": [ + 200 + ], + "versions": { + "haproxy": "haproxy:3.2-alpine", + "haproxy_runtime": "HAProxy version 3.2.26-3e0f11a 2026/10/14 - https://haproxy.org/", + "k6": "k6 v0.54.0 (commit/baba871c8a, go1.23.1, linux/amd64)", + "node_host": "v25.7.0", + "postgres": "postgres:16.4-alpine", + "postgres_runtime": "postgres (PostgreSQL) 16.4", + "redis": "redis:7.4.2-alpine", + "redis_runtime": "Redis server v=7.4.2 sha=00000000:0 malloc=jemalloc-5.3.0 bits=64 build=9d14eaf5f599583e" + } +} diff --git a/evidence/track1/raw/failover#3adeb4c637a0.phases.json b/evidence/track1/raw/failover#3adeb4c637a0.phases.json new file mode 100644 index 0000000..eb68930 --- /dev/null +++ b/evidence/track1/raw/failover#3adeb4c637a0.phases.json @@ -0,0 +1,15 @@ +{ + "first_success_after_exit_s": 0.2, + "lb_down_at": "2026-10-15T20:18:39Z", + "probe_offsets": { + "container_exit": 11.22, + "last_sample": 60.48, + "restore_command": 31.42, + "stop_command": 8.0 + }, + "replica_rejoined_s": 2.84, + "restore_at": "2026-10-15T20:19:02Z", + "stop_command_at": "2026-10-15T20:18:41Z", + "stop_command_took_s": 3.22, + "stop_exit_code": 0 +} diff --git a/evidence/track1/raw/failover#3adeb4c637a0.samples.jsonl.gz b/evidence/track1/raw/failover#3adeb4c637a0.samples.jsonl.gz new file mode 100644 index 0000000..50379be Binary files /dev/null and b/evidence/track1/raw/failover#3adeb4c637a0.samples.jsonl.gz differ diff --git a/evidence/track1/raw/failover#e4b62a206858.k6.json b/evidence/track1/raw/failover#e4b62a206858.k6.json new file mode 100644 index 0000000..0ad1ef3 --- /dev/null +++ b/evidence/track1/raw/failover#e4b62a206858.k6.json @@ -0,0 +1,35 @@ +{ + "base_url": "http://haproxy:8080", + "checks": { + "fails": 0, + "passes": 0 + }, + "dropped_iterations": 0, + "http_req_duration_p95": 6.05, + "http_reqs": 11631, + "iterations": 11601, + "kind": "steady", + "label": "failover#e4b62a206858", + "mode": "write", + "profile": "spread", + "steps": [ + { + "achieved_rps": 200.02, + "avg": 3.8, + "duration_s": 58, + "failed": 0, + "max": 80.8, + "p50": 3.43, + "p95": 6.02, + "p99": 7.83, + "rate": 200, + "requests": 11601, + "vus_active": 1 + } + ], + "vus_configured": { + "max": 200, + "preAllocated": 60 + }, + "vus_max": 60 +} diff --git a/evidence/track1/raw/failover#e4b62a206858.manifest.json b/evidence/track1/raw/failover#e4b62a206858.manifest.json new file mode 100644 index 0000000..b453625 --- /dev/null +++ b/evidence/track1/raw/failover#e4b62a206858.manifest.json @@ -0,0 +1,101 @@ +{ + "commit": "eabb1d2909a02aa6020694769ca1304cd3de63a5", + "dataset": { + "chat_members": 157, + "chats": 58, + "idempotency_keys": 23800, + "messages": 23800, + "users": 22 + }, + "finished_at": "2026-10-15T20:20:33Z", + "generator": { + "cpus": 1.5, + "max_vus": "200", + "memory": "256m", + "pre_vus": "60" + }, + "host": { + "background": { + "load_average": [ + 4.31, + 4.79, + 6.28 + ], + "mem_available_mib": 4928, + "mem_total_mib": 15771, + "quiesced": false + }, + "cpus": 8 + }, + "kind": "steady", + "label": "failover#e4b62a206858", + "mode": "write", + "phases": { + "first_success_after_exit_s": 0.2, + "lb_down_at": "2026-10-15T20:19:41Z", + "probe_offsets": { + "container_exit": 11.2, + "last_sample": 60.0, + "restore_command": 31.4, + "stop_command": 8.0 + }, + "replica_rejoined_s": 3.86, + "restore_at": "2026-10-15T20:20:04Z", + "stop_command_at": "2026-10-15T20:19:44Z", + "stop_command_took_s": 3.2, + "stop_exit_code": 0 + }, + "profile": "spread", + "quiesce": null, + "replicas": { + "3adeb4c637a0": { + "container": "3adeb4c637a0", + "cpus": 1.5, + "image_id": "sha256:b57c58f89a8839b6803a4e815b505a5138a2390173a87ba597d4614e30245c5b", + "memory_mb": 768, + "name": "mtusi-lab3-api-2", + "started_at": "2026-10-15T20:19:02.244821917Z" + }, + "e4b62a206858": { + "container": "e4b62a206858", + "cpus": 1.5, + "image_id": "sha256:b57c58f89a8839b6803a4e815b505a5138a2390173a87ba597d4614e30245c5b", + "memory_mb": 768, + "name": "mtusi-lab3-api-1", + "started_at": "2026-10-15T20:20:04.61530436Z" + } + }, + "sample_count": 1491, + "sampler_errors": [], + "stand": { + "API_IMAGE": "mtusi-messenger:lab3", + "API_REPLICAS": 2, + "API_TOTAL_CPUS": 3.0, + "API_TOTAL_MEMORY_MB": 1536, + "BUILD_ID": "base", + "DATABASE_POOL_MAX": 10, + "LB_CPUS": 0.25, + "LB_MEMORY": "128m", + "LOADGEN_CPUS": 1.5, + "LOADGEN_MEMORY": "256m", + "PG_CPUS": 1.5, + "PG_MEMORY": "512m", + "REDIS_CPUS": 0.25, + "REDIS_MEMORY": "128m" + }, + "started_at": "2026-10-15T20:19:33Z", + "step_seconds": 58, + "steps": [ + 200 + ], + "versions": { + "haproxy": "haproxy:3.2-alpine", + "haproxy_runtime": "HAProxy version 3.2.26-3e0f11a 2026/10/14 - https://haproxy.org/", + "k6": "k6 v0.54.0 (commit/baba871c8a, go1.23.1, linux/amd64)", + "node_host": "v25.7.0", + "postgres": "postgres:16.4-alpine", + "postgres_runtime": "postgres (PostgreSQL) 16.4", + "redis": "redis:7.4.2-alpine", + "redis_runtime": "Redis server v=7.4.2 sha=00000000:0 malloc=jemalloc-5.3.0 bits=64 build=9d14eaf5f599583e" + } +} diff --git a/evidence/track1/raw/failover#e4b62a206858.phases.json b/evidence/track1/raw/failover#e4b62a206858.phases.json new file mode 100644 index 0000000..711094a --- /dev/null +++ b/evidence/track1/raw/failover#e4b62a206858.phases.json @@ -0,0 +1,15 @@ +{ + "first_success_after_exit_s": 0.2, + "lb_down_at": "2026-10-15T20:19:41Z", + "probe_offsets": { + "container_exit": 11.2, + "last_sample": 60.0, + "restore_command": 31.4, + "stop_command": 8.0 + }, + "replica_rejoined_s": 3.86, + "restore_at": "2026-10-15T20:20:04Z", + "stop_command_at": "2026-10-15T20:19:44Z", + "stop_command_took_s": 3.2, + "stop_exit_code": 0 +} diff --git a/evidence/track1/raw/failover#e4b62a206858.samples.jsonl.gz b/evidence/track1/raw/failover#e4b62a206858.samples.jsonl.gz new file mode 100644 index 0000000..a30d975 Binary files /dev/null and b/evidence/track1/raw/failover#e4b62a206858.samples.jsonl.gz differ diff --git a/evidence/track1/raw/rolling.k6.json b/evidence/track1/raw/rolling.k6.json index ab524e1..6f15079 100644 --- a/evidence/track1/raw/rolling.k6.json +++ b/evidence/track1/raw/rolling.k6.json @@ -5,7 +5,7 @@ "passes": 0 }, "dropped_iterations": 0, - "http_req_duration_p95": 8.69, + "http_req_duration_p95": 6.55, "http_reqs": 36031, "iterations": 36001, "kind": "steady", @@ -15,17 +15,21 @@ "steps": [ { "achieved_rps": 200.01, - "avg": 5.17, + "avg": 4.17, "duration_s": 180, "failed": 0, - "max": 104.1, - "p50": 4.34, - "p95": 8.69, - "p99": 10.7, + "max": 117.52, + "p50": 3.55, + "p95": 6.55, + "p99": 9.22, "rate": 200, "requests": 36001, "vus_active": 1 } ], + "vus_configured": { + "max": 200, + "preAllocated": 60 + }, "vus_max": 60 } diff --git a/evidence/track1/raw/rolling.manifest.json b/evidence/track1/raw/rolling.manifest.json index e072cc7..6966294 100644 --- a/evidence/track1/raw/rolling.manifest.json +++ b/evidence/track1/raw/rolling.manifest.json @@ -1,21 +1,27 @@ { - "commit": "410e1ea3ad1b12b1bc650484c3531e6bf76479be", + "commit": "eabb1d2909a02aa6020694769ca1304cd3de63a5", "dataset": { - "chat_members": 1150, - "chats": 401, - "idempotency_keys": 178742, - "messages": 178742, - "users": 115 + "chat_members": 230, + "chats": 83, + "idempotency_keys": 60685, + "messages": 60685, + "users": 26 + }, + "finished_at": "2026-10-15T20:24:00Z", + "generator": { + "cpus": "1.5", + "max_vus": "200", + "memory": "256m", + "pre_vus": "60" }, - "finished_at": "2026-10-15T18:47:06Z", "host": { "background": { "load_average": [ - 5.74, - 5.24, - 4.94 + 4.52, + 4.76, + 5.97 ], - "mem_available_mib": 6376, + "mem_available_mib": 4864, "mem_total_mib": 15771, "quiesced": false }, @@ -27,23 +33,23 @@ "phases": { "steps": [ { - "migrate_started_at": "2026-10-15T18:32:35.902006021Z", - "new_build_id": "20261015T184401Z", - "new_container": "69f2e591c9a3", - "new_instance": "69f2e591c9a3", - "old_build_id": "20261015T184032Z", - "old_container": "f8ec14fe702a", - "old_instance": "f8ec14fe702a", + "migrate_started_at": "2026-10-15T20:17:45.344168816Z", + "new_build_id": "final-20261015T202034Z", + "new_container": "7dbde7f3e6c6", + "new_instance": "7dbde7f3e6c6", + "old_build_id": "unknown", + "old_container": "3adeb4c637a0", + "old_instance": "3adeb4c637a0", "step": 1 }, { - "migrate_started_at": "2026-10-15T18:32:35.902006021Z", - "new_build_id": "20261015T184401Z", - "new_container": "5d2fd511f464", - "new_instance": "5d2fd511f464", - "old_build_id": "20261015T184032Z", - "old_container": "61d3c1458343", - "old_instance": "61d3c1458343", + "migrate_started_at": "2026-10-15T20:17:45.344168816Z", + "new_build_id": "final-20261015T202034Z", + "new_container": "e4acf5ca4ae3", + "new_instance": "e4acf5ca4ae3", + "old_build_id": "unknown", + "old_container": "e4b62a206858", + "old_instance": "e4b62a206858", "step": 2 } ] @@ -51,24 +57,24 @@ "profile": "spread", "quiesce": null, "replicas": { - "5d2fd511f464": { - "container": "5d2fd511f464", + "7dbde7f3e6c6": { + "container": "7dbde7f3e6c6", "cpus": 1.5, - "image_id": "sha256:9a78a626f3be95235160d38b9389f0045fa4092731cc184605bc3d5ccd580a55", + "image_id": "sha256:8facbd10b145de8a8d0c5f2f03ef0843d5227e1c9b22177ad2039f3469d29016", "memory_mb": 768, - "name": "mtusi-lab3-api-7", - "started_at": "2026-10-15T18:44:19.559155405Z" + "name": "mtusi-lab3-api-2", + "started_at": "2026-10-15T20:21:02.060443486Z" }, - "69f2e591c9a3": { - "container": "69f2e591c9a3", + "e4acf5ca4ae3": { + "container": "e4acf5ca4ae3", "cpus": 1.5, - "image_id": "sha256:9a78a626f3be95235160d38b9389f0045fa4092731cc184605bc3d5ccd580a55", + "image_id": "sha256:8facbd10b145de8a8d0c5f2f03ef0843d5227e1c9b22177ad2039f3469d29016", "memory_mb": 768, - "name": "mtusi-lab3-api-6", - "started_at": "2026-10-15T18:44:08.141677703Z" + "name": "mtusi-lab3-api-3", + "started_at": "2026-10-15T20:21:13.561623701Z" } }, - "sample_count": 4545, + "sample_count": 4544, "sampler_errors": [], "stand": { "API_CPUS_PER_REPLICA": "1.5", @@ -88,7 +94,7 @@ "REDIS_CPUS": "0.25", "REDIS_MEMORY": "128m" }, - "started_at": "2026-10-15T18:44:04Z", + "started_at": "2026-10-15T20:20:58Z", "step_seconds": 180, "steps": [ 200 diff --git a/evidence/track1/raw/rolling.phases.json b/evidence/track1/raw/rolling.phases.json index 37078de..10b64c7 100644 --- a/evidence/track1/raw/rolling.phases.json +++ b/evidence/track1/raw/rolling.phases.json @@ -1,23 +1,23 @@ { "steps": [ { - "migrate_started_at": "2026-10-15T18:32:35.902006021Z", - "new_build_id": "20261015T184401Z", - "new_container": "69f2e591c9a3", - "new_instance": "69f2e591c9a3", - "old_build_id": "20261015T184032Z", - "old_container": "f8ec14fe702a", - "old_instance": "f8ec14fe702a", + "migrate_started_at": "2026-10-15T20:17:45.344168816Z", + "new_build_id": "final-20261015T202034Z", + "new_container": "7dbde7f3e6c6", + "new_instance": "7dbde7f3e6c6", + "old_build_id": "unknown", + "old_container": "3adeb4c637a0", + "old_instance": "3adeb4c637a0", "step": 1 }, { - "migrate_started_at": "2026-10-15T18:32:35.902006021Z", - "new_build_id": "20261015T184401Z", - "new_container": "5d2fd511f464", - "new_instance": "5d2fd511f464", - "old_build_id": "20261015T184032Z", - "old_container": "61d3c1458343", - "old_instance": "61d3c1458343", + "migrate_started_at": "2026-10-15T20:17:45.344168816Z", + "new_build_id": "final-20261015T202034Z", + "new_container": "e4acf5ca4ae3", + "new_instance": "e4acf5ca4ae3", + "old_build_id": "unknown", + "old_container": "e4b62a206858", + "old_instance": "e4b62a206858", "step": 2 } ] diff --git a/evidence/track1/raw/rolling.samples.jsonl.gz b/evidence/track1/raw/rolling.samples.jsonl.gz index 2856c03..1177962 100644 Binary files a/evidence/track1/raw/rolling.samples.jsonl.gz and b/evidence/track1/raw/rolling.samples.jsonl.gz differ