docs: add attachments evidence and report
- re-capture coverage-exclusion evidence on the lab 4 stand: cache,
auth storage, replay after API recreate, test-persistence,
attachment-failover (stop, kill), attachment-storage
- green evidence/tests: student_test via LB, test_isolated,
failure_checks; smoke, slow transfer, test-failover per replica
- bench_rollout label attachments: e1161f7 -> schema with attachments
under load, no errors, messages = keys
- REPORT.md: attachments architecture, write order, defense answers,
level 3/4 evidence rows, limitations; PLAN.md stage 2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -228,7 +228,7 @@ RustFS. В `.claude/rules/settings.md` MinIO упомянут только пр
|
||||
|---|---|---|---|---|
|
||||
| 0 | — (подготовка) | — | этот план; выбор трека и S3-сервера | план сохранён 2026-10-20; трек 2, SeaweedFS, решения 3 и 7 подтверждены 2026-10-20 |
|
||||
| 1 | `s3-storage` | 3 (+4: bootstrap, права, консоль, приватность) | S3 в стенде, bootstrap, клиент в `src/infra/s3/`, настройки, все стенды знают о S3 | bootstrap дважды без ошибок, анонимный GET / LIST → 403, ключ API не может admin-операций, `make test-persistence` перезапускает и S3; **заархивирован 2026-10-20** — `openspec/changes/archive/2026-10-20-s3-storage/` |
|
||||
| 2 | `attachments` | 3 (+4: тесты) | миграция, домен `attachment`, ручки, `attachment_ids` в сообщениях, тесты | зелёные `make test`, `test-persistence`, `test-failover`, свой сценарий отказа загрузившей реплики; негативные тесты; evidence исключений переснят |
|
||||
| 2 | `attachments` | 3 (+4: тесты) | миграция, домен `attachment`, ручки, `attachment_ids` в сообщениях, тесты | зелёные `make test`, `test-persistence`, `test-failover`, свой сценарий отказа загрузившей реплики; негативные тесты; evidence исключений переснят; **код — `116d8d3`, гонка `bench_restore` — `d56b892`; evidence 2026-10-20 с `d56b892`: `evidence/attachments/`, исключения, `evidence/tests/` (196 / 191 + 5 пропусков / 23 из 23), `test-failover` для обеих реплик, `bench_rollout` `attachments` (ошибок 0, 12 001 = 12 001)** |
|
||||
| 3 | `upload-consistency` | 4 | намерение до PUT, компенсация, уборщик с grace period, тесты отказов | `make failure_checks` с отказами S3 и БД посреди загрузки: нет 500, нет неучтённых объектов старше grace period |
|
||||
| 4 | `upload-memory` | 4 | лимит загрузок на реплику, сроки тела, профиль `uploads` | замер ≥ 10 одновременных загрузок: пик памяти ограничен, механизм объяснён |
|
||||
| 5 | `track2-reconciler` | 5 | reconciler: dry-run / apply, классы расхождений, внедрение ошибок, доказательство безопасности | ошибки внесены с обеих сторон, обнаружены, исправлены; повторный запуск находит 0; под загрузками ничего подтверждённого не удалено |
|
||||
|
||||
@@ -55,10 +55,11 @@ TODO: схема процессов/хранилищ, источник исти
|
||||
### Роль S3 и её граница
|
||||
|
||||
S3 хранит только байты вложений, метаданные лежат в PostgreSQL. Пишет и
|
||||
читает S3 только API своим ключом. Ручки вложений, `attachment_ids`,
|
||||
компенсация и уборщик — следующие changes (`attachments`,
|
||||
`upload-consistency`, `upload-memory`). Change `s3-storage` даёт сервер,
|
||||
bootstrap, клиент `src/infra/s3/` и стенды.
|
||||
читает S3 только API своим ключом. Change `s3-storage` дал сервер,
|
||||
bootstrap, клиент `src/infra/s3/` и стенды, change `attachments` —
|
||||
ручки вложений и `attachment_ids` (раздел «Вложения» ниже). Компенсация
|
||||
и уборщик зависших строк — `upload-consistency`, лимит одновременных
|
||||
загрузок — `upload-memory`.
|
||||
|
||||
Сервер — SeaweedFS 4.48. Выбор и проба — `PLAN.md` (решение 1) и
|
||||
design change `s3-storage`. Три места, где SeaweedFS по умолчанию не
|
||||
@@ -145,29 +146,159 @@ compose up -d s3-bootstrap` или `make run_docker`.
|
||||
| `S3_REGION` | `[s3] region` в `config.toml` (`us-east-1`, нужен подписи SigV4) |
|
||||
| `S3_BUCKET` | `S3__BUCKET` в `.env`: читают приложение и bootstrap |
|
||||
| `S3_ACCESS_KEY_ID_FILE`, `S3_SECRET_ACCESS_KEY_FILE` | мосты `S3__ACCESS_KEY_ID=${S3_API_ACCESS_KEY}` и `S3__SECRET_ACCESS_KEY=${S3_API_SECRET_KEY}`: значения из `.env`, а не файлы, как с `DATABASE_URL_FILE` |
|
||||
| `MAX_UPLOAD_BYTES`, `MAX_IMAGE_PIXELS` | change `attachments` |
|
||||
| `MAX_UPLOAD_BYTES` | константа `ATTACHMENT_MAX_BYTES` = 10 485 760 в `src/utils/constants.py`, не настройка |
|
||||
| `MAX_IMAGE_PIXELS` | константа `ATTACHMENT_MAX_PIXELS` = 40 000 000 там же, не настройка |
|
||||
|
||||
Лимиты вложений задаёт контракт, они одинаковы на всех установках,
|
||||
поэтому ни окружение, ни `.env`, ни `config.toml` их не меняют (спек
|
||||
`service-configuration`, «Лимиты вложений — константы контракта»;
|
||||
design change `attachments`, решение 10).
|
||||
|
||||
### Вложения (change `attachments`)
|
||||
|
||||
```
|
||||
POST /chats/{id}/attachments?filename=… (сырые байты PNG/JPEG)
|
||||
└─ вход: короткая сессия БД (CurrentUserBrief)
|
||||
└─ членство (404) → Content-Type (415) → Content-Length (413)
|
||||
└─ тело со счётчиком (413 сразу за 10 MiB) → структура и SHA-256 в потоке (415 / 413)
|
||||
└─ tx1: строка attachments, status=pending, object_key ── commit
|
||||
└─ PUT <база>/attachments/<id>/original в S3
|
||||
└─ tx2: pending → ready ── commit
|
||||
└─ 201 Attachment (status=ready)
|
||||
GET /attachments/{id} → строка ready + членство (404 иначе)
|
||||
GET /attachments/{id}/content → GET из S3 потоком (StreamingResponse), Content-Length, nosniff
|
||||
POST /chats/{id}/messages {text, attachment_ids}
|
||||
└─ членство → вложения: все ready и этого чата (404 до захвата ключа)
|
||||
└─ ключ → сообщение → message_attachments (position) ── один commit
|
||||
```
|
||||
|
||||
**Порядок записи и отказы** (design, решение 1). Объект, который
|
||||
записывает API, известен PostgreSQL до записи: строка `pending` с
|
||||
`object_key` фиксируется до PUT.
|
||||
|
||||
| Где отказ | Ответ | Что остаётся |
|
||||
| --- | --- | --- |
|
||||
| проверки и чтение тела (401, 400, 404, 413, 415, обрыв) | 4xx | ничего — проверено `test_rejected_uploads_leave_nothing` |
|
||||
| tx1 (БД) | 503 | ничего |
|
||||
| PUT (S3 недоступен, срок, отказ) | 503 | строка `pending`; объекта нет или он записан после срока |
|
||||
| tx2 (БД) | 503 | строка `pending` и объект |
|
||||
|
||||
Строку `pending` API не показывает (метаданные, байты и ссылка в
|
||||
сообщении — 404). Её и объект уберёт уборщик с grace period (change
|
||||
`upload-consistency`, ещё не сделан); пока зависшие строки видны
|
||||
счётчиком hook стенда отказов.
|
||||
|
||||
**Проверка изображения — структурная, без декодирования пикселей**
|
||||
(`src/utils/image.py`, только stdlib; решение 2, подтверждено
|
||||
пользователем):
|
||||
- тип — по сигнатуре; `Content-Type` запроса должен совпасть, иначе 415
|
||||
(подмена MIME);
|
||||
- размеры — из IHDR / SOF до распаковки: больше 40 млн пикселей — 413;
|
||||
- PNG: CRC каждого чанка, IHDR первым, нет неизвестных критичных чанков,
|
||||
IDAT подряд, поток zlib распаковывается блоками по 1 MiB и ровно в
|
||||
объём по размерам (больше — «бомба», 415 сразу), типы фильтров строк,
|
||||
IEND; JPEG: маркеры и длины сегментов, один SOF, скан, EOI;
|
||||
- байты после IEND / EOI — часть оригинала (входят в размер и SHA-256);
|
||||
- пик памяти проверки ≤ 3,2 MiB при любом числе пикселей; «бомба»
|
||||
100 × 100 → 200 MiB отклоняется за 1 мс; корпус 12 000 файлов macOS —
|
||||
ни одного ложного отказа (design change `attachments`, «Уточнения
|
||||
apply», п. 1).
|
||||
|
||||
**Схема** (ревизия `ef25d50b01a8`, решение 3): `attachments` (составной
|
||||
FK `(chat_id, owner_id) → chat_members` — владелец участник чата; CHECK
|
||||
размера, SHA-256 32 байта, типа, статуса, длины и символов `filename`;
|
||||
уникальный `object_key`) и `message_attachments` (PK `(message_id,
|
||||
position)`, 0–9, уникальная пара сообщение–вложение, индекс по
|
||||
`attachment_id`, FK без каскада — вложение со ссылкой не удалить).
|
||||
Ограничение текста сообщения ослаблено с 1–4000 до 0–4000 `NOT VALID`:
|
||||
пустой текст допустим с вложением, правило «текст или вложение» держит
|
||||
сервис. История читает `attachment_ids` коррелированным подзапросом
|
||||
`ARRAY(…)` по `pk_message_attachments`, индекс истории прежний.
|
||||
|
||||
**Ключ объекта** — `<имя базы>/attachments/<id>/original`, назначает
|
||||
сервер и хранит в строке. `filename` только отображается: 400 на `/`,
|
||||
`\`, управляющие символы (Cc) и пустое / из пробелов, остальное
|
||||
(включая символы направления текста) хранится как прислано.
|
||||
|
||||
**Без сессии БД на всю передачу** (решение 5). Зависимость
|
||||
`get_db_session` закрывается после отправки ответа, и её срок 6 с
|
||||
охватил бы чтение тела и отдачу потока: медленные 10 MiB оборвались бы
|
||||
на 6-й секунде. Ручки вложений берут короткий вход (`CurrentUserBrief`),
|
||||
сервис открывает сессию на каждый шаг; загрузка 12,4 с и скачивание
|
||||
13,1 с через LB проходят (`evidence/attachments/slow-transfer.md`).
|
||||
|
||||
**Идемпотентность**: хэш тела включает `attachment_ids` только
|
||||
непустым (`str(UUID)`, нижний регистр, порядок значим) — ключ,
|
||||
сохранённый версией без вложений, повторяется после замены реплик; повтор
|
||||
отдаёт те же `attachment_ids`.
|
||||
|
||||
**Отказ S3** — 503 только на загрузке и скачивании байтов; метаданные,
|
||||
сообщения со ссылкой на готовое вложение, история и `/health/ready`
|
||||
работают.
|
||||
|
||||
### Согласования с преподавателем
|
||||
|
||||
| Дата | Договорённость | Где учтено |
|
||||
| --- | --- | --- |
|
||||
| 2026-09-26 | FastAPI с его расхождениями (лишний 422 переведён в 400, порядок кодов: битый JSON — 400 раньше 401, остальная валидация после зависимостей) | `.claude/rules/api-contract.md`, «FastAPI» |
|
||||
| 2026-09-29 | Конфиг через pydantic-settings: окружение → `.env` → `config.toml` → дефолты, `config.toml` монтируется read-only, не в образе | `.claude/rules/settings.md`, `docker.md` |
|
||||
| 2026-10-20 | «Пустой текст» сообщения — и `""`, и из одних пробелов: с хотя бы одним вложением — 201 (текст как прислан), без вложений — 400 | спек `messages`, «Пробелы с вложением»; `CreateMessage`; `test_blank_text_with_attachment` |
|
||||
|
||||
### Ответы на вопросы защиты
|
||||
|
||||
- **Почему файл в Docker volume одной API-реплики не равен объектному
|
||||
хранилищу?** Volume привязан к контейнеру одной реплики: другая
|
||||
реплика его не видит, остановка или пересоздание реплики уносит
|
||||
доступ, а LB шлёт скачивание на любую. Объектное хранилище — общий
|
||||
сервис со своим volume и правами: оригинал отдаёт любая реплика
|
||||
(`evidence/attachments/failover.md`: принял `5f5d4ac50d9a`, после его
|
||||
`stop` / `kill` отдал `847ad74efd7c`), реплики не пишут файлы
|
||||
оригиналов на диск (`docker diff`, `evidence/attachments/storage.md`),
|
||||
и в S3 видно, какой объект какому вложению принадлежит (ключ с `id`).
|
||||
- **Когда можно честно вернуть 201?** Когда оригинал записан в S3 (PUT
|
||||
подтверждён) и метаданные в PostgreSQL зафиксированы в статусе `ready`
|
||||
— после второго commit. До этого — проверки и строка `pending`,
|
||||
которую API никому не показывает. Отказ S3 или БД на любом шаге — 503,
|
||||
а не 201 (`test_upload_with_object_storage_stopped`).
|
||||
- **Что произойдёт, если S3 PUT прошёл, а COMMIT в PostgreSQL — нет?**
|
||||
У нас commit `ready` идёт после PUT, а строка `pending` с ключом
|
||||
объекта зафиксирована **до** PUT. Поэтому объект не «вечный
|
||||
неучтённый»: клиент получил 503, вложения для API нет (404), а в БД
|
||||
есть строка `pending`, по которой уборщик (change
|
||||
`upload-consistency`, grace period) удалит объект и строку. Объект без
|
||||
строки возможен только мимо API — его ищет reconciler трека 2.
|
||||
- **Почему случайный object key и приватный bucket не заменяют ACL
|
||||
приложения?** Приватный bucket закрывает прямой доступ к S3, но
|
||||
скачивает через API любой аутентифицированный пользователь: кто
|
||||
может читать — решает приложение по членству в чате вложения
|
||||
(посторонний — 404, `evidence/attachments/smoke.md`). Случайный ключ —
|
||||
не секрет: он попадает в логи, в БД, в URL, а `attachment_id`
|
||||
клиенты передают друг другу в сообщениях; знание id не даёт права.
|
||||
Ключ API S3 у всех реплик один и права пользователя не различает.
|
||||
|
||||
## Доказательства критериев
|
||||
|
||||
Строки S3 (change `s3-storage`) сняты на стенде `make run_docker`, если
|
||||
не сказано иное: `evidence/s3/restart.md`, `client.md`, `privacy.md`,
|
||||
`evidence/tests/failure-checks.md` и `test-isolated-s3-cleanup.md` — с
|
||||
commit `f57635a` (правки по verify), остальные файлы `evidence/s3/` — с
|
||||
`2e997f9`.
|
||||
|
||||
**До change `attachments` красные:**
|
||||
- публичный smoke, `make test-persistence` и `make test-failover`: они
|
||||
загружают вложение;
|
||||
- тесты схемы `Message` в `make student_test` и `make failure_checks` —
|
||||
список уточнён в шапке `tasks.md` change `s3-storage`.
|
||||
|
||||
Причина одна: контракт лабы 4 требует `attachment_ids`, а код лабы 3 его
|
||||
не отдаёт. Чинит change `attachments` (ответ пользователя 2026-10-20,
|
||||
`PLAN.md`). Строки про вложения появятся с ним.
|
||||
Строки S3 (change `s3-storage`) сняты на стенде `make run_docker`:
|
||||
`evidence/s3/restart.md`, `client.md`, `privacy.md` — с commit
|
||||
`f57635a` (правки по verify), остальные файлы `evidence/s3/` — с
|
||||
`2e997f9`. Строки вложений (change `attachments`) и прогоны наборов
|
||||
`evidence/tests/` — с `d56b892` (код приложения `116d8d3`) на стенде
|
||||
`mtusi-lab4`: HAProxy и две реплики, PostgreSQL, Valkey, S3.
|
||||
|
||||
| Критерий README | Команда/сценарий | Наблюдаемый результат | Файл доказательства |
|
||||
| --- | --- | --- | --- |
|
||||
| Публичный smoke | make test | TODO после `attachments`; сейчас `FAIL: response: missing required fields` (нет `attachment_ids`) | `evidence/tests/test-isolated-s3-cleanup.md` |
|
||||
| Публичный smoke | `make test` (через LB и в `make test_isolated`) | `PASS: lab4 public API scenario` — загрузка fixture, сообщение с `attachment_ids`, SHA-256 скачанного оригинала, 404 постороннему | `evidence/attachments/smoke.md`, `evidence/tests/test-isolated-s3-cleanup.md` (раздел 0) |
|
||||
| «3»: метаданные в PostgreSQL, байты в приватном bucket, не в БД и не на диске API | `driver.py --port 8080 attachment-storage` | из `bytea` только `sha256` (32 байта), `object_key` = `messenger/attachments/<id>/original` без `filename`, объект в bucket 111 817 байт, анонимный GET ключа — 403, `docker diff` реплик — только `config.toml`; открыли bucket анониму — код 1 «anonymous GET … got 200», bootstrap закрыл — код 0 | `evidence/attachments/storage.md` |
|
||||
| «3»: POST вложения → 201 `ready` после сохранения; оригинал побайтно; `attachment_ids` читают участники; посторонний — 404 | `make test`; `driver.py upload` Алисой, `download` Бобом и Евой | 201 `status=ready`, `variants=[]`, `sha256` 99ea7533…; Боб — 200 `image/png`, 111 817 байт, тот же SHA-256 (отдала другая реплика); Ева — 404 на метаданные и байты | `evidence/attachments/smoke.md` |
|
||||
| «3»: реальные PNG/JPEG ≤10 MiB и ≤40 млн пикселей, MIME по содержимому, опасные `filename`, лимиты до чтения и распаковки | `make student_test` через LB (`test_attachments.py`, 32 теста) | 10 485 760 байт — 201, +1 — 413; 30 MiB с `Content-Length` и частями — 413; 8000 × 5000 — 201, 40 000 001+ — 413; «бомба» — 415; текст, GIF, пустое тело, подмена MIME — 415; `filename` с `/`, `\`, NUL, `\n`, DEL, U+0085, пустой, 256 — 400; U+202E, U+2066, U+2028 — 201 как есть | `evidence/tests/student-test-lb.md` |
|
||||
| «3»: остановка загрузившей реплики → download через другую | `driver.py --port 8080 attachment-failover --mode stop` и `--mode kill` | загрузку принял `5f5d4ac50d9a`, `stop-one.sh` его остановил (`stop` и `kill`), метаданные и байты — 200 от `847ad74efd7c`, SHA-256 совпал, посторонний — 404; после `make replicas` обе `UP` | `evidence/attachments/failover.md` |
|
||||
| «3»: restart с volumes сохраняет метаданные и оригинал | `make test-persistence ACTION_SCRIPT=scripts/restart.sh` (`LAB=4`) | PASS: прежний токен читает данные, оригинал вложения тот же; `StartedAt` новый у всех шести контейнеров (API, LB, PostgreSQL, Valkey, S3) | `evidence/replicas/test-persistence.md` |
|
||||
| «3»: failover с записью | `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` (каждая реплика, `stop` и `kill`, `STOP_ONE_LOG`) | 4 × PASS, `failed_attempts` 0, восстановление 0,76–1,2 с; `STOP_ONE_LOG` — оба контейнера | `evidence/failover/test-failover.md` |
|
||||
| «4»: тесты большого файла, повреждённого изображения, подмены MIME, чужого `attachment_id`, вложения другого чата; ошибка клиента не 500 | `make student_test` через LB (`test_rubric.py`: пять сценариев лабы 4), `make test_isolated` | 196 passed (через LB), 191 passed + 5 пропусков тестов реплик (один процесс); каждая ошибка — 4xx в формате контракта | `evidence/tests/student-test-lb.md`, `evidence/tests/test-isolated-s3-cleanup.md` |
|
||||
| «4»: отказы S3 на путях вложений, отклонённая загрузка ничего не оставляет | `make failure_checks` | 23 / 23, код 0: S3 остановлен — загрузка 503, `pending` +1, `ready` столько же, текст 201; скачивание 503, метаданные 200, `/health/ready` 200; S3 заморожен — 503 не позже 5,6 с; отклонённые загрузки — счётчики строк и объектов не изменились; рабочий стенд не тронут | `evidence/tests/failure-checks.md` |
|
||||
| «4»: длинная передача не держит соединение пула | разовый скрипт: 10 MiB частями ~12 с и скачивание с медленным чтением | 201 за 12,4 с и 200 за 13,1 с через LB, «Database is unavailable» — 0 | `evidence/attachments/slow-transfer.md` |
|
||||
| «4»: rolling update на схему вложений под нагрузкой | `make bench_rollout ROLLOUT_LABEL=attachments` (с образа `e1161f7`) | замена 8,26 с, ревизия `e656d6bbe1a4` → `ef25d50b01a8`, ошибок 0 / 0 / 0, 200 успешных в секунду всё время, сообщений = ключей 12 001 | `evidence/rollout/attachments/summary.md` |
|
||||
| «4»: исключения сверки покрытия сняты на стенде лабы 4 | `driver.py cache-check --require-test-keys`, `auth-storage`, повтор после `--force-recreate api` | кэш: одно чтение `chats` на 50; Argon2id и SHA-256 токена; повтор через новый процесс — тот же `Message` с `attachment_ids` | `evidence/cache/hits.md`, `evidence/auth/passwords-and-tokens.md`, `evidence/idempotency/replay-after-restart.md` |
|
||||
| «3»: S3-сервер в общем стенде, закреплённая версия | `make run_docker`, `docker compose ps` | `s3` на `chrislusf/seaweedfs:4.48@sha256:…` в состоянии `healthy`, `s3-bootstrap` — `Exited (0)`, реплики стартуют после него | `evidence/s3/privacy.md` (раздел 3), `evidence/s3/bootstrap.md` (раздел 1) |
|
||||
| «3»: persistent volume S3 | `scripts/restart.sh`; `docker kill -s KILL` сервера S3 | ID контейнеров прежние, `StartedAt` новый, hook — 24 с; объекты клиента читаются с тем же SHA-256: 3/3 после рестарта, 20/20 после SIGKILL; первая запись после `healthy` — 11–27 мс в 18 рестартах и `kill` (healthcheck ждёт регистрации тома) | `evidence/s3/restart.md` |
|
||||
| «3»: создание приватного bucket описано | `docker compose logs s3-bootstrap` на пустом volume | «bucket messenger created», «user api created», «checks passed» | `evidence/s3/bootstrap.md` (раздел 1), «Архитектура и решения» → «Bootstrap S3» |
|
||||
@@ -178,8 +309,7 @@ commit `f57635a` (правки по verify), остальные файлы `evid
|
||||
| «4»: S3 недоступен — ограниченная ошибка, текст работает | `make failure_checks` (`test_readiness_without_object_storage[stop\|pause]`); клиент при `pause` и `stop` S3, без bucket и без прав | без S3 (остановлен или заморожен) `/health/ready` и `/health/live` — 200 быстрее 1 с; вызов клиента — 503 за 1–8 мс (`stop`) или за срок 5 с (`pause`), после возврата проходит; удалённый bucket и отказ в доступе — 503 с записью `error`, а не «объекта нет» (кроме HEAD: 404 без тела) | `evidence/tests/failure-checks.md`, `evidence/s3/client.md` |
|
||||
| «4»: память S3 при одновременных загрузках | 20 одновременных PUT по 10 MiB ключом API | все подтверждены, анонимная память не больше 379 MiB из 512, OOM нет; подбор лимитов API — `upload-memory` | `evidence/s3/memory.md` |
|
||||
| Логи без секретов S3 | `grep -F` значений секретов по журналу клиента и `make run_dev` | 0 совпадений; отказ в учётных данных — одна запись `error` без секрета | `evidence/s3/client.md` |
|
||||
| Остальные пункты уровня 3 | TODO (`attachments`) | TODO | TODO |
|
||||
| Все пункты уровня 4, если заявлен | TODO (`attachments`, `upload-consistency`, `upload-memory`) | TODO | TODO |
|
||||
| Остальные пункты уровня 4 | TODO (`upload-consistency`: компенсация и уборщик; `upload-memory`: ≥10 одновременных загрузок и пик памяти) | TODO | TODO |
|
||||
| Выбранный трек 5, если заявлен | TODO | TODO | TODO |
|
||||
|
||||
TODO: добавить отдельную строку для каждого заявленного критерия. PASS без команды и наблюдения не является доказательством. Проверка инфраструктурного hook требует подтверждения, какие реальные процессы/хранилища перезапущены.
|
||||
@@ -198,6 +328,11 @@ TODO: гипотеза, hardware/лимиты, версии, объём данн
|
||||
| Bucket удалён мимо bootstrap | 503 и `error`, а не «объекта нет» | `put`, `open`, `delete` — 503, «bucket is missing» / «denied access»; `head` — `None` (404 без тела, ограничение принято) | — |
|
||||
| Неверный секрет API у приложения | 503 и запись `error` без секрета | `SignatureDoesNotMatch` → 503, одна запись `error` | — |
|
||||
| 20 одновременных PUT по 10 MiB | без OOM | анонимная память не больше 379 MiB; последний PUT подтверждён через 7,2 с (очередь лимита загрузок 64 МБ) | — |
|
||||
| Загрузка при остановленном S3 (`make failure_checks`) | 503 быстрее срока вызова, строка `pending` остаётся, текст работает | 503; `pending` +1, `ready` без изменений; текстовое сообщение 201 | после `start` та же загрузка — 201 |
|
||||
| S3 заморожен при загрузке и скачивании | 503 не позже 5 с + запас | 503 за ~5,0 с (срок вызова S3) | после `unpause` скачивание — те же байты |
|
||||
| Остановка (`stop` / `kill`) реплики, принявшей вложение | скачивание через другую реплику | 200 от другой реплики, тот же SHA-256 | потерь нет, реплика возвращается `make replicas` |
|
||||
| Поочерёдная замена на схему вложений под нагрузкой 100 записей + 100 чтений/с (`bench_rollout`, метка `attachments`; 2 реплики 1 CPU / 512 МБ, PostgreSQL 2 / 1 ГБ; хост на сети, давления памяти нет, `Swapouts` 0) | ошибок нет, миграция одна, старые реплики работают на новой схеме | замена 8,26 с, ошибок 0, p95 до 4 мс / макс. 43 мс в замене; ревизия `e656d6bbe1a4` → `ef25d50b01a8` (две таблицы и `NOT VALID`) | потерь нет: 12 001 сообщение = 12 001 ключ |
|
||||
| Медленный клиент 10 MiB (~12 с) | загрузка и скачивание проходят, сессия БД не держится | 201 за 12,4 с, 200 за 13,1 с; в реплике загрузка — 12 430 мс | — |
|
||||
|
||||
## Ограничения и следующие шаги
|
||||
|
||||
@@ -221,5 +356,32 @@ S3 (change `s3-storage`):
|
||||
клиента (5 с). Подбор лимитов и сроков — change `upload-memory`.
|
||||
- **Отказ S3 — 503 на путях вложений, а не отказ сервиса.** Отказ хоста,
|
||||
LB и PostgreSQL стенд не переживает (`.claude/rules/balancer.md`).
|
||||
- **Профиль `rollout` стенда `bench/` ссылается на commit лабы 3**
|
||||
(`KNOWN_ISSUES.md`), чинится в change `attachments`.
|
||||
|
||||
Вложения (change `attachments`):
|
||||
- **Проверка изображения структурная.** Обрезанный файл, битая CRC,
|
||||
сломанные маркеры, поток zlib не того объёма и «бомба» — 415. Но байт,
|
||||
изменённый внутри сжатых данных скана JPEG, не ловится: у формата нет
|
||||
контрольной суммы, Pillow его тоже не ловит. Полное декодирование —
|
||||
worker лабы 5.
|
||||
- **Строки `pending` пока никто не убирает.** Отказ S3 или БД между
|
||||
`pending` и `ready` оставляет строку (и, возможно, объект); API их не
|
||||
показывает. Уборщик с grace period — change `upload-consistency`.
|
||||
- **Память загрузки ~2 тела, без лимита одновременных загрузок.** Около
|
||||
30 одновременных загрузок по 10 MiB доведут реплику 512 МБ до OOM.
|
||||
Лимит, сроки чтения тела и замер — change `upload-memory`.
|
||||
- **Ограничение текста сообщения `NOT VALID`.** Ослабленное условие
|
||||
(`<= 4000`) добавлено без проверки прежних строк: они уже
|
||||
удовлетворяли более строгому, а `VALIDATE` держал бы `ACCESS
|
||||
EXCLUSIVE` на всё время просмотра `messages`. В каталоге ограничение
|
||||
помечено «не проверено»; новые строки оно проверяет как обычно.
|
||||
- **Ранний ответ на большое тело.** 404, 413 и 415 отдаются, не дочитав
|
||||
тело. Через LB ответ получает любой клиент (HAProxy дочитывает тело),
|
||||
httpx и `curl` — и напрямую; Python `urllib` напрямую к одной реплике
|
||||
ответ теряет (`BrokenPipe`).
|
||||
- **Смешанные версии во время `make rolling_update`.** Старая реплика не
|
||||
знает путей вложений (404) и `attachment_ids` (400) и отдаёт сообщения
|
||||
без `attachment_ids`. Окно — секунды замены. Откат на `e1161f7`
|
||||
(`MIGRATE=0`) прячет вложения и показывает пустые тексты сообщений с
|
||||
вложениями; схема при этом не откатывается.
|
||||
- **Обрыв S3 посреди скачивания** — статус уже ушёл: клиент получает
|
||||
тело короче `Content-Length`, причина — в журнале.
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# Вложение после остановки принявшей реплики
|
||||
|
||||
- Дата: 2026-10-20, 22:32:26–22:32:34 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`, две
|
||||
реплики API, PostgreSQL, Valkey, S3 (SeaweedFS).
|
||||
- Исключение сверки покрытия `attachments` «Принявшая реплика
|
||||
остановлена» (`student-tests/spec_coverage.py`); design change
|
||||
`attachments`, решение 13.
|
||||
|
||||
Команда `driver.py attachment-failover` создаёт синтетических Алису,
|
||||
Боба и Еву и чат Алисы и Боба, загружает `tests/fixtures/sample.png`
|
||||
через LB (реплика A — `X-Instance-Id` ответа), отправляет сообщение со
|
||||
ссылкой, вызывает `scripts/stop-one.sh` с `TARGET_INSTANCE_ID=A`, затем
|
||||
Боб скачивает оригинал через LB, пока не ответит другая реплика, и
|
||||
сверяет SHA-256; Ева получает 404. В `finally` — `make replicas` и
|
||||
ожидание, пока LB снова видит обе реплики `UP`. Токены не печатаются.
|
||||
|
||||
## `--mode stop` (SIGTERM, graceful)
|
||||
|
||||
```
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 attachment-failover --mode stop
|
||||
mode: stop; fixture sample.png: 111817 bytes
|
||||
docker compose ps (before):
|
||||
NAME STATUS
|
||||
mtusi-lab4-api-5 Up 18 seconds (healthy)
|
||||
mtusi-lab4-api-6 Up 18 seconds (healthy)
|
||||
mtusi-lab4-haproxy-1 Up 18 seconds (healthy)
|
||||
mtusi-lab4-postgres-1 Up 43 seconds (healthy)
|
||||
mtusi-lab4-s3-1 Up 28 seconds (healthy)
|
||||
mtusi-lab4-valkey-1 Up 43 seconds (healthy)
|
||||
sha256 uploaded: 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
sha256 downloaded: 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
docker compose ps (after):
|
||||
NAME STATUS
|
||||
mtusi-lab4-api-5 Up 1 second (healthy)
|
||||
mtusi-lab4-api-6 Up 22 seconds (healthy)
|
||||
mtusi-lab4-haproxy-1 Up 22 seconds (healthy)
|
||||
mtusi-lab4-postgres-1 Up 47 seconds (healthy)
|
||||
mtusi-lab4-s3-1 Up 32 seconds (healthy)
|
||||
mtusi-lab4-valkey-1 Up 47 seconds (healthy)
|
||||
step expected got x-instance-id
|
||||
alice uploads 201 201 5f5d4ac50d9a
|
||||
alice sends message 201 201 847ad74efd7c
|
||||
stop-one.sh stop 0 0 5f5d4ac50d9a
|
||||
bob downloads 200 200 847ad74efd7c
|
||||
bob reads metadata 200 200 847ad74efd7c
|
||||
eve reads metadata 404 404 847ad74efd7c
|
||||
eve downloads 404 404 847ad74efd7c
|
||||
make replicas 0 0
|
||||
LB slots UP 2 2
|
||||
PASS: accepted by 5f5d4ac50d9a, served by 847ad74efd7c after stop, sha256 matches, outsider gets 404
|
||||
exit=0
|
||||
```
|
||||
|
||||
## `--mode kill` (SIGKILL, отказ процесса)
|
||||
|
||||
```
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 attachment-failover --mode kill
|
||||
mode: kill; fixture sample.png: 111817 bytes
|
||||
docker compose ps (before):
|
||||
NAME STATUS
|
||||
mtusi-lab4-api-5 Up 2 seconds (healthy)
|
||||
mtusi-lab4-api-6 Up 22 seconds (healthy)
|
||||
mtusi-lab4-haproxy-1 Up 22 seconds (healthy)
|
||||
mtusi-lab4-postgres-1 Up 47 seconds (healthy)
|
||||
mtusi-lab4-s3-1 Up 32 seconds (healthy)
|
||||
mtusi-lab4-valkey-1 Up 47 seconds (healthy)
|
||||
sha256 uploaded: 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
sha256 downloaded: 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
docker compose ps (after):
|
||||
NAME STATUS
|
||||
mtusi-lab4-api-5 Up 1 second (healthy)
|
||||
mtusi-lab4-api-6 Up 25 seconds (healthy)
|
||||
mtusi-lab4-haproxy-1 Up 25 seconds (healthy)
|
||||
mtusi-lab4-postgres-1 Up 50 seconds (healthy)
|
||||
mtusi-lab4-s3-1 Up 35 seconds (healthy)
|
||||
mtusi-lab4-valkey-1 Up 50 seconds (healthy)
|
||||
step expected got x-instance-id
|
||||
alice uploads 201 201 5f5d4ac50d9a
|
||||
alice sends message 201 201 847ad74efd7c
|
||||
stop-one.sh kill 0 0 5f5d4ac50d9a
|
||||
bob downloads 200 200 847ad74efd7c
|
||||
bob reads metadata 200 200 847ad74efd7c
|
||||
eve reads metadata 404 404 847ad74efd7c
|
||||
eve downloads 404 404 847ad74efd7c
|
||||
make replicas 0 0
|
||||
LB slots UP 2 2
|
||||
PASS: accepted by 5f5d4ac50d9a, served by 847ad74efd7c after kill, sha256 matches, outsider gets 404
|
||||
exit=0
|
||||
```
|
||||
|
||||
- В обоих режимах вложение принял `5f5d4ac50d9a` (`mtusi-lab4-api-5`),
|
||||
его остановил `stop-one.sh`, а метаданные и оригинал отдал
|
||||
`847ad74efd7c` (`mtusi-lab4-api-6`). SHA-256 скачанного совпал с
|
||||
загруженным и с публичным fixture (`99ea7533…`).
|
||||
- Посторонний получает 404 на метаданные и байты.
|
||||
- После `make replicas` у `mtusi-lab4-api-5` новое время старта («Up 1
|
||||
second»): остановленная реплика вернулась тем же контейнером, LB видит
|
||||
обе `UP`.
|
||||
@@ -0,0 +1,45 @@
|
||||
# Медленные 10 MiB через LB: сессия БД не держится всю передачу
|
||||
|
||||
- Дата: 2026-10-20, 22:34:18–22:34:44 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`
|
||||
(`timeout client 30s`, `timeout server 10s`, `timeout http-request
|
||||
10s` — `infra/haproxy/haproxy.cfg`), две реплики API, PostgreSQL,
|
||||
Valkey, S3.
|
||||
- Design change `attachments`, решение 5: срок запроса к БД
|
||||
(`[database] request_timeout_seconds`, 6 с) раньше охватывал весь
|
||||
запрос (зависимость с `yield` закрывается после отправки ответа), и
|
||||
загрузка или отдача 10 MiB медленному клиенту оборвалась бы на 6-й
|
||||
секунде с «Database is unavailable». Теперь вход — короткая сессия
|
||||
(`CurrentUserBrief`), сервис открывает сессию на каждый шаг с БД.
|
||||
|
||||
Разовый скрипт `slow_transfer.py` (scratchpad, httpx): синтетический
|
||||
пользователь и чат; PNG ровно 10 485 760 байт (64 × 64 и чанк `tEXt`);
|
||||
загрузка 48 частями без `Content-Length` с паузами — ~12 с; затем
|
||||
скачивание с чтением по 64 KiB и паузами — ~12 с.
|
||||
|
||||
```
|
||||
$ uv run python -I slow_transfer.py http://127.0.0.1:8080 12
|
||||
PNG 10485760 bytes, sha256 8dedb8f04881e4aba6be3e87badcf4a1c2dc91efb827046146140fb0feb98ee5
|
||||
upload 10 MiB in 48 parts without Content-Length: 201 in 12.4 s, instance 847ad74efd7c
|
||||
download 10 MiB read slowly: 200 in 13.1 s, instance 5f5d4ac50d9a, 10485760 bytes, sha256 match True
|
||||
exit=0
|
||||
$ docker compose logs --no-log-prefix --since 2m api | grep -c "Database is unavailable"
|
||||
0
|
||||
```
|
||||
|
||||
Записи журнала реплик об этих запросах (поля выбраны, `filename` не
|
||||
пишется — только длина):
|
||||
|
||||
```
|
||||
{'event': 'Uploaded attachment', 'instance_id': '847ad74efd7c', 'size_bytes': 10485760, 'content_type': "<ImageType.PNG: 'image/png'>", 'width': 64, 'height': 64, 'duration_ms': 12430, 'filename_length': 8}
|
||||
{'event': 'HTTP request completed', 'instance_id': '5f5d4ac50d9a', 'method': 'GET', 'path': '/api/v1/attachments/{attachment_id}/content', 'status': 200, 'duration_ms': 4571.991}
|
||||
```
|
||||
|
||||
- Загрузка в реплике шла 12,4 с — вдвое дольше срока сессии БД (6 с), и
|
||||
закончилась 201: соединение пула занято только на короткие шаги
|
||||
(членство, строка `pending`, `ready`), а не на чтение тела.
|
||||
- Скачивание у клиента — 13,1 с, у реплики запрос закрыт через 4,6 с:
|
||||
остаток 10 MiB держат буферы сокетов и HAProxy. Отдача потоком идёт
|
||||
без сессии БД; SHA-256 полученного совпал.
|
||||
- Записи «Database is unavailable» нет.
|
||||
@@ -0,0 +1,64 @@
|
||||
# Вложения: публичный smoke и ручная проверка драйвером
|
||||
|
||||
- Дата: 2026-10-20, 22:33:47–22:33:50 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`, две
|
||||
реплики API, PostgreSQL, Valkey, S3.
|
||||
- Рубрика «3» лабы 4: загрузка PNG с `201 Attachment` (`ready`),
|
||||
побайтный download через API участнику, 404 постороннему.
|
||||
|
||||
## `make test`
|
||||
|
||||
Публичный сценарий лабы 4 (`tests/smoke.py`) загружает
|
||||
`tests/fixtures/sample.png`, отправляет сообщение с `attachment_ids`,
|
||||
скачивает оригинал и сверяет SHA-256, проверяет 404 постороннему.
|
||||
|
||||
```
|
||||
$ make test
|
||||
python3 tests/smoke.py --base-url "http://localhost:8080"
|
||||
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
|
||||
exit=0
|
||||
```
|
||||
|
||||
## `driver.py upload` / `download`
|
||||
|
||||
Синтетические Алиса и Боб — участники чата, Ева — посторонняя. Токены в
|
||||
выводе заменены на `<…-token>`, путь к fixture — относительный.
|
||||
|
||||
```
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 upload --token <alice-token> --chat 3bf818d6-6536-4c5a-9c9c-d74d1193e1c8 --file tests/fixtures/sample.png
|
||||
201 x-instance-id: 5f5d4ac50d9a
|
||||
{
|
||||
"id": "6757bc3a-66b5-4243-9ca1-ec942381ba7e",
|
||||
"chat_id": "3bf818d6-6536-4c5a-9c9c-d74d1193e1c8",
|
||||
"owner_id": "c0cf8ccb-ab83-4c51-90f8-74bcd4e3e0cf",
|
||||
"filename": "sample.png",
|
||||
"content_type": "image/png",
|
||||
"size_bytes": 111817,
|
||||
"sha256": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
|
||||
"status": "ready",
|
||||
"variants": [],
|
||||
"error_code": null,
|
||||
"created_at": "2026-10-20T22:33:48.827582Z",
|
||||
"updated_at": "2026-10-20T22:33:48.832614Z"
|
||||
}
|
||||
file: 111817 bytes, sha256 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
sha256 matches: True
|
||||
exit=0
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 download --token <bob-token> --attachment 6757bc3a-66b5-4243-9ca1-ec942381ba7e
|
||||
200 x-instance-id: 847ad74efd7c content-type: image/png content-length: 111817
|
||||
received: 111817 bytes, sha256 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
exit=0
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 download --token <eve-token> --attachment 6757bc3a-66b5-4243-9ca1-ec942381ba7e
|
||||
404 x-instance-id: 5f5d4ac50d9a content-type: application/json content-length: 100
|
||||
{"error":{"code":"not_found","message":"Not found","request_id":"0008c43dbac945a2b6bf659917a31580"}}
|
||||
exit=2
|
||||
GET /api/v1/attachments/6757bc3a-66b5-4243-9ca1-ec942381ba7e as eve -> 404 not_found
|
||||
```
|
||||
|
||||
- Загрузку принял `5f5d4ac50d9a`, оригинал Бобу отдал `847ad74efd7c`:
|
||||
байты из S3, а не с диска принявшей реплики. SHA-256 совпадает с
|
||||
файлом и с `tests/fixtures/README.md`.
|
||||
- `Content-Type` скачивания — `image/png` без параметров,
|
||||
`Content-Length` — размер оригинала.
|
||||
- Ева (не участник) получает одинаковый 404 на метаданные и байты.
|
||||
@@ -0,0 +1,105 @@
|
||||
# Где лежат вложения: метаданные в PostgreSQL, байты в приватном bucket
|
||||
|
||||
- Дата: 2026-10-20, 22:32:55–22:33:20 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`, две
|
||||
реплики API, PostgreSQL, Valkey, S3 (SeaweedFS, bucket `messenger`,
|
||||
API S3 на хосте только `127.0.0.1:S3_OUT_PORT`).
|
||||
- Исключение сверки покрытия `attachments` «Байты не в БД и не на диске
|
||||
API» (`student-tests/spec_coverage.py`); пункты 2 и 4 сценария защиты;
|
||||
design change `attachments`, решения 3, 4, 13.
|
||||
|
||||
Команда `driver.py attachment-storage` загружает `tests/fixtures/sample.png`
|
||||
дважды через LB (с именем `storage-check-<случайное>-<n>.png`) и
|
||||
проверяет, без печати токенов и секретов:
|
||||
- колонки таблиц вложений (psql): из `bytea` — только `sha256` ровно 32
|
||||
байта, колонки с байтами изображения нет;
|
||||
- `object_key` каждой загрузки — `<база>/attachments/<id>/original`, без
|
||||
`filename`;
|
||||
- объект с этим ключом есть в bucket, размер совпадает (`weed shell
|
||||
fs.ls -l` в контейнере `s3`);
|
||||
- анонимный GET ключа с хоста — 403;
|
||||
- участник скачивает тот же SHA-256;
|
||||
- `docker diff` каждой реплики — файлов оригиналов нет.
|
||||
|
||||
## Проверка
|
||||
|
||||
```
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 attachment-storage
|
||||
columns of attachment tables:
|
||||
attachments id uuid
|
||||
attachments chat_id uuid
|
||||
attachments owner_id uuid
|
||||
attachments filename text
|
||||
attachments content_type text
|
||||
attachments size_bytes integer
|
||||
attachments sha256 bytea
|
||||
attachments object_key text
|
||||
attachments status text
|
||||
attachments updated_at timestamp with time zone
|
||||
attachments created_at timestamp with time zone
|
||||
message_attachments message_id uuid
|
||||
message_attachments position smallint
|
||||
message_attachments attachment_id uuid
|
||||
message_attachments created_at timestamp with time zone
|
||||
upload via 5f5d4ac50d9a: object_key messenger/attachments/5e3a529e-abb2-49e1-a461-8334c7ba0eae/original
|
||||
sha256 uploaded 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75, downloaded by a member (200) 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
upload via 847ad74efd7c: object_key messenger/attachments/6afac582-4256-4ca0-8ae1-10aef5de8705/original
|
||||
sha256 uploaded 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75, downloaded by a member (200) 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
docker diff 5f5d4ac50d9a: 2 changes
|
||||
C /app
|
||||
A /app/config.toml
|
||||
docker diff 847ad74efd7c: 2 changes
|
||||
C /app
|
||||
A /app/config.toml
|
||||
bytea columns expected sha256 got sha256
|
||||
octet_length(sha256) expected 32 got 32
|
||||
object_key form expected messenger/attachments/5e3a529e-abb2-49e1-a461-8334c7ba0eae/original got messenger/attachments/5e3a529e-abb2-49e1-a461-8334c7ba0eae/original
|
||||
filename not in key expected no got no
|
||||
object in bucket, bytes expected 111817 got 111817
|
||||
anonymous GET of the key expected 403 got 403
|
||||
downloaded sha256 = uploaded expected yes got yes
|
||||
object_key form expected messenger/attachments/6afac582-4256-4ca0-8ae1-10aef5de8705/original got messenger/attachments/6afac582-4256-4ca0-8ae1-10aef5de8705/original
|
||||
filename not in key expected no got no
|
||||
object in bucket, bytes expected 111817 got 111817
|
||||
anonymous GET of the key expected 403 got 403
|
||||
downloaded sha256 = uploaded expected yes got yes
|
||||
original files in 5f5d4ac50d9a expected 0 got 0
|
||||
original files in 847ad74efd7c expected 0 got 0
|
||||
sha256 of the fixture: 99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75
|
||||
PASS: metadata in PostgreSQL, bytes only in the private bucket messenger, none in the API containers
|
||||
exit=0
|
||||
```
|
||||
|
||||
- Загрузки приняли обе реплики; у каждого вложения свой объект, ключ
|
||||
назначил сервер по имени базы и `id`.
|
||||
- `docker diff` реплик — только смонтированный `config.toml`: оригиналы
|
||||
на диск API не пишутся (тело живёт в памяти процесса до PUT).
|
||||
|
||||
## Отрицательная проверка: bucket открыт анониму
|
||||
|
||||
Анонимное чтение bucket открыто руками через `weed shell` (как ошибка
|
||||
оператора), затем повторный bootstrap его закрывает.
|
||||
|
||||
```
|
||||
$ printf 's3.anonymous.set -bucket messenger -access Read\n' | docker compose exec -T s3 sh -c "exec weed shell -master=127.0.0.1:9333 2>&1"
|
||||
Set anonymous access on bucket "messenger" to: Read
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 attachment-storage | grep -E "anonymous|FAIL|PASS"
|
||||
anonymous GET of the key expected 403 got 200 FAIL
|
||||
anonymous GET of the key expected 403 got 200 FAIL
|
||||
FAIL: 2 check(s) failed
|
||||
exit=1
|
||||
$ docker compose up -d s3-bootstrap
|
||||
$ docker compose logs --no-log-prefix s3-bootstrap | tail -3
|
||||
s3-bootstrap: anonymous access to messenger closed
|
||||
s3-bootstrap: user api exists
|
||||
s3-bootstrap: checks passed (api key put/get/delete; api key list and bucket create, anonymous get and list denied)
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 attachment-storage | grep -E "anonymous|FAIL|PASS"
|
||||
anonymous GET of the key expected 403 got 403
|
||||
anonymous GET of the key expected 403 got 403
|
||||
PASS: metadata in PostgreSQL, bytes only in the private bucket messenger, none in the API containers
|
||||
exit=0
|
||||
```
|
||||
|
||||
Команда замечает открытый bucket (код 1 и названная проверка), а
|
||||
штатный bootstrap возвращает приватность.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Пароли и токены в хранилище на стенде лабы 4
|
||||
|
||||
- Дата: 2026-10-20, 22:30:54 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`, две
|
||||
реплики API, PostgreSQL, Valkey, S3.
|
||||
- Исключения сверки покрытия: `sessions` «Токена нет в хранилище»,
|
||||
`users` «Хэш в БД» (`student-tests/spec_coverage.py`).
|
||||
|
||||
Пароль P и токен T синтетические и живут только в памяти драйвера;
|
||||
печатаются факты, длины и префикс строки Argon2id до соли. Дамп базы
|
||||
читается в память и никуда не пишется (9 блоков `COPY` — с таблицами
|
||||
вложений лабы 4).
|
||||
|
||||
```
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 auth-storage
|
||||
users auth_… and auth_…: same password P (36 characters with ':' and Cyrillic)
|
||||
password_hash $argon2id$v=19$m=65536,t=3,p=4$… (97 chars)
|
||||
password_hash $argon2id$v=19$m=65536,t=3,p=4$… (97 chars)
|
||||
hashes differ: t
|
||||
session of the first user: token T (43 characters); rows of sessions with token_hash = SHA-256(T): 1, octet_length 32
|
||||
pg_dump --data-only: 9 COPY blocks; T as text: no, T in hex: no, P as text: no
|
||||
PASS: only Argon2id strings and SHA-256 of the token are stored
|
||||
exit=0
|
||||
```
|
||||
|
||||
- «Хэш в БД»: в `users` — строки Argon2id (`m=65536,t=3,p=4`), у двух
|
||||
пользователей с одним паролем хэши разные (своя соль).
|
||||
- «Токена нет в хранилище»: в `sessions` — SHA-256 токена (32 байта),
|
||||
ни токена (текстом и в hex), ни пароля в `pg_dump` нет.
|
||||
Vendored
+32
@@ -0,0 +1,32 @@
|
||||
# Кэш карточки чата на стенде лабы 4
|
||||
|
||||
- Дата: 2026-10-20, 22:30:18 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`, две
|
||||
реплики API, PostgreSQL, Valkey, S3. Перед командой — `make
|
||||
test_isolated` (временная база `test_messenger`, 22:28:23–22:30:15
|
||||
UTC): её ключи кэша остаются в Valkey до срока.
|
||||
- Исключения сверки покрытия `chat-cache`: «Чтение из кэша», «Срок
|
||||
записи», «Временная база» (`student-tests/spec_coverage.py`).
|
||||
|
||||
```
|
||||
$ python3 .claude/skills/run-mtusi-messenger/driver.py --port 8080 cache-check --require-test-keys
|
||||
users alice_… (creator) and bob_… (reader), chat 848eb2e5-b9b9-463e-9695-11e13f6b9381
|
||||
key messenger:v1:messenger:chat:848eb2e5-b9b9-463e-9695-11e13f6b9381
|
||||
right after 201: EXISTS 1, TTL 3600 s (configured [cache] chat_ttl_seconds 3600 s in the replica); value keys ['created_at', 'id', 'member_ids', 'title'], member_ids in creation order: yes
|
||||
DEL key: the first read is a miss; waiting 11 s
|
||||
50 × GET /api/v1/chats/{id} by the reader through LB: 50 equal to the creation response; by replica 2bc86cb81f10 25, c0ae55959fb2 25
|
||||
after 11 s: reads of table chats (seq_scan + idx_scan) +1, keyspace_hits +49, keyspace_misses +1
|
||||
TTL 3589 s, then 10 more reads, TTL 3588 s: reads do not extend the key
|
||||
keys messenger:v1:messenger:chat:*: 3
|
||||
keys messenger:v1:test_messenger:chat:*: 138
|
||||
PASS: one read of chats, the rest from Valkey; TTL is bounded; temporary database: 138 keys under its own prefix
|
||||
exit=0
|
||||
```
|
||||
|
||||
- «Чтение из кэша»: 50 чтений через LB (по 25 на реплику) — одно чтение
|
||||
таблицы `chats` (промах после `DEL`), 49 попаданий в Valkey.
|
||||
- «Срок записи»: TTL не больше настроенного (3600 с) и чтениями не
|
||||
продлевается.
|
||||
- «Временная база»: ключи рабочей (`messenger`) и временной
|
||||
(`test_messenger`) базы — под разными префиксами.
|
||||
@@ -0,0 +1,104 @@
|
||||
# `make test-failover` на стенде лабы 4
|
||||
|
||||
- Дата: 2026-10-20, 22:35:09–22:36:40 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`, две
|
||||
реплики API (`mtusi-lab4-api-5` = `5f5d4ac50d9a`, `mtusi-lab4-api-6`
|
||||
= `847ad74efd7c`), PostgreSQL, Valkey, S3.
|
||||
- Профиль проверки — `LAB=4` (`lab.json`): `tests/resilience.py` ещё до
|
||||
отказа загружает вложение, 15 с пишет сообщения с ключом параллельно
|
||||
hook, повторяет неподтверждённые с тем же ключом, сверяет данные и
|
||||
после отказа снова скачивает оригинал вложения (SHA-256).
|
||||
- Hook — `scripts/stop-one.sh`: останавливает реплику, ответившую на
|
||||
первую отправку теста (`TARGET_INSTANCE_ID`), и оставляет её
|
||||
остановленной; `STOP_ONE_LOG` — какой контейнер остановлен. Между
|
||||
прогонами — `make replicas` и ожидание обеих реплик `UP` в LB.
|
||||
|
||||
## Прогон 1: `stop`, реплика `5f5d4ac50d9a`
|
||||
|
||||
```
|
||||
$ docker compose ps --format 'table {{.Name}}\t{{.ID}}\t{{.Status}}'
|
||||
NAME CONTAINER ID STATUS
|
||||
mtusi-lab4-api-5 5f5d4ac50d9a Up 2 minutes (healthy)
|
||||
mtusi-lab4-api-6 847ad74efd7c Up 3 minutes (healthy)
|
||||
mtusi-lab4-haproxy-1 1d3b522e85fa Up 3 minutes (healthy)
|
||||
mtusi-lab4-postgres-1 29c7b37863c1 Up 3 minutes (healthy)
|
||||
mtusi-lab4-s3-1 805bf7f382ac Up 3 minutes (healthy)
|
||||
mtusi-lab4-valkey-1 bd4329833fcc Up 3 minutes (healthy)
|
||||
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh STOP_ONE_LOG=stop-one.log
|
||||
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
|
||||
{"observed_instances_before": 2, "successful_writes": 117, "failed_attempts": 0, "first_recovery_seconds": 1.198}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
exit=0
|
||||
$ docker compose ps -a --format 'table {{.Name}}\t{{.ID}}\t{{.Status}}'
|
||||
NAME CONTAINER ID STATUS
|
||||
mtusi-lab4-api-5 5f5d4ac50d9a Exited (0) 14 seconds ago
|
||||
mtusi-lab4-api-6 847ad74efd7c Up 3 minutes (healthy)
|
||||
mtusi-lab4-haproxy-1 1d3b522e85fa Up 3 minutes (healthy)
|
||||
mtusi-lab4-migrate-1 90b7cf1186b7 Exited (0) 3 minutes ago
|
||||
mtusi-lab4-postgres-1 29c7b37863c1 Up 3 minutes (healthy)
|
||||
mtusi-lab4-s3-1 805bf7f382ac Up 3 minutes (healthy)
|
||||
mtusi-lab4-s3-bootstrap-1 7bfe3a481799 Exited (0) 2 minutes ago
|
||||
mtusi-lab4-valkey-1 bd4329833fcc Up 3 minutes (healthy)
|
||||
$ make replicas
|
||||
```
|
||||
|
||||
Прогон, запущенный сразу после `make replicas`, снова выбрал
|
||||
`5f5d4ac50d9a` (код 0, `first_recovery_seconds` 1.103): тест
|
||||
останавливает реплику, ответившую на его первую отправку, а порядок
|
||||
round-robin после возврата реплики тот же. Перед прогоном 2 — ожидание
|
||||
обеих реплик `UP` и один запрос `GET /health/live`, сдвигающий очередь
|
||||
LB (SKILL.md, стенд compose).
|
||||
|
||||
## Прогон 2: `stop`, реплика `847ad74efd7c`
|
||||
|
||||
```
|
||||
$ make lb_stats | grep '^api-'
|
||||
api-1 192.168.0.6:8080 UP L7OK 0 363 0 0 0 0
|
||||
api-2 192.168.0.5:8080 UP L7OK 0 110 0 0 10 0
|
||||
$ curl -s -o /dev/null http://127.0.0.1:8080/health/live
|
||||
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh STOP_ONE_LOG=stop-one.log
|
||||
{"observed_instances_before": 2, "successful_writes": 117, "failed_attempts": 0, "first_recovery_seconds": 1.115}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
exit=0
|
||||
$ docker compose ps -a --format 'table {{.Name}}\t{{.ID}}\t{{.Status}}' | grep api
|
||||
mtusi-lab4-api-5 5f5d4ac50d9a Up 26 seconds (healthy)
|
||||
mtusi-lab4-api-6 847ad74efd7c Exited (0) 14 seconds ago
|
||||
$ make replicas
|
||||
```
|
||||
|
||||
## Прогон 3: `STOP_MODE=kill`, реплика `847ad74efd7c`
|
||||
|
||||
```
|
||||
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh STOP_MODE=kill STOP_ONE_LOG=stop-one.log
|
||||
{"observed_instances_before": 2, "successful_writes": 116, "failed_attempts": 0, "first_recovery_seconds": 0.757}
|
||||
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
exit=0
|
||||
$ docker compose ps -a --format 'table {{.Name}}\t{{.ID}}\t{{.Status}}' | grep api
|
||||
mtusi-lab4-api-5 5f5d4ac50d9a Up 50 seconds (healthy)
|
||||
mtusi-lab4-api-6 847ad74efd7c Exited (137) 15 seconds ago
|
||||
$ make replicas
|
||||
$ docker compose ps --format 'table {{.Name}}\t{{.ID}}\t{{.Status}}'
|
||||
NAME CONTAINER ID STATUS
|
||||
mtusi-lab4-api-5 5f5d4ac50d9a Up 53 seconds (healthy)
|
||||
mtusi-lab4-api-6 847ad74efd7c Up 2 seconds (healthy)
|
||||
mtusi-lab4-haproxy-1 1d3b522e85fa Up 4 minutes (healthy)
|
||||
mtusi-lab4-postgres-1 29c7b37863c1 Up 4 minutes (healthy)
|
||||
mtusi-lab4-s3-1 805bf7f382ac Up 4 minutes (healthy)
|
||||
mtusi-lab4-valkey-1 bd4329833fcc Up 4 minutes (healthy)
|
||||
```
|
||||
|
||||
## `STOP_ONE_LOG`
|
||||
|
||||
```
|
||||
2026-10-20T22:35:10Z TARGET_INSTANCE_ID=5f5d4ac50d9a container=5f5d4ac50d9a5025dcd8834544bdd99e8d926be6007e89aec8d979caef6f80ee mode=stop
|
||||
2026-10-20T22:35:29Z TARGET_INSTANCE_ID=5f5d4ac50d9a container=5f5d4ac50d9a5025dcd8834544bdd99e8d926be6007e89aec8d979caef6f80ee mode=stop
|
||||
2026-10-20T22:35:55Z TARGET_INSTANCE_ID=847ad74efd7c container=847ad74efd7cc516f97edbe787c242cfd2b3d8a95bee0b6632e0a20565622db8 mode=stop
|
||||
2026-10-20T22:36:19Z TARGET_INSTANCE_ID=847ad74efd7c container=847ad74efd7cc516f97edbe787c242cfd2b3d8a95bee0b6632e0a20565622db8 mode=kill
|
||||
```
|
||||
|
||||
- Каждая реплика остановлена хотя бы раз (`stop`), вторая — ещё и
|
||||
`kill` (код выхода 137). Во всех прогонах `failed_attempts` 0, запись
|
||||
восстановилась за 0,76–1,2 с (порог теста — 10 с), дублей нет.
|
||||
- Остановленная реплика остаётся `Exited` до `make replicas`, затем
|
||||
возвращается тем же контейнером (тот же `CONTAINER ID`).
|
||||
@@ -0,0 +1,37 @@
|
||||
# Повтор с ключом после пересоздания реплик API
|
||||
|
||||
- Дата: 2026-10-20, 22:31:19 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`, две
|
||||
реплики API, PostgreSQL, Valkey, S3.
|
||||
- Исключение сверки покрытия `idempotency` «Повтор после рестарта API»
|
||||
(`student-tests/spec_coverage.py`): нужен новый процесс API, тесты
|
||||
сервер не трогают.
|
||||
|
||||
Разовый скрипт `replay_restart.py` (scratchpad, httpx): синтетический
|
||||
пользователь, чат, загрузка `tests/fixtures/sample.png`, отправка
|
||||
сообщения с вложением и ключом K; затем `docker compose up -d
|
||||
--force-recreate api` (обе реплики — новые контейнеры), ожидание
|
||||
`/health/ready` через LB и повтор того же тела с K. Токен и ключ не
|
||||
печатаются.
|
||||
|
||||
```
|
||||
$ uv run python -I replay_restart.py http://127.0.0.1:8080 .
|
||||
replicas before: ['2bc86cb81f10', 'c0ae55959fb2']
|
||||
send: 201 via 2bc86cb81f10, message 91d51305-f72d-49df-bf52-de818d790f5d, attachment_ids ['0d1482a5-e03e-4d8c-ad51-599bdc9cfa74']
|
||||
replicas after: ['5f5d4ac50d9a', '847ad74efd7c']
|
||||
replay 1: 201 via 5f5d4ac50d9a, same Message (JSON equal): True
|
||||
replay 2: 201 via 5f5d4ac50d9a, same Message (JSON equal): True
|
||||
history: 1 message(s) with this text: 1
|
||||
same key, other body: 409 conflict
|
||||
attachment after recreate: 200, sha256 matches: True
|
||||
exit=0
|
||||
```
|
||||
|
||||
- Повтор после пересоздания: другой `X-Instance-Id` (новый процесс), тот
|
||||
же 201 и тот же `Message` целиком — `id`, `created_at` до
|
||||
микросекунды, `attachment_ids`; в истории одно сообщение.
|
||||
- Ключ по-прежнему занят: другое тело — 409. Ключ и результат — в
|
||||
PostgreSQL, а не в памяти процесса.
|
||||
- Оригинал вложения после пересоздания реплик отдаётся с тем же
|
||||
SHA-256: байты в S3, а не на диске реплики.
|
||||
@@ -0,0 +1,71 @@
|
||||
# `make test-persistence` на стенде лабы 4
|
||||
|
||||
- Дата: 2026-10-20, 22:31:41–22:32:09 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — HAProxy на `127.0.0.1:8080`, две
|
||||
реплики API, PostgreSQL, Valkey, S3 (volumes `postgres_data`,
|
||||
`s3_data`).
|
||||
- Профиль проверки — `LAB=4` (`lab.json`): `tests/resilience.py`
|
||||
создаёт пользователей, чат, сообщения и **вложение** (загрузка
|
||||
`tests/fixtures/sample.png`), вызывает hook и прежним токеном читает
|
||||
те же данные, метаданные вложения и его байты (`attachment lost or
|
||||
changed across failure` при расхождении).
|
||||
- Исключения сверки покрытия: `data-storage` «Прежний пользователь после
|
||||
рестарта», `sessions` «Прежний токен после рестарта», `attachments`
|
||||
«Рестарт стенда» (`student-tests/spec_coverage.py`).
|
||||
|
||||
## Контейнеры до
|
||||
|
||||
```
|
||||
$ docker compose ps --format 'table {{.Name}}\t{{.Status}}'
|
||||
NAME STATUS
|
||||
mtusi-lab4-api-5 Up 18 seconds (healthy)
|
||||
mtusi-lab4-api-6 Up 18 seconds (healthy)
|
||||
mtusi-lab4-haproxy-1 Up 31 minutes (healthy)
|
||||
mtusi-lab4-postgres-1 Up 31 minutes (healthy)
|
||||
mtusi-lab4-s3-1 Up 31 minutes (healthy)
|
||||
mtusi-lab4-valkey-1 Up 31 minutes (healthy)
|
||||
$ docker inspect -f '{{.Name}} {{.State.StartedAt}}' $(docker compose ps -q)
|
||||
/mtusi-lab4-api-5 2026-10-20T22:31:23.061386013Z
|
||||
/mtusi-lab4-api-6 2026-10-20T22:31:23.117223013Z
|
||||
/mtusi-lab4-haproxy-1 2026-10-20T21:59:55.165833875Z
|
||||
/mtusi-lab4-postgres-1 2026-10-20T21:59:55.176395458Z
|
||||
/mtusi-lab4-s3-1 2026-10-20T21:59:55.173230792Z
|
||||
/mtusi-lab4-valkey-1 2026-10-20T21:59:55.154728458Z
|
||||
```
|
||||
|
||||
## Проверка
|
||||
|
||||
```
|
||||
$ make test-persistence ACTION_SCRIPT=scripts/restart.sh
|
||||
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://localhost:8080"
|
||||
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
|
||||
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
|
||||
exit=0
|
||||
```
|
||||
|
||||
## Контейнеры после
|
||||
|
||||
```
|
||||
$ docker inspect -f '{{.Name}} {{.State.StartedAt}}' $(docker compose ps -q)
|
||||
/mtusi-lab4-api-5 2026-10-20T22:32:07.759464589Z
|
||||
/mtusi-lab4-api-6 2026-10-20T22:32:07.813974797Z
|
||||
/mtusi-lab4-haproxy-1 2026-10-20T22:32:07.873866047Z
|
||||
/mtusi-lab4-postgres-1 2026-10-20T22:31:42.768951925Z
|
||||
/mtusi-lab4-s3-1 2026-10-20T22:31:57.790816751Z
|
||||
/mtusi-lab4-valkey-1 2026-10-20T22:31:42.807267216Z
|
||||
$ docker compose ps --format 'table {{.Name}}\t{{.Status}}'
|
||||
NAME STATUS
|
||||
mtusi-lab4-api-5 Up 2 seconds (healthy)
|
||||
mtusi-lab4-api-6 Up 1 second (healthy)
|
||||
mtusi-lab4-haproxy-1 Up 1 second (healthy)
|
||||
mtusi-lab4-postgres-1 Up 27 seconds (healthy)
|
||||
mtusi-lab4-s3-1 Up 11 seconds (healthy)
|
||||
mtusi-lab4-valkey-1 Up 26 seconds (healthy)
|
||||
```
|
||||
|
||||
- `StartedAt` у **всех** шести контейнеров новый: `scripts/restart.sh`
|
||||
перезапустил обе реплики, LB, PostgreSQL, Valkey и S3 (S3 — плавно,
|
||||
~15 с), те же контейнеры и volumes.
|
||||
- После этого прежний токен читает те же данные, а скачанный оригинал
|
||||
вложения совпадает с загруженным (проверка `resilience.py` с `LAB=4`).
|
||||
@@ -0,0 +1,373 @@
|
||||
{
|
||||
"label": "attachments",
|
||||
"status": "complete",
|
||||
"started_at": "2026-10-20T22:44:19+00:00",
|
||||
"command": "uv run python bench/run.py --profile rollout --label attachments",
|
||||
"git": {
|
||||
"commit": "d56b892",
|
||||
"dirty": true
|
||||
},
|
||||
"overrides": {},
|
||||
"host_state": {
|
||||
"start": {
|
||||
"swap_used_mib": 0.0,
|
||||
"swap_total_mib": 0.0,
|
||||
"swapouts": 0,
|
||||
"memory_free_percent": 66,
|
||||
"power": "AC Power",
|
||||
"battery_percent": 84
|
||||
},
|
||||
"end": {
|
||||
"swap_used_mib": 0.0,
|
||||
"swap_total_mib": 0.0,
|
||||
"swapouts": 0,
|
||||
"memory_free_percent": 67,
|
||||
"power": "AC Power",
|
||||
"battery_percent": 84
|
||||
},
|
||||
"swapouts_delta": 0,
|
||||
"slept_seconds": 0
|
||||
},
|
||||
"host_warnings": [],
|
||||
"other_containers": [],
|
||||
"resources": {
|
||||
"budget": {
|
||||
"cpus": 6.75,
|
||||
"memory_gib": 3.0,
|
||||
"peak_replicas": 3,
|
||||
"vm_cpus": 12,
|
||||
"vm_memory_gib": 7.75,
|
||||
"generator": "no limit"
|
||||
},
|
||||
"mode": "scan",
|
||||
"interval_s": 1.0,
|
||||
"containers": {
|
||||
"e1726165eec2": {
|
||||
"name": "mtusi-bench-haproxy",
|
||||
"role": "haproxy"
|
||||
},
|
||||
"9c7a7d981f5d": {
|
||||
"name": "mtusi-bench-api-1",
|
||||
"role": "api"
|
||||
},
|
||||
"836903d5ebd9": {
|
||||
"name": "mtusi-bench-api-2",
|
||||
"role": "api"
|
||||
},
|
||||
"2708a61ed609": {
|
||||
"name": "mtusi-bench-postgres",
|
||||
"role": "postgres"
|
||||
},
|
||||
"dbe0fec0557d": {
|
||||
"name": "mtusi-bench-valkey",
|
||||
"role": "valkey"
|
||||
},
|
||||
"d3da387bc8da": {
|
||||
"name": "mtusi-bench-k6",
|
||||
"role": "k6"
|
||||
},
|
||||
"565f94da69f4": {
|
||||
"name": "mtusi-bench-sampler",
|
||||
"role": "sampler"
|
||||
},
|
||||
"9128897910ef": {
|
||||
"name": "mtusi-bench-api-4",
|
||||
"role": "api"
|
||||
},
|
||||
"2ec1a8170637": {
|
||||
"name": "mtusi-bench-api-3",
|
||||
"role": "api"
|
||||
},
|
||||
"87f513915217": {
|
||||
"name": "mtusi-bench-migrate-1",
|
||||
"role": "migrate"
|
||||
}
|
||||
},
|
||||
"unmapped": 0
|
||||
},
|
||||
"app_image": {
|
||||
"tag": "mtusi-messenger:bench",
|
||||
"id": "sha256:3d1afe784eda8e58a37dd40606e4980d0819656e51bdd5cae597d049bbe33295",
|
||||
"versions": {
|
||||
"python": "3.12.14",
|
||||
"fastapi": "0.141.1",
|
||||
"starlette": "1.7.0",
|
||||
"pydantic": "2.13.5",
|
||||
"uvicorn": "0.54.0",
|
||||
"sqlalchemy": "2.1.1",
|
||||
"asyncpg": "0.31.0",
|
||||
"redis": "8.1.0"
|
||||
}
|
||||
},
|
||||
"generator": {
|
||||
"image": "grafana/k6:2.3.0@sha256:9c2dee7f8ed74d317e4027c06a10f169b625638189de8d4555d0b3486a5aeb34",
|
||||
"version": "k6 v2.3.0 (commit/e088784614, go1.27.1, linux/arm64)"
|
||||
},
|
||||
"docker": {
|
||||
"server": "29.9.1",
|
||||
"vm_os": "Docker Desktop",
|
||||
"vm_cpus": 12,
|
||||
"vm_memory_gib": 7.7,
|
||||
"cgroup": "2",
|
||||
"logging_driver": "local"
|
||||
},
|
||||
"host": {
|
||||
"os": "macOS 26.6.2",
|
||||
"machine": "Apple M3 Pro",
|
||||
"memory_gib": 18.0,
|
||||
"performance_cores": 6,
|
||||
"efficiency_cores": 6
|
||||
},
|
||||
"profile": {
|
||||
"name": "rollout",
|
||||
"sizes": [
|
||||
1000
|
||||
],
|
||||
"concurrency": [],
|
||||
"repeats": 0,
|
||||
"read_warmup_s": 0,
|
||||
"read_measure_s": 0,
|
||||
"write_warmup": 0,
|
||||
"write_measure": 0,
|
||||
"prefill_chunk": 25000,
|
||||
"prefill_vus": 8,
|
||||
"text_length": 64,
|
||||
"page_limit": 50,
|
||||
"walk_limit": 100,
|
||||
"cursor_pool": 1000,
|
||||
"members": 10,
|
||||
"seed": 20260930,
|
||||
"warmup_seed_shift": 1000003,
|
||||
"generator_cpu_limit": 4.0,
|
||||
"cpus": "1",
|
||||
"memory": "512m",
|
||||
"db_cpus": "2",
|
||||
"db_memory": "1g",
|
||||
"cache_cpus": "0.5",
|
||||
"cache_memory": "128m",
|
||||
"session_ttl_seconds": 21600,
|
||||
"background": [],
|
||||
"index_variants": [
|
||||
[
|
||||
"chat_only",
|
||||
"(chat_id)"
|
||||
],
|
||||
[
|
||||
"covering",
|
||||
"(chat_id, created_at, id) INCLUDE (sender_id, text)"
|
||||
]
|
||||
],
|
||||
"write_probe_rows": 20000,
|
||||
"write_probe_repeats": 3,
|
||||
"restore": false,
|
||||
"restore_jobs": 4,
|
||||
"restore_write_load": 5000,
|
||||
"restore_write_vus": 10,
|
||||
"replicas": 2,
|
||||
"lb_cpus": "1",
|
||||
"lb_memory": "256m",
|
||||
"failover_modes": [],
|
||||
"failover_write_rate": 100,
|
||||
"failover_read_rate": 100,
|
||||
"failover_before_s": 0,
|
||||
"failover_down_s": 0,
|
||||
"failover_after_s": 0,
|
||||
"failover_success": 0.99,
|
||||
"failover_pre_vus": 600,
|
||||
"failover_max_vus": 1500,
|
||||
"rollout_from": "e1161f7",
|
||||
"rollout_before_s": 15,
|
||||
"rollout_budget_s": 90,
|
||||
"rollout_after_s": 15,
|
||||
"sample_resources": true,
|
||||
"db_max_connections": 100,
|
||||
"series": "",
|
||||
"configs": [],
|
||||
"write_chats": 1,
|
||||
"mix": [],
|
||||
"fixed_rps": [],
|
||||
"ladder_base": 100,
|
||||
"step_factor": 1.25,
|
||||
"start_rps_per_replica": 300,
|
||||
"bisect_steps": 2,
|
||||
"max_steps": 12,
|
||||
"first_warmup_s": 15,
|
||||
"step_warmup_s": 5,
|
||||
"step_measure_s": 20,
|
||||
"cooldown_s": 10,
|
||||
"slo_p95_ms": 200,
|
||||
"slo_error_ratio": 0.01,
|
||||
"abort_error_ratio": 0.5,
|
||||
"abort_p95_ms": 2000,
|
||||
"request_timeout_s": 15,
|
||||
"prevus_per_rps": 0.05,
|
||||
"vus_per_rps": 0.2,
|
||||
"generator_cpus": "",
|
||||
"generator_memory": "",
|
||||
"sampler_cpus": "0.25",
|
||||
"sampler_memory": "128m",
|
||||
"sample_interval_s": 1.0,
|
||||
"order_seed": 20261011
|
||||
},
|
||||
"sizes": [],
|
||||
"rollout": {
|
||||
"from_commit": "e1161f7",
|
||||
"new_image": "sha256:3d1afe784eda8e58a37dd40606e4980d0819656e51bdd5cae597d049bbe33295",
|
||||
"old_image": "sha256:ba551a152f2cc2c3091abe2bf02d5bced68b8c986ed202a5bef4204ebc317755",
|
||||
"containers_before": [
|
||||
{
|
||||
"name": "mtusi-bench-api-1",
|
||||
"instance": "9c7a7d981f5d",
|
||||
"tag": "mtusi-messenger:bench",
|
||||
"image": "sha256:ba551a152f2cc2c3091abe2bf02d5bced68b8c986ed202a5bef4204ebc317755",
|
||||
"running": true
|
||||
},
|
||||
{
|
||||
"name": "mtusi-bench-api-2",
|
||||
"instance": "836903d5ebd9",
|
||||
"tag": "mtusi-messenger:bench",
|
||||
"image": "sha256:ba551a152f2cc2c3091abe2bf02d5bced68b8c986ed202a5bef4204ebc317755",
|
||||
"running": true
|
||||
}
|
||||
],
|
||||
"revision_before": "e656d6bbe1a4",
|
||||
"containers_after": [
|
||||
{
|
||||
"name": "mtusi-bench-api-3",
|
||||
"instance": "2ec1a8170637",
|
||||
"tag": "mtusi-messenger:bench",
|
||||
"image": "sha256:ce2cce1235d3f02ca52fdaefa4221b206785834b65e7a9af793a8b21b4ec281b",
|
||||
"running": true
|
||||
},
|
||||
{
|
||||
"name": "mtusi-bench-api-4",
|
||||
"instance": "9128897910ef",
|
||||
"tag": "mtusi-messenger:bench",
|
||||
"image": "sha256:ce2cce1235d3f02ca52fdaefa4221b206785834b65e7a9af793a8b21b4ec281b",
|
||||
"running": true
|
||||
}
|
||||
],
|
||||
"revision_after": "ef25d50b01a8",
|
||||
"tag_after": "sha256:ce2cce1235d3f02ca52fdaefa4221b206785834b65e7a9af793a8b21b4ec281b",
|
||||
"script_start_at": "2026-10-20T22:44:49.498+00:00"
|
||||
},
|
||||
"limits": {
|
||||
"api": {
|
||||
"nano_cpus": 1000000000,
|
||||
"memory": 536870912,
|
||||
"memory_swap": 536870912,
|
||||
"published_ports": {},
|
||||
"healthcheck": [
|
||||
"NONE"
|
||||
],
|
||||
"log_driver": "local"
|
||||
},
|
||||
"postgres": {
|
||||
"nano_cpus": 2000000000,
|
||||
"memory": 1073741824,
|
||||
"memory_swap": 1073741824,
|
||||
"published_ports": {},
|
||||
"healthcheck": [
|
||||
"CMD-SHELL",
|
||||
"pg_isready -h 127.0.0.1 -U \"$POSTGRES_USER\" -d \"$POSTGRES_DB\""
|
||||
],
|
||||
"log_driver": "local"
|
||||
},
|
||||
"valkey": {
|
||||
"nano_cpus": 500000000,
|
||||
"memory": 134217728,
|
||||
"memory_swap": 134217728,
|
||||
"published_ports": {},
|
||||
"healthcheck": [
|
||||
"CMD-SHELL",
|
||||
"test \"$(valkey-cli -p \"$VALKEY_PORT\" ping)\" = PONG"
|
||||
],
|
||||
"log_driver": "local"
|
||||
},
|
||||
"haproxy": {
|
||||
"nano_cpus": 1000000000,
|
||||
"memory": 268435456,
|
||||
"memory_swap": 268435456,
|
||||
"published_ports": {},
|
||||
"healthcheck": [
|
||||
"CMD",
|
||||
"bash",
|
||||
"-c",
|
||||
"exec 3<>/dev/tcp/127.0.0.1/8404 && printf 'GET /lb/ready HTTP/1.0\\r\\n\\r\\n' >&3 && head -n 1 <&3 | grep -q ' 200 '"
|
||||
],
|
||||
"log_driver": "local"
|
||||
},
|
||||
"k6": {
|
||||
"nano_cpus": 0,
|
||||
"memory": 0,
|
||||
"memory_swap": 0,
|
||||
"published_ports": {},
|
||||
"healthcheck": null,
|
||||
"log_driver": "local"
|
||||
},
|
||||
"api_replicas": 2,
|
||||
"max_connections": 100,
|
||||
"sampler": {
|
||||
"nano_cpus": 250000000,
|
||||
"memory": 134217728,
|
||||
"memory_swap": 134217728,
|
||||
"published_ports": {},
|
||||
"healthcheck": null,
|
||||
"log_driver": "local"
|
||||
}
|
||||
},
|
||||
"stand": {
|
||||
"postgres": {
|
||||
"image": "postgres:18.6-trixie@sha256:5a5a84b19854a9ffaa54082c166ff4ec27473a361e496e5ea167f298f2da9722",
|
||||
"id": "sha256:5a5a84b19854a9ffaa54082c166ff4ec27473a361e496e5ea167f298f2da9722",
|
||||
"version": "18.6 (Debian 18.6-1.pgdg13+2)",
|
||||
"settings": {
|
||||
"autovacuum": "on",
|
||||
"checkpoint_timeout": "5min",
|
||||
"effective_cache_size": "4GB",
|
||||
"fsync": "on",
|
||||
"jit": "on",
|
||||
"maintenance_work_mem": "64MB",
|
||||
"max_connections": "100",
|
||||
"max_wal_size": "1GB",
|
||||
"plan_cache_mode": "auto",
|
||||
"random_page_cost": "4",
|
||||
"shared_buffers": "128MB",
|
||||
"shared_preload_libraries": "auto_explain",
|
||||
"synchronous_commit": "on",
|
||||
"wal_level": "replica",
|
||||
"work_mem": "4MB"
|
||||
}
|
||||
},
|
||||
"valkey": {
|
||||
"image": "valkey/valkey:9.1.2-trixie@sha256:418652cfb58ef879d4978c33553735d7147016032d5aefaa14c828e611eb9dfd",
|
||||
"id": "sha256:418652cfb58ef879d4978c33553735d7147016032d5aefaa14c828e611eb9dfd",
|
||||
"version": "9.1.2",
|
||||
"maxmemory": "67108864",
|
||||
"maxmemory-clients": "0",
|
||||
"maxmemory-eviction-tenacity": "10",
|
||||
"maxmemory-policy": "allkeys-lru",
|
||||
"maxmemory-samples": "5"
|
||||
},
|
||||
"haproxy": {
|
||||
"image": "haproxy:3.4.6-trixie@sha256:e273915691a8eab3f105bf3e39fd4c29a4ecf334e33421903b5589d25e14db26",
|
||||
"id": "sha256:e273915691a8eab3f105bf3e39fd4c29a4ecf334e33421903b5589d25e14db26",
|
||||
"version": "HAProxy version 3.4.6-56332c5 2026/09/28 - https://haproxy.org/"
|
||||
},
|
||||
"app_env": {
|
||||
"APP__PORT": "8080",
|
||||
"APP__RUN_MODE": "docker",
|
||||
"AUTH__SESSION_TTL_SECONDS": "21600",
|
||||
"CACHE__HOST": "valkey",
|
||||
"CACHE__PORT": "6379",
|
||||
"DATABASE__HOST": "postgres",
|
||||
"DATABASE__NAME": "bench",
|
||||
"DATABASE__PORT": "5432",
|
||||
"DATABASE__USER": "bench",
|
||||
"S3__BUCKET": "bench",
|
||||
"S3__HOST": "s3",
|
||||
"S3__PORT": "8333"
|
||||
}
|
||||
},
|
||||
"finished_at": "2026-10-20T22:46:37+00:00"
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
size,chunk,messages_before,messages_after,duration_s,requests_ok,requests_failed,rps,error_ratio,p50_ms,p95_ms,p99_ms,max_ms,app_cpu_cores,rss_end_mib,mem_cgroup_end_mib,generator_cpu_cores,flags,db_cpu_cores,cell,attempt
|
||||
1000,1,0,1000,1.037,1000,0,964.1,0.0,6.633,10.11,50.291,57.769,1.905,242.3,222.4,0.225,,0.321,,
|
||||
|
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"label": "attachments",
|
||||
"profile": "rollout",
|
||||
"stage": "done",
|
||||
"size": 1000,
|
||||
"runs_done": 1,
|
||||
"runs_total": 1,
|
||||
"elapsed_s": 138,
|
||||
"eta_s": 0,
|
||||
"warnings": [],
|
||||
"updated_at": "2026-10-20T22:46:37+00:00"
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"root_group":{"path":"","id":"d41d8cd98f00b204e9800998ecf8427e","groups":[],"checks":[{"id":"6445aff337e1ade0bc10402fbac79882","passes":1000,"fails":0,"name":"message 201","path":"::message 201"}],"name":""},"options":{"summaryTrendStats":["avg","min","med","p(50)","p(95)","p(99)","max"],"summaryTimeUnit":"","noColor":true},"state":{"isStdOutTTY":false,"isStdErrTTY":false,"testRunDurationMs":1037.217251},"metrics":{"http_req_duration":{"type":"trend","contains":"time","values":{"max":57.768667,"avg":8.210651240000006,"min":1.748667,"med":6.633458,"p(50)":6.633458,"p(95)":10.1098577,"p(99)":50.29116207999999}},"http_req_waiting":{"type":"trend","contains":"time","values":{"p(95)":10.09407465,"p(99)":50.2625755,"max":57.730917,"avg":8.189873326999999,"min":1.735833,"med":6.6125004999999994,"p(50)":6.6125004999999994}},"http_req_receiving":{"values":{"p(50)":0.0137295,"p(95)":0.02754615,"p(99)":0.04563,"max":0.114875,"avg":0.015436868999999982,"min":0.0045,"med":0.0137295},"type":"trend","contains":"time"},"vus":{"contains":"default","values":{"value":8,"min":8,"max":8},"type":"gauge"},"iteration_duration":{"type":"trend","contains":"time","values":{"p(99)":50.36639832999999,"max":57.827041,"avg":8.276699092000012,"min":1.787375,"med":6.694062499999999,"p(50)":6.694062499999999,"p(95)":10.205659799999994}},"vus_max":{"type":"gauge","contains":"default","values":{"value":8,"min":8,"max":8}},"http_req_failed":{"values":{"rate":0,"passes":0,"fails":1000},"type":"rate","contains":"default"},"iterations":{"contains":"default","values":{"count":1000,"rate":964.1181719990502},"type":"counter"},"http_req_connecting":{"type":"trend","contains":"time","values":{"p(95)":0,"p(99)":0,"max":0.116875,"avg":0.0007123349999999999,"min":0,"med":0,"p(50)":0}},"checks":{"type":"rate","contains":"default","values":{"rate":1,"passes":1000,"fails":0}},"http_reqs":{"type":"counter","contains":"default","values":{"count":1000,"rate":964.1181719990502}},"data_sent":{"type":"counter","contains":"data","values":{"count":376000,"rate":362508.43267164286}},"http_req_duration{expected_response:true}":{"type":"trend","contains":"time","values":{"p(95)":10.1098577,"p(99)":50.29116207999999,"max":57.768667,"avg":8.210651240000006,"min":1.748667,"med":6.633458,"p(50)":6.633458},"thresholds":{"max>=0":{"ok":true}}},"http_req_sending":{"type":"trend","contains":"time","values":{"avg":0.005341044000000003,"min":0.002,"med":0.0037705,"p(50)":0.0037705,"p(95)":0.009544099999999998,"p(99)":0.03509766999999984,"max":0.170708}},"http_req_tls_handshaking":{"contains":"time","values":{"avg":0,"min":0,"med":0,"p(50)":0,"p(95)":0,"p(99)":0,"max":0},"type":"trend"},"data_received":{"type":"counter","contains":"data","values":{"count":470000,"rate":453135.5408395536}},"http_req_blocked":{"type":"trend","contains":"time","values":{"p(50)":0.000958,"p(95)":0.001917,"p(99)":0.01567999999999995,"max":0.605,"avg":0.005147167000000029,"min":0.0005,"med":0.000958}}}}
|
||||
@@ -0,0 +1 @@
|
||||
{"root_group":{"name":"","path":"","id":"d41d8cd98f00b204e9800998ecf8427e","groups":[],"checks":[]},"options":{"summaryTrendStats":["avg","min","med","p(50)","p(95)","p(99)","max"],"summaryTimeUnit":"","noColor":true},"state":{"isStdOutTTY":false,"isStdErrTTY":false,"testRunDurationMs":120005.538458},"metrics":{"http_reqs":{"values":{"rate":200.00743556015385,"count":24002},"type":"counter","contains":"default"},"http_req_duration{expected_response:true}":{"thresholds":{"max>=0":{"ok":true}},"type":"trend","contains":"time","values":{"min":1.730834,"med":3.3515,"p(50)":3.3515,"p(95)":4.4306976,"p(99)":8.629847589999988,"max":116.259708,"avg":3.7897147009832346}},"iterations{scenario:write}":{"type":"counter","contains":"default","values":{"rate":100.00371778007693,"count":12001},"thresholds":{"count>=0":{"ok":true}}},"http_req_blocked":{"values":{"p(99)":0.15879074999999995,"max":8.844834,"avg":0.011468904007998942,"min":0.001,"med":0.003,"p(50)":0.003,"p(95)":0.05991219999999902},"type":"trend","contains":"time"},"http_req_failed":{"type":"rate","contains":"default","values":{"rate":0,"passes":0,"fails":24002}},"vus_max":{"type":"gauge","contains":"default","values":{"value":1200,"min":1200,"max":1200}},"http_req_receiving":{"contains":"time","values":{"med":0.028208,"p(50)":0.028208,"p(95)":0.070583,"p(99)":0.10120899,"max":1.888458,"avg":0.034012066702774645,"min":0.008042},"type":"trend"},"http_req_connecting":{"type":"trend","contains":"time","values":{"p(95)":0,"p(99)":0.11891392999999995,"max":8.794958,"avg":0.0066497711024081285,"min":0,"med":0,"p(50)":0}},"iteration_duration":{"type":"trend","contains":"time","values":{"p(50)":3.5813125,"p(95)":4.721968099999998,"p(99)":9.51691499999995,"max":116.542042,"avg":4.026741841304922,"min":1.863083,"med":3.5813125}},"iterations{scenario:read}":{"type":"counter","contains":"default","values":{"rate":100.00371778007693,"count":12001},"thresholds":{"count>=0":{"ok":true}}},"iterations":{"type":"counter","contains":"default","values":{"count":24002,"rate":200.00743556015385}},"http_req_tls_handshaking":{"type":"trend","contains":"time","values":{"med":0,"p(50)":0,"p(95)":0,"p(99)":0,"max":0,"avg":0,"min":0}},"http_req_duration":{"values":{"avg":3.7897147009832346,"min":1.730834,"med":3.3515,"p(50)":3.3515,"p(95)":4.4306976,"p(99)":8.629847589999988,"max":116.259708},"type":"trend","contains":"time"},"http_req_sending":{"contains":"time","values":{"avg":0.013190699691692502,"min":0.003375,"med":0.012708,"p(50)":0.012708,"p(95)":0.021834,"p(99)":0.03099958999999999,"max":0.234667},"type":"trend"},"data_received":{"values":{"count":178365269,"rate":1486308.6428500544},"type":"counter","contains":"data"},"http_req_waiting":{"type":"trend","contains":"time","values":{"p(99)":8.503545419999993,"max":116.2065,"avg":3.7425119345887623,"min":1.705167,"med":3.3048545000000003,"p(50)":3.3048545000000003,"p(95)":4.3819854}},"vus":{"type":"gauge","contains":"default","values":{"value":2,"min":0,"max":11}},"data_sent":{"type":"counter","contains":"data","values":{"count":9277664,"rate":77310.2985013232}}}}
|
||||
@@ -0,0 +1,12 @@
|
||||
2026-10-20T22:44:52.949642763Z Server api/api-3 ('api') is DOWN/READY (resolves again).
|
||||
2026-10-20T22:44:52.949643596Z Server api/api-3 administratively READY thanks to valid DNS answer.
|
||||
2026-10-20T22:44:54.691038097Z Server api/api-3 is UP, reason: Layer7 check passed, code: 200, check duration: 1ms. 3 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
|
||||
2026-10-20T22:44:55.196048750Z Server api/api-2 enters drain state. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
|
||||
2026-10-20T22:44:55.338123333Z Server api/api-2 is going DOWN for maintenance. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
|
||||
2026-10-20T22:44:56.310988792Z Server api/api-2 is leaving forced maintenance but remains in maintenance.
|
||||
2026-10-20T22:44:56.954118875Z Server api/api-2 ('api') is DOWN/READY (resolves again).
|
||||
2026-10-20T22:44:56.954119750Z Server api/api-2 administratively READY thanks to valid DNS answer.
|
||||
2026-10-20T22:44:57.667797167Z Server api/api-2 is UP, reason: Layer7 check passed, code: 200, check duration: 16ms. 3 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
|
||||
2026-10-20T22:44:58.137383918Z Server api/api-1 enters drain state. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
|
||||
2026-10-20T22:44:58.280975626Z Server api/api-1 is going DOWN for maintenance. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
|
||||
2026-10-20T22:44:59.302039460Z Server api/api-1 is leaving forced maintenance but remains in maintenance.
|
||||
@@ -0,0 +1,18 @@
|
||||
22:44:50 rolling-update: start: 2 replicas 9c7a7d981f5d 836903d5ebd9, image mtusi-messenger:bench from working tree
|
||||
22:44:50 rolling-update: build done in 0s
|
||||
22:44:52 rolling-update: migrate done in 2s
|
||||
22:44:52 rolling-update: replica 1/2: started new 2ec1a8170637
|
||||
22:44:55 rolling-update: replica 1/2: new 2ec1a8170637 addr 172.31.0.8 slot api-3 UP in 3s
|
||||
22:44:55 rolling-update: replica 1/2: drain old 9c7a7d981f5d addr 172.31.0.4 slot api-2
|
||||
22:44:55 rolling-update: replica 1/2: old 9c7a7d981f5d idle after 0s
|
||||
22:44:55 rolling-update: replica 1/2: maint slot api-2, stopping old 9c7a7d981f5d
|
||||
22:44:55 rolling-update: replica 1/2: removed old 9c7a7d981f5d (stop 0s, exit 0)
|
||||
22:44:56 rolling-update: replica 1/2: slot api-2 ready
|
||||
22:44:56 rolling-update: replica 2/2: started new 9128897910ef
|
||||
22:44:57 rolling-update: replica 2/2: new 9128897910ef addr 172.31.0.4 slot api-2 UP in 1s
|
||||
22:44:58 rolling-update: replica 2/2: drain old 836903d5ebd9 addr 172.31.0.5 slot api-1
|
||||
22:44:58 rolling-update: replica 2/2: old 836903d5ebd9 idle after 0s
|
||||
22:44:58 rolling-update: replica 2/2: maint slot api-1, stopping old 836903d5ebd9
|
||||
22:44:58 rolling-update: replica 2/2: removed old 836903d5ebd9 (stop 0s, exit 0)
|
||||
22:44:59 rolling-update: replica 2/2: slot api-1 ready
|
||||
22:44:59 rolling-update: done: 2 replicas 2ec1a8170637 9128897910ef UP in 9s
|
||||
@@ -0,0 +1,3 @@
|
||||
step,new_instance,new_slot,old_instance,old_slot,new_created_s,new_up_s,ready_s,drain_s,drain_timeout,stop_s,old_removed_s,slot_ready_s,replace_s,errors,after_drain
|
||||
1,2ec1a8170637,api-3,9c7a7d981f5d,api-2,1.81,4.17,2.26,0.14,False,0.35,5.28,5.79,3.47,0,0
|
||||
2,9128897910ef,api-2,836903d5ebd9,api-1,5.99,7.15,1.07,0.14,False,0.39,8.26,8.78,2.28,0,0
|
||||
|
@@ -0,0 +1,49 @@
|
||||
phase,mode,window,start_s,end_s,seconds,role,instance,present_s,cpu_mean_cores,cpu_max_cores,cpu_seconds,throttled_ratio,pressure_share,mem_peak_mib,ok_rps,us_per_ok_request,flags
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,api-old,836903d5ebd9,12.99,0.2753,0.3418,3.575,0.0,0.0015,119.5,199.9,1377.2,
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,api-old,9c7a7d981f5d,12.99,0.2791,0.3529,3.624,0.0,0.0016,119.9,199.9,1396.2,
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,api-new,2ec1a8170637,0.0,0.0,,0.0,,0.0,,199.9,,absent
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,api-new,9128897910ef,0.0,0.0,,0.0,,0.0,,199.9,,absent
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,api-total,,12.99,0.5544,0.6947,7.199,0.0,0.0031,239.5,199.9,2773.4,
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,migrate,87f513915217,0.0,0.0,,0.0,,0.0,,199.9,,absent
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,postgres,2708a61ed609,12.99,0.0788,0.1032,1.024,0.0,0.0008,141.4,199.9,394.2,
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,haproxy,e1726165eec2,12.99,0.0299,0.0422,0.388,0.0,0.0006,35.8,199.9,149.6,
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,valkey,dbe0fec0557d,12.99,0.01,0.0195,0.13,0.0,0.0004,13.3,199.9,50.0,
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,k6,d3da387bc8da,12.99,0.128,0.3564,1.662,,0.0017,362.8,199.9,640.3,
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,sampler,565f94da69f4,12.99,0.0032,0.0036,0.041,0.0,0.0,2.0,199.9,16.0,
|
||||
rollout,rollout,before,-14.0,-1.02,12.99,vm,,12.99,0.9593,1.2642,12.457,,0.0039,,199.9,4798.9,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,api-old,836903d5ebd9,1.02,0.2874,0.2881,0.293,0.0,0.0006,119.7,200.2,,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,api-old,9c7a7d981f5d,1.02,0.2869,0.2888,0.292,0.0,0.0007,120.6,200.2,,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,api-new,2ec1a8170637,0.0,0.0,,0.0,,0.0,,200.2,,absent
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,api-new,9128897910ef,0.0,0.0,,0.0,,0.0,,200.2,,absent
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,api-total,,1.02,0.5743,0.5745,0.585,0.0,0.0013,240.3,200.2,,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,migrate,87f513915217,0.0,0.0,,0.0,,0.0,,200.2,,absent
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,postgres,2708a61ed609,1.02,0.0855,0.1032,0.087,0.0,0.0009,137.7,200.2,,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,haproxy,e1726165eec2,1.02,0.0545,0.0601,0.056,0.0,0.0005,35.9,200.2,,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,valkey,dbe0fec0557d,1.02,0.0075,0.0077,0.008,0.0,0.0002,12.5,200.2,,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,k6,d3da387bc8da,1.02,0.0918,0.0921,0.094,,0.0029,362.8,200.2,,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,sampler,565f94da69f4,1.02,0.0026,0.0029,0.003,0.0,0.0,1.9,200.2,,
|
||||
rollout,rollout,prepare,-1.02,0.0,1.02,vm,,1.02,2.0412,2.313,2.08,,0.0242,,200.2,,
|
||||
rollout,rollout,during,0.0,8.26,8.26,api-old,836903d5ebd9,7.22,0.2187,0.2987,1.808,0.0,0.0005,122.8,200.1,,partial
|
||||
rollout,rollout,during,0.0,8.26,8.26,api-old,9c7a7d981f5d,4.21,0.1273,0.2864,1.052,0.0,0.0004,122.2,200.1,,partial
|
||||
rollout,rollout,during,0.0,8.26,8.26,api-new,2ec1a8170637,6.36,0.2536,1.0393,2.096,0.0174,0.0011,110.6,200.1,,partial
|
||||
rollout,rollout,during,0.0,8.26,8.26,api-new,9128897910ef,2.18,0.1508,1.0913,1.246,0.0911,0.0011,106.2,200.1,,partial
|
||||
rollout,rollout,during,0.0,8.26,8.26,api-total,,8.26,0.7504,1.7661,6.201,0.0155,0.0031,348.8,200.1,,
|
||||
rollout,rollout,during,0.0,8.26,8.26,migrate,87f513915217,1.1,0.0698,0.7729,0.577,0.9091,0.0629,65.4,200.1,,partial
|
||||
rollout,rollout,during,0.0,8.26,8.26,postgres,2708a61ed609,8.26,0.0904,0.145,0.747,0.0,0.0007,158.2,200.1,,
|
||||
rollout,rollout,during,0.0,8.26,8.26,haproxy,e1726165eec2,8.26,0.0547,0.0959,0.452,0.0,0.0004,36.9,200.1,,
|
||||
rollout,rollout,during,0.0,8.26,8.26,valkey,dbe0fec0557d,8.26,0.0093,0.0178,0.077,0.0,0.0003,13.0,200.1,,
|
||||
rollout,rollout,during,0.0,8.26,8.26,k6,d3da387bc8da,8.26,0.0734,0.0921,0.607,,0.0028,363.5,200.1,,
|
||||
rollout,rollout,during,0.0,8.26,8.26,sampler,565f94da69f4,8.26,0.0027,0.003,0.023,0.0,0.0,1.7,200.1,,
|
||||
rollout,rollout,during,0.0,8.26,8.26,vm,,8.26,1.6978,2.3824,14.031,,0.0602,,200.1,,
|
||||
rollout,rollout,after,8.26,103.99,95.73,api-old,836903d5ebd9,0.0,0.0,,0.0,,0.0,,200.0,,absent
|
||||
rollout,rollout,after,8.26,103.99,95.73,api-old,9c7a7d981f5d,0.0,0.0,,0.0,,0.0,,200.0,,absent
|
||||
rollout,rollout,after,8.26,103.99,95.73,api-new,2ec1a8170637,95.73,0.2852,0.3614,27.307,0.0,0.0016,125.5,200.0,1426.0,
|
||||
rollout,rollout,after,8.26,103.99,95.73,api-new,9128897910ef,95.73,0.2895,0.3792,27.712,0.001,0.0014,125.5,200.0,1447.5,
|
||||
rollout,rollout,after,8.26,103.99,95.73,api-total,,95.73,0.5747,0.6717,55.018,0.0005,0.003,250.5,200.0,2873.5,
|
||||
rollout,rollout,after,8.26,103.99,95.73,migrate,87f513915217,0.0,0.0,,0.0,,0.0,,200.0,,absent
|
||||
rollout,rollout,after,8.26,103.99,95.73,postgres,2708a61ed609,95.73,0.0954,0.145,9.132,0.0,0.0008,201.9,200.0,477.0,
|
||||
rollout,rollout,after,8.26,103.99,95.73,haproxy,e1726165eec2,95.73,0.0251,0.0767,2.402,0.0,0.0005,45.3,200.0,125.5,
|
||||
rollout,rollout,after,8.26,103.99,95.73,valkey,dbe0fec0557d,95.73,0.01,0.0207,0.955,0.0,0.0004,13.5,200.0,50.0,
|
||||
rollout,rollout,after,8.26,103.99,95.73,k6,d3da387bc8da,95.73,0.0952,0.3909,9.112,,0.0015,455.3,200.0,476.0,
|
||||
rollout,rollout,after,8.26,103.99,95.73,sampler,565f94da69f4,95.73,0.0033,0.004,0.318,0.0,0.0,2.2,200.0,16.5,
|
||||
rollout,rollout,after,8.26,103.99,95.73,vm,,95.73,0.9683,1.6123,92.692,,0.0037,,200.0,4841.5,
|
||||
|
@@ -0,0 +1,2 @@
|
||||
label,from_commit,replicas,revision_before,revision_after,build_s,migrate_s,rollout_s,script_exit,requests,errors_before,codes_before,errors_during,codes_during,errors_after,codes_after,min_success,seconds_below,ok_rps_before,ok_rps_during_min,p95_before_ms,p95_during_max_ms,first_new_s,last_old_s,old_after_drain,writes,writes_failed,retried_ok,write_chat_messages,distinct_texts,dropped_iterations,flags
|
||||
attachments,e1161f7,2,e656d6bbe1a4,ef25d50b01a8,0,2,8.26,0,24002,0,,0,,0,,1.0,0,200.0,200,4,43,4.18,7.61,0,12001,0,0,12001,12001,0,
|
||||
|
@@ -0,0 +1,968 @@
|
||||
phase,window,t_s,role,instance,cpu_cores,throttled_ms,pressure_ms,mem_mib
|
||||
rollout,warmup,-15.86,api-old,836903d5ebd9,0.0412,0.0,0.25,105.6
|
||||
rollout,warmup,-15.86,api-old,9c7a7d981f5d,0.0358,0.0,0.23,105.7
|
||||
rollout,warmup,-15.86,postgres,2708a61ed609,0.0282,0.0,0.54,136.5
|
||||
rollout,warmup,-15.86,haproxy,e1726165eec2,0.0054,0.0,0.12,20.3
|
||||
rollout,warmup,-15.86,valkey,dbe0fec0557d,0.0029,0.0,0.15,12.6
|
||||
rollout,warmup,-15.86,k6,d3da387bc8da,0.6005,0.0,1.42,221.2
|
||||
rollout,warmup,-15.86,sampler,565f94da69f4,0.0035,0.0,0.31,2.3
|
||||
rollout,warmup,-15.86,vm,,0.9826,,6.42,
|
||||
rollout,warmup,-14.86,api-old,836903d5ebd9,0.2512,0.0,1.83,106.1
|
||||
rollout,warmup,-14.86,api-old,9c7a7d981f5d,0.2537,0.0,1.64,106.9
|
||||
rollout,warmup,-14.86,postgres,2708a61ed609,0.0764,0.0,0.53,135.6
|
||||
rollout,warmup,-14.86,haproxy,e1726165eec2,0.0293,0.0,0.6,22.7
|
||||
rollout,warmup,-14.86,valkey,dbe0fec0557d,0.0066,0.0,0.39,12.4
|
||||
rollout,warmup,-14.86,k6,d3da387bc8da,0.1071,0.0,1.64,240.0
|
||||
rollout,warmup,-14.86,sampler,565f94da69f4,0.003,0.0,0.01,1.7
|
||||
rollout,warmup,-14.86,vm,,0.887,,2.86,
|
||||
rollout,before,-13.85,api-old,836903d5ebd9,0.2657,0.0,0.75,107.3
|
||||
rollout,before,-13.85,api-old,9c7a7d981f5d,0.2652,0.0,1.15,107.5
|
||||
rollout,before,-13.85,postgres,2708a61ed609,0.0721,0.0,0.65,136.0
|
||||
rollout,before,-13.85,haproxy,e1726165eec2,0.0302,0.0,0.57,24.1
|
||||
rollout,before,-13.85,valkey,dbe0fec0557d,0.0072,0.0,0.25,12.7
|
||||
rollout,before,-13.85,k6,d3da387bc8da,0.1131,0.0,1.41,261.4
|
||||
rollout,before,-13.85,sampler,565f94da69f4,0.0035,0.0,0.02,1.6
|
||||
rollout,before,-13.85,vm,,0.8965,,2.69,
|
||||
rollout,before,-12.85,api-old,836903d5ebd9,0.2628,0.0,1.82,109.1
|
||||
rollout,before,-12.85,api-old,9c7a7d981f5d,0.2632,0.0,3.11,109.2
|
||||
rollout,before,-12.85,postgres,2708a61ed609,0.0702,0.0,1.52,136.7
|
||||
rollout,before,-12.85,haproxy,e1726165eec2,0.0301,0.0,1.64,26.4
|
||||
rollout,before,-12.85,valkey,dbe0fec0557d,0.007,0.0,0.27,12.7
|
||||
rollout,before,-12.85,k6,d3da387bc8da,0.3301,0.0,6.46,268.5
|
||||
rollout,before,-12.85,sampler,565f94da69f4,0.0033,0.0,0.07,1.7
|
||||
rollout,before,-12.85,vm,,1.0658,,7.11,
|
||||
rollout,before,-11.85,api-old,836903d5ebd9,0.2637,0.0,0.76,110.3
|
||||
rollout,before,-11.85,api-old,9c7a7d981f5d,0.2619,0.0,0.83,110.5
|
||||
rollout,before,-11.85,postgres,2708a61ed609,0.078,0.0,0.61,141.4
|
||||
rollout,before,-11.85,haproxy,e1726165eec2,0.0422,0.0,0.27,28.6
|
||||
rollout,before,-11.85,valkey,dbe0fec0557d,0.0187,0.0,0.31,12.7
|
||||
rollout,before,-11.85,k6,d3da387bc8da,0.1206,0.0,1.08,269.9
|
||||
rollout,before,-11.85,sampler,565f94da69f4,0.003,0.0,0.01,1.5
|
||||
rollout,before,-11.85,vm,,0.9665,,3.51,
|
||||
rollout,before,-10.84,api-old,836903d5ebd9,0.2602,0.0,1.66,111.9
|
||||
rollout,before,-10.84,api-old,9c7a7d981f5d,0.2604,0.0,1.2,112.3
|
||||
rollout,before,-10.84,postgres,2708a61ed609,0.0881,0.0,0.64,136.8
|
||||
rollout,before,-10.84,haproxy,e1726165eec2,0.0297,0.0,0.64,30.5
|
||||
rollout,before,-10.84,valkey,dbe0fec0557d,0.0068,0.0,0.33,12.6
|
||||
rollout,before,-10.84,k6,d3da387bc8da,0.1166,0.0,1.07,281.2
|
||||
rollout,before,-10.84,sampler,565f94da69f4,0.0028,0.0,0.01,1.7
|
||||
rollout,before,-10.84,vm,,0.8671,,2.76,
|
||||
rollout,before,-9.84,api-old,836903d5ebd9,0.2647,0.0,0.74,113.9
|
||||
rollout,before,-9.84,api-old,9c7a7d981f5d,0.2673,0.0,1.11,113.8
|
||||
rollout,before,-9.84,postgres,2708a61ed609,0.0723,0.0,0.84,137.3
|
||||
rollout,before,-9.84,haproxy,e1726165eec2,0.0294,0.0,0.56,32.5
|
||||
rollout,before,-9.84,valkey,dbe0fec0557d,0.0067,0.0,0.38,12.4
|
||||
rollout,before,-9.84,k6,d3da387bc8da,0.1089,0.0,0.97,295.7
|
||||
rollout,before,-9.84,sampler,565f94da69f4,0.0034,0.0,0.02,1.7
|
||||
rollout,before,-9.84,vm,,0.8767,,2.81,
|
||||
rollout,before,-8.84,api-old,836903d5ebd9,0.2754,0.0,1.42,115.0
|
||||
rollout,before,-8.84,api-old,9c7a7d981f5d,0.277,0.0,1.3,114.8
|
||||
rollout,before,-8.84,postgres,2708a61ed609,0.0754,0.0,0.74,137.0
|
||||
rollout,before,-8.84,haproxy,e1726165eec2,0.0262,0.0,0.71,32.8
|
||||
rollout,before,-8.84,valkey,dbe0fec0557d,0.0073,0.0,0.27,12.3
|
||||
rollout,before,-8.84,k6,d3da387bc8da,0.0813,0.0,1.23,304.2
|
||||
rollout,before,-8.84,sampler,565f94da69f4,0.0035,0.0,0.01,1.9
|
||||
rollout,before,-8.84,vm,,0.8965,,2.73,
|
||||
rollout,before,-7.83,api-old,836903d5ebd9,0.3418,0.0,0.91,115.6
|
||||
rollout,before,-7.83,api-old,9c7a7d981f5d,0.3529,0.0,1.26,115.8
|
||||
rollout,before,-7.83,postgres,2708a61ed609,0.0756,0.0,1.08,137.6
|
||||
rollout,before,-7.83,haproxy,e1726165eec2,0.0266,0.0,0.28,33.2
|
||||
rollout,before,-7.83,valkey,dbe0fec0557d,0.0076,0.0,0.35,12.2
|
||||
rollout,before,-7.83,k6,d3da387bc8da,0.0822,0.0,1.02,313.9
|
||||
rollout,before,-7.83,sampler,565f94da69f4,0.0033,0.0,0.04,2.0
|
||||
rollout,before,-7.83,vm,,1.2642,,6.36,
|
||||
rollout,before,-6.83,api-old,836903d5ebd9,0.2638,0.0,1.16,115.9
|
||||
rollout,before,-6.83,api-old,9c7a7d981f5d,0.2702,0.0,1.39,116.2
|
||||
rollout,before,-6.83,postgres,2708a61ed609,0.0712,0.0,0.64,136.9
|
||||
rollout,before,-6.83,haproxy,e1726165eec2,0.0371,0.0,0.47,33.5
|
||||
rollout,before,-6.83,valkey,dbe0fec0557d,0.0195,0.0,0.27,13.2
|
||||
rollout,before,-6.83,k6,d3da387bc8da,0.0731,0.0,1.13,322.1
|
||||
rollout,before,-6.83,sampler,565f94da69f4,0.0028,0.0,0.02,1.4
|
||||
rollout,before,-6.83,vm,,0.9067,,3.23,
|
||||
rollout,before,-5.82,api-old,836903d5ebd9,0.2749,0.0,1.57,116.7
|
||||
rollout,before,-5.82,api-old,9c7a7d981f5d,0.2778,0.0,2.14,117.7
|
||||
rollout,before,-5.82,postgres,2708a61ed609,0.0989,0.0,0.8,137.2
|
||||
rollout,before,-5.82,haproxy,e1726165eec2,0.0253,0.0,0.67,34.1
|
||||
rollout,before,-5.82,valkey,dbe0fec0557d,0.0072,0.0,0.22,13.3
|
||||
rollout,before,-5.82,k6,d3da387bc8da,0.0754,0.0,0.98,327.8
|
||||
rollout,before,-5.82,sampler,565f94da69f4,0.003,0.0,0.01,1.7
|
||||
rollout,before,-5.82,vm,,0.9168,,3.13,
|
||||
rollout,before,-4.82,api-old,836903d5ebd9,0.2742,0.0,2.08,117.7
|
||||
rollout,before,-4.82,api-old,9c7a7d981f5d,0.2782,0.0,2.28,118.3
|
||||
rollout,before,-4.82,postgres,2708a61ed609,0.0742,0.0,0.74,137.3
|
||||
rollout,before,-4.82,haproxy,e1726165eec2,0.0267,0.0,0.55,35.0
|
||||
rollout,before,-4.82,valkey,dbe0fec0557d,0.0073,0.0,0.35,13.1
|
||||
rollout,before,-4.82,k6,d3da387bc8da,0.0751,0.0,1.25,335.9
|
||||
rollout,before,-4.82,sampler,565f94da69f4,0.0031,0.0,0.02,1.6
|
||||
rollout,before,-4.82,vm,,0.7871,,2.65,
|
||||
rollout,before,-3.82,api-old,836903d5ebd9,0.2736,0.0,1.09,118.6
|
||||
rollout,before,-3.82,api-old,9c7a7d981f5d,0.2751,0.0,1.26,119.5
|
||||
rollout,before,-3.82,postgres,2708a61ed609,0.074,0.0,1.06,137.5
|
||||
rollout,before,-3.82,haproxy,e1726165eec2,0.0252,0.0,0.36,35.3
|
||||
rollout,before,-3.82,valkey,dbe0fec0557d,0.0073,0.0,0.5,13.1
|
||||
rollout,before,-3.82,k6,d3da387bc8da,0.073,0.0,1.06,344.6
|
||||
rollout,before,-3.82,sampler,565f94da69f4,0.0032,0.0,0.01,1.9
|
||||
rollout,before,-3.82,vm,,0.8364,,2.4,
|
||||
rollout,before,-2.81,api-old,836903d5ebd9,0.2785,0.0,1.22,119.4
|
||||
rollout,before,-2.81,api-old,9c7a7d981f5d,0.2825,0.0,1.57,119.6
|
||||
rollout,before,-2.81,postgres,2708a61ed609,0.0761,0.0,0.68,137.1
|
||||
rollout,before,-2.81,haproxy,e1726165eec2,0.0226,0.0,0.74,35.4
|
||||
rollout,before,-2.81,valkey,dbe0fec0557d,0.0071,0.0,0.44,12.8
|
||||
rollout,before,-2.81,k6,d3da387bc8da,0.075,0.0,1.09,352.4
|
||||
rollout,before,-2.81,sampler,565f94da69f4,0.0036,0.0,0.01,2.0
|
||||
rollout,before,-2.81,vm,,0.8564,,2.62,
|
||||
rollout,before,-1.81,api-old,836903d5ebd9,0.2645,0.0,4.12,119.5
|
||||
rollout,before,-1.81,api-old,9c7a7d981f5d,0.2768,0.0,1.76,119.9
|
||||
rollout,before,-1.81,postgres,2708a61ed609,0.0737,0.0,0.87,137.4
|
||||
rollout,before,-1.81,haproxy,e1726165eec2,0.0343,0.0,0.83,35.8
|
||||
rollout,before,-1.81,valkey,dbe0fec0557d,0.0194,0.0,1.71,12.6
|
||||
rollout,before,-1.81,k6,d3da387bc8da,0.3564,0.0,3.02,362.8
|
||||
rollout,before,-1.81,sampler,565f94da69f4,0.003,0.0,0.02,1.8
|
||||
rollout,before,-1.81,vm,,1.2355,,8.15,
|
||||
rollout,prepare,-0.81,api-old,836903d5ebd9,0.2848,0.0,1.07,119.7
|
||||
rollout,prepare,-0.81,api-old,9c7a7d981f5d,0.2888,0.0,1.49,120.6
|
||||
rollout,prepare,-0.81,postgres,2708a61ed609,0.1032,0.0,0.6,137.7
|
||||
rollout,prepare,-0.81,haproxy,e1726165eec2,0.0335,0.0,0.55,35.9
|
||||
rollout,prepare,-0.81,valkey,dbe0fec0557d,0.0077,0.0,0.29,12.5
|
||||
rollout,prepare,-0.81,k6,d3da387bc8da,0.0905,0.0,1.55,362.8
|
||||
rollout,prepare,-0.81,sampler,565f94da69f4,0.0029,0.0,0.02,1.9
|
||||
rollout,prepare,-0.81,vm,,1.0167,,3.84,
|
||||
rollout,during,0.2,api-old,836903d5ebd9,0.2881,0.0,0.5,120.4
|
||||
rollout,during,0.2,api-old,9c7a7d981f5d,0.2864,0.0,0.49,120.7
|
||||
rollout,during,0.2,migrate,87f513915217,0.7729,0.0,348.76,11.6
|
||||
rollout,during,0.2,postgres,2708a61ed609,0.0808,0.0,0.97,137.8
|
||||
rollout,during,0.2,haproxy,e1726165eec2,0.0601,0.0,0.46,36.4
|
||||
rollout,during,0.2,valkey,dbe0fec0557d,0.0075,0.0,0.2,11.7
|
||||
rollout,during,0.2,k6,d3da387bc8da,0.0921,0.0,3.31,363.5
|
||||
rollout,during,0.2,sampler,565f94da69f4,0.0025,0.0,0.01,1.5
|
||||
rollout,during,0.2,vm,,2.313,,29.57,
|
||||
rollout,during,1.2,api-old,836903d5ebd9,0.2044,0.0,0.51,120.9
|
||||
rollout,during,1.2,api-old,9c7a7d981f5d,0.2069,0.0,1.42,120.8
|
||||
rollout,during,1.2,migrate,87f513915217,0.4996,496.1,484.14,65.4
|
||||
rollout,during,1.2,postgres,2708a61ed609,0.0552,0.0,0.53,139.1
|
||||
rollout,during,1.2,haproxy,e1726165eec2,0.0155,0.0,0.53,36.0
|
||||
rollout,during,1.2,valkey,dbe0fec0557d,0.0051,0.0,0.24,11.5
|
||||
rollout,during,1.2,k6,d3da387bc8da,0.0577,0.0,6.89,362.4
|
||||
rollout,during,1.2,sampler,565f94da69f4,0.0024,0.0,0.05,1.7
|
||||
rollout,during,1.2,vm,,1.2556,,335.7,
|
||||
rollout,during,2.2,api-old,836903d5ebd9,0.2725,0.0,0.75,121.5
|
||||
rollout,during,2.2,api-old,9c7a7d981f5d,0.2669,0.0,0.52,121.8
|
||||
rollout,during,2.2,api-new,2ec1a8170637,1.0393,0.0,0.06,20.8
|
||||
rollout,during,2.2,postgres,2708a61ed609,0.0913,0.0,1.17,138.6
|
||||
rollout,during,2.2,haproxy,e1726165eec2,0.0332,0.0,0.73,36.3
|
||||
rollout,during,2.2,valkey,dbe0fec0557d,0.0075,0.0,0.44,11.7
|
||||
rollout,during,2.2,k6,d3da387bc8da,0.0844,0.0,2.36,362.5
|
||||
rollout,during,2.2,sampler,565f94da69f4,0.0028,0.0,0.03,1.2
|
||||
rollout,during,2.2,vm,,2.3824,,92.05,
|
||||
rollout,during,3.21,api-old,836903d5ebd9,0.2489,0.0,0.9,121.9
|
||||
rollout,during,3.21,api-old,9c7a7d981f5d,0.2475,0.0,0.57,122.2
|
||||
rollout,during,3.21,api-new,2ec1a8170637,0.6598,90.35,5.79,103.2
|
||||
rollout,during,3.21,postgres,2708a61ed609,0.087,0.0,0.55,137.7
|
||||
rollout,during,3.21,haproxy,e1726165eec2,0.0468,0.0,0.32,36.9
|
||||
rollout,during,3.21,valkey,dbe0fec0557d,0.0178,0.0,0.3,13.0
|
||||
rollout,during,3.21,k6,d3da387bc8da,0.0731,0.0,2.16,363.0
|
||||
rollout,during,3.21,sampler,565f94da69f4,0.003,0.0,0.02,1.5
|
||||
rollout,during,3.21,vm,,1.7045,,10.65,
|
||||
rollout,during,4.21,api-old,836903d5ebd9,0.2678,0.0,0.57,121.5
|
||||
rollout,during,4.21,api-old,9c7a7d981f5d,0.2707,0.0,0.73,122.0
|
||||
rollout,during,4.21,api-new,2ec1a8170637,0.0433,0.0,0.12,105.3
|
||||
rollout,during,4.21,postgres,2708a61ed609,0.1059,0.0,0.82,142.2
|
||||
rollout,during,4.21,haproxy,e1726165eec2,0.0498,0.0,0.36,36.7
|
||||
rollout,during,4.21,valkey,dbe0fec0557d,0.0073,0.0,0.38,13.0
|
||||
rollout,during,4.21,k6,d3da387bc8da,0.0762,0.0,1.42,362.9
|
||||
rollout,during,4.21,sampler,565f94da69f4,0.0029,0.0,0.01,1.4
|
||||
rollout,during,4.21,vm,,1.126,,5.32,
|
||||
rollout,during,5.21,api-old,836903d5ebd9,0.216,0.0,0.4,121.9
|
||||
rollout,during,5.21,api-new,2ec1a8170637,0.2387,0.0,0.48,105.6
|
||||
rollout,during,5.21,postgres,2708a61ed609,0.0953,0.0,0.59,132.5
|
||||
rollout,during,5.21,haproxy,e1726165eec2,0.0835,0.0,0.35,32.9
|
||||
rollout,during,5.21,valkey,dbe0fec0557d,0.0068,0.0,0.15,11.5
|
||||
rollout,during,5.21,k6,d3da387bc8da,0.0697,0.0,1.92,362.6
|
||||
rollout,during,5.21,sampler,565f94da69f4,0.0026,0.0,0.01,1.5
|
||||
rollout,during,5.21,vm,,1.3857,,6.71,
|
||||
rollout,during,6.22,api-old,836903d5ebd9,0.2987,0.0,0.44,122.2
|
||||
rollout,during,6.22,api-new,2ec1a8170637,0.3761,0.0,1.29,108.1
|
||||
rollout,during,6.22,api-new,9128897910ef,1.0913,0.0,0.33,16.2
|
||||
rollout,during,6.22,postgres,2708a61ed609,0.1172,0.0,1.15,149.7
|
||||
rollout,during,6.22,haproxy,e1726165eec2,0.0589,0.0,0.31,33.7
|
||||
rollout,during,6.22,valkey,dbe0fec0557d,0.0083,0.0,0.23,12.2
|
||||
rollout,during,6.22,k6,d3da387bc8da,0.0867,0.0,2.82,363.2
|
||||
rollout,during,6.22,sampler,565f94da69f4,0.0028,0.0,0.04,1.5
|
||||
rollout,during,6.22,vm,,2.2716,,18.94,
|
||||
rollout,during,7.22,api-old,836903d5ebd9,0.2363,0.0,0.77,122.8
|
||||
rollout,during,7.22,api-new,2ec1a8170637,0.2323,0.0,0.37,109.3
|
||||
rollout,during,7.22,api-new,9128897910ef,0.855,69.94,8.03,105.0
|
||||
rollout,during,7.22,postgres,2708a61ed609,0.0806,0.0,0.52,158.2
|
||||
rollout,during,7.22,haproxy,e1726165eec2,0.0518,0.0,0.4,34.4
|
||||
rollout,during,7.22,valkey,dbe0fec0557d,0.0061,0.0,0.19,12.0
|
||||
rollout,during,7.22,k6,d3da387bc8da,0.0641,0.0,2.74,362.7
|
||||
rollout,during,7.22,sampler,565f94da69f4,0.0026,0.0,0.01,1.5
|
||||
rollout,during,7.22,vm,,1.8031,,12.77,
|
||||
rollout,during,8.23,api-new,2ec1a8170637,0.2214,0.0,0.61,110.6
|
||||
rollout,during,8.23,api-new,9128897910ef,0.2218,0.0,0.58,106.2
|
||||
rollout,during,8.23,postgres,2708a61ed609,0.0902,0.0,0.53,144.5
|
||||
rollout,during,8.23,haproxy,e1726165eec2,0.0959,0.0,0.36,31.1
|
||||
rollout,during,8.23,valkey,dbe0fec0557d,0.0159,0.0,0.26,12.7
|
||||
rollout,during,8.23,k6,d3da387bc8da,0.0716,0.0,1.83,362.2
|
||||
rollout,during,8.23,sampler,565f94da69f4,0.0029,0.0,0.02,1.5
|
||||
rollout,during,8.23,vm,,1.5353,,7.73,
|
||||
rollout,after,9.23,api-new,2ec1a8170637,0.2742,0.0,0.55,112.4
|
||||
rollout,after,9.23,api-new,9128897910ef,0.3792,0.2,0.52,108.1
|
||||
rollout,after,9.23,postgres,2708a61ed609,0.145,0.0,0.7,167.0
|
||||
rollout,after,9.23,haproxy,e1726165eec2,0.0767,0.0,0.58,30.7
|
||||
rollout,after,9.23,valkey,dbe0fec0557d,0.0078,0.0,0.16,12.6
|
||||
rollout,after,9.23,k6,d3da387bc8da,0.0788,0.0,2.76,363.4
|
||||
rollout,after,9.23,sampler,565f94da69f4,0.003,0.0,0.01,1.4
|
||||
rollout,after,9.23,vm,,1.6123,,10.02,
|
||||
rollout,after,10.23,api-new,2ec1a8170637,0.2861,0.0,1.72,113.9
|
||||
rollout,after,10.23,api-new,9128897910ef,0.2821,0.0,0.94,109.1
|
||||
rollout,after,10.23,postgres,2708a61ed609,0.0908,0.0,0.53,169.4
|
||||
rollout,after,10.23,haproxy,e1726165eec2,0.0352,0.0,0.66,31.7
|
||||
rollout,after,10.23,valkey,dbe0fec0557d,0.0076,0.0,0.5,12.3
|
||||
rollout,after,10.23,k6,d3da387bc8da,0.0813,0.0,1.11,367.4
|
||||
rollout,after,10.23,sampler,565f94da69f4,0.0023,0.0,0.01,1.5
|
||||
rollout,after,10.23,vm,,0.9074,,2.29,
|
||||
rollout,after,11.24,api-new,2ec1a8170637,0.3096,0.0,1.12,114.2
|
||||
rollout,after,11.24,api-new,9128897910ef,0.3089,0.0,0.95,111.1
|
||||
rollout,after,11.24,postgres,2708a61ed609,0.0894,0.0,0.49,169.3
|
||||
rollout,after,11.24,haproxy,e1726165eec2,0.0359,0.0,0.6,32.6
|
||||
rollout,after,11.24,valkey,dbe0fec0557d,0.0086,0.0,0.52,12.3
|
||||
rollout,after,11.24,k6,d3da387bc8da,0.0868,0.0,1.44,375.5
|
||||
rollout,after,11.24,sampler,565f94da69f4,0.0029,0.0,0.01,1.8
|
||||
rollout,after,11.24,vm,,0.997,,2.42,
|
||||
rollout,after,12.24,api-new,2ec1a8170637,0.2923,0.0,2.06,115.6
|
||||
rollout,after,12.24,api-new,9128897910ef,0.2951,0.0,1.26,112.7
|
||||
rollout,after,12.24,postgres,2708a61ed609,0.0896,0.0,0.83,170.1
|
||||
rollout,after,12.24,haproxy,e1726165eec2,0.0336,0.0,0.64,32.9
|
||||
rollout,after,12.24,valkey,dbe0fec0557d,0.0082,0.0,1.36,12.3
|
||||
rollout,after,12.24,k6,d3da387bc8da,0.0846,0.0,2.07,386.0
|
||||
rollout,after,12.24,sampler,565f94da69f4,0.003,0.0,0.01,1.9
|
||||
rollout,after,12.24,vm,,1.1549,,7.11,
|
||||
rollout,after,13.24,api-new,2ec1a8170637,0.2654,0.0,1.6,116.3
|
||||
rollout,after,13.24,api-new,9128897910ef,0.2735,0.0,1.16,113.9
|
||||
rollout,after,13.24,postgres,2708a61ed609,0.0792,0.0,1.31,170.6
|
||||
rollout,after,13.24,haproxy,e1726165eec2,0.0403,0.0,0.46,33.6
|
||||
rollout,after,13.24,valkey,dbe0fec0557d,0.0193,0.0,0.55,13.0
|
||||
rollout,after,13.24,k6,d3da387bc8da,0.3642,0.0,6.86,397.9
|
||||
rollout,after,13.24,sampler,565f94da69f4,0.0029,0.0,0.0,1.7
|
||||
rollout,after,13.24,vm,,1.2156,,9.29,
|
||||
rollout,after,14.25,api-new,2ec1a8170637,0.2787,0.0,1.36,116.9
|
||||
rollout,after,14.25,api-new,9128897910ef,0.2784,0.0,0.78,115.2
|
||||
rollout,after,14.25,postgres,2708a61ed609,0.1109,0.0,0.74,170.5
|
||||
rollout,after,14.25,haproxy,e1726165eec2,0.0287,0.0,0.63,34.1
|
||||
rollout,after,14.25,valkey,dbe0fec0557d,0.0076,0.0,0.31,13.0
|
||||
rollout,after,14.25,k6,d3da387bc8da,0.0883,0.0,1.09,398.2
|
||||
rollout,after,14.25,sampler,565f94da69f4,0.0031,0.0,0.01,1.6
|
||||
rollout,after,14.25,vm,,0.9365,,3.43,
|
||||
rollout,after,15.25,api-new,2ec1a8170637,0.2747,0.0,1.29,118.0
|
||||
rollout,after,15.25,api-new,9128897910ef,0.2784,0.0,2.06,115.8
|
||||
rollout,after,15.25,postgres,2708a61ed609,0.083,0.0,0.63,170.5
|
||||
rollout,after,15.25,haproxy,e1726165eec2,0.0278,0.0,0.43,34.1
|
||||
rollout,after,15.25,valkey,dbe0fec0557d,0.0075,0.0,0.33,13.1
|
||||
rollout,after,15.25,k6,d3da387bc8da,0.0871,0.0,1.23,398.1
|
||||
rollout,after,15.25,sampler,565f94da69f4,0.003,0.0,0.01,1.6
|
||||
rollout,after,15.25,vm,,0.8868,,2.93,
|
||||
rollout,after,16.25,api-new,2ec1a8170637,0.3331,0.0,2.07,118.3
|
||||
rollout,after,16.25,api-new,9128897910ef,0.2644,0.0,1.21,117.1
|
||||
rollout,after,16.25,postgres,2708a61ed609,0.0865,0.0,0.6,174.5
|
||||
rollout,after,16.25,haproxy,e1726165eec2,0.0258,0.0,0.44,34.6
|
||||
rollout,after,16.25,valkey,dbe0fec0557d,0.0074,0.0,0.59,13.2
|
||||
rollout,after,16.25,k6,d3da387bc8da,0.0794,0.0,1.22,398.4
|
||||
rollout,after,16.25,sampler,565f94da69f4,0.0029,0.0,0.01,1.6
|
||||
rollout,after,16.25,vm,,0.9565,,2.9,
|
||||
rollout,after,17.26,api-new,2ec1a8170637,0.28,0.0,1.32,119.0
|
||||
rollout,after,17.26,api-new,9128897910ef,0.283,0.0,0.87,117.7
|
||||
rollout,after,17.26,postgres,2708a61ed609,0.0868,0.0,0.73,175.0
|
||||
rollout,after,17.26,haproxy,e1726165eec2,0.0275,0.0,0.46,35.5
|
||||
rollout,after,17.26,valkey,dbe0fec0557d,0.0075,0.0,0.29,12.7
|
||||
rollout,after,17.26,k6,d3da387bc8da,0.0858,0.0,1.76,398.7
|
||||
rollout,after,17.26,sampler,565f94da69f4,0.0033,0.0,0.04,1.6
|
||||
rollout,after,17.26,vm,,0.9065,,2.63,
|
||||
rollout,after,18.26,api-new,2ec1a8170637,0.2769,0.0,1.05,119.9
|
||||
rollout,after,18.26,api-new,9128897910ef,0.3569,0.0,1.08,118.3
|
||||
rollout,after,18.26,postgres,2708a61ed609,0.091,0.0,0.63,179.5
|
||||
rollout,after,18.26,haproxy,e1726165eec2,0.0393,0.0,0.34,35.5
|
||||
rollout,after,18.26,valkey,dbe0fec0557d,0.0193,0.0,0.31,12.9
|
||||
rollout,after,18.26,k6,d3da387bc8da,0.0851,0.0,1.49,398.1
|
||||
rollout,after,18.26,sampler,565f94da69f4,0.0032,0.0,0.01,1.8
|
||||
rollout,after,18.26,vm,,1.0452,,3.2,
|
||||
rollout,after,19.27,api-new,2ec1a8170637,0.2672,0.0,1.93,120.2
|
||||
rollout,after,19.27,api-new,9128897910ef,0.2739,0.0,1.66,118.9
|
||||
rollout,after,19.27,postgres,2708a61ed609,0.1077,0.0,0.77,179.1
|
||||
rollout,after,19.27,haproxy,e1726165eec2,0.025,0.0,0.4,36.1
|
||||
rollout,after,19.27,valkey,dbe0fec0557d,0.0071,0.0,0.53,12.2
|
||||
rollout,after,19.27,k6,d3da387bc8da,0.0751,0.0,2.12,398.3
|
||||
rollout,after,19.27,sampler,565f94da69f4,0.0031,0.0,0.02,1.5
|
||||
rollout,after,19.27,vm,,0.9073,,3.42,
|
||||
rollout,after,20.27,api-new,2ec1a8170637,0.272,0.0,1.4,120.5
|
||||
rollout,after,20.27,api-new,9128897910ef,0.2779,0.0,1.32,119.6
|
||||
rollout,after,20.27,postgres,2708a61ed609,0.0852,0.0,0.86,179.3
|
||||
rollout,after,20.27,haproxy,e1726165eec2,0.0233,0.0,0.41,35.2
|
||||
rollout,after,20.27,valkey,dbe0fec0557d,0.0074,0.0,0.52,11.9
|
||||
rollout,after,20.27,k6,d3da387bc8da,0.0762,0.0,1.05,398.2
|
||||
rollout,after,20.27,sampler,565f94da69f4,0.0038,0.0,0.07,1.6
|
||||
rollout,after,20.27,vm,,0.8159,,2.52,
|
||||
rollout,after,21.28,api-new,2ec1a8170637,0.2845,0.0,1.46,120.8
|
||||
rollout,after,21.28,api-new,9128897910ef,0.29,0.0,1.88,120.3
|
||||
rollout,after,21.28,postgres,2708a61ed609,0.0876,0.0,0.95,179.7
|
||||
rollout,after,21.28,haproxy,e1726165eec2,0.0246,0.0,0.6,36.0
|
||||
rollout,after,21.28,valkey,dbe0fec0557d,0.0077,0.0,0.31,12.2
|
||||
rollout,after,21.28,k6,d3da387bc8da,0.0801,0.0,1.25,398.0
|
||||
rollout,after,21.28,sampler,565f94da69f4,0.0036,0.0,0.01,1.7
|
||||
rollout,after,21.28,vm,,0.8863,,2.72,
|
||||
rollout,after,22.28,api-new,2ec1a8170637,0.2889,0.0,0.94,121.6
|
||||
rollout,after,22.28,api-new,9128897910ef,0.2964,0.0,1.65,120.4
|
||||
rollout,after,22.28,postgres,2708a61ed609,0.0912,0.0,1.32,179.8
|
||||
rollout,after,22.28,haproxy,e1726165eec2,0.0261,0.0,0.45,36.0
|
||||
rollout,after,22.28,valkey,dbe0fec0557d,0.0077,0.0,0.22,12.4
|
||||
rollout,after,22.28,k6,d3da387bc8da,0.0822,0.0,1.49,398.3
|
||||
rollout,after,22.28,sampler,565f94da69f4,0.0035,0.0,0.03,1.8
|
||||
rollout,after,22.28,vm,,1.1249,,8.45,
|
||||
rollout,after,23.28,api-new,2ec1a8170637,0.2688,0.0,1.32,121.5
|
||||
rollout,after,23.28,api-new,9128897910ef,0.2747,0.0,0.81,120.3
|
||||
rollout,after,23.28,postgres,2708a61ed609,0.1075,0.0,0.6,182.9
|
||||
rollout,after,23.28,haproxy,e1726165eec2,0.0341,0.0,0.46,36.1
|
||||
rollout,after,23.28,valkey,dbe0fec0557d,0.0199,0.0,0.35,13.4
|
||||
rollout,after,23.28,k6,d3da387bc8da,0.0761,0.0,1.11,398.5
|
||||
rollout,after,23.28,sampler,565f94da69f4,0.0032,0.0,0.01,1.5
|
||||
rollout,after,23.28,vm,,0.9556,,4.1,
|
||||
rollout,after,24.29,api-new,2ec1a8170637,0.2802,0.0,1.42,121.4
|
||||
rollout,after,24.29,api-new,9128897910ef,0.2804,0.0,1.59,121.0
|
||||
rollout,after,24.29,postgres,2708a61ed609,0.1147,0.0,0.9,182.5
|
||||
rollout,after,24.29,haproxy,e1726165eec2,0.0244,0.0,0.39,36.5
|
||||
rollout,after,24.29,valkey,dbe0fec0557d,0.0075,0.0,0.24,13.3
|
||||
rollout,after,24.29,k6,d3da387bc8da,0.0788,0.0,1.09,397.7
|
||||
rollout,after,24.29,sampler,565f94da69f4,0.0031,0.0,0.01,1.8
|
||||
rollout,after,24.29,vm,,0.9364,,4.09,
|
||||
rollout,after,25.29,api-new,2ec1a8170637,0.2808,0.0,1.32,122.0
|
||||
rollout,after,25.29,api-new,9128897910ef,0.2823,0.0,1.24,121.3
|
||||
rollout,after,25.29,postgres,2708a61ed609,0.0861,0.0,0.66,182.7
|
||||
rollout,after,25.29,haproxy,e1726165eec2,0.0236,0.0,0.66,36.7
|
||||
rollout,after,25.29,valkey,dbe0fec0557d,0.0076,0.0,0.56,12.7
|
||||
rollout,after,25.29,k6,d3da387bc8da,0.0788,0.0,1.1,398.3
|
||||
rollout,after,25.29,sampler,565f94da69f4,0.003,0.0,0.01,1.8
|
||||
rollout,after,25.29,vm,,0.7966,,3.31,
|
||||
rollout,after,26.3,api-new,2ec1a8170637,0.2805,0.0,1.16,122.1
|
||||
rollout,after,26.3,api-new,9128897910ef,0.2855,0.0,1.16,122.0
|
||||
rollout,after,26.3,postgres,2708a61ed609,0.0871,0.0,0.82,183.2
|
||||
rollout,after,26.3,haproxy,e1726165eec2,0.0236,0.0,0.35,36.8
|
||||
rollout,after,26.3,valkey,dbe0fec0557d,0.0075,0.0,0.34,13.2
|
||||
rollout,after,26.3,k6,d3da387bc8da,0.0781,0.0,1.2,398.3
|
||||
rollout,after,26.3,sampler,565f94da69f4,0.003,0.0,0.02,1.8
|
||||
rollout,after,26.3,vm,,0.8771,,3.14,
|
||||
rollout,after,27.3,api-new,2ec1a8170637,0.28,0.0,1.62,122.2
|
||||
rollout,after,27.3,api-new,9128897910ef,0.2818,0.0,0.9,122.3
|
||||
rollout,after,27.3,postgres,2708a61ed609,0.0863,0.0,0.68,183.0
|
||||
rollout,after,27.3,haproxy,e1726165eec2,0.0223,0.0,0.65,37.1
|
||||
rollout,after,27.3,valkey,dbe0fec0557d,0.0076,0.0,0.34,12.7
|
||||
rollout,after,27.3,k6,d3da387bc8da,0.0771,0.0,1.09,398.5
|
||||
rollout,after,27.3,sampler,565f94da69f4,0.0035,0.0,0.01,1.6
|
||||
rollout,after,27.3,vm,,0.8362,,2.65,
|
||||
rollout,after,28.3,api-new,2ec1a8170637,0.2714,0.0,3.04,122.8
|
||||
rollout,after,28.3,api-new,9128897910ef,0.2766,0.0,1.4,121.7
|
||||
rollout,after,28.3,postgres,2708a61ed609,0.0852,0.0,1.04,183.4
|
||||
rollout,after,28.3,haproxy,e1726165eec2,0.0342,0.0,0.81,36.4
|
||||
rollout,after,28.3,valkey,dbe0fec0557d,0.0196,0.0,0.61,13.3
|
||||
rollout,after,28.3,k6,d3da387bc8da,0.3909,0.0,2.7,398.5
|
||||
rollout,after,28.3,sampler,565f94da69f4,0.0036,0.0,0.02,1.7
|
||||
rollout,after,28.3,vm,,1.2352,,6.86,
|
||||
rollout,after,29.31,api-new,2ec1a8170637,0.3614,0.0,1.07,123.3
|
||||
rollout,after,29.31,api-new,9128897910ef,0.2717,0.0,0.88,122.2
|
||||
rollout,after,29.31,postgres,2708a61ed609,0.1239,0.0,0.62,191.4
|
||||
rollout,after,29.31,haproxy,e1726165eec2,0.0209,0.0,0.28,37.3
|
||||
rollout,after,29.31,valkey,dbe0fec0557d,0.0073,0.0,0.28,12.6
|
||||
rollout,after,29.31,k6,d3da387bc8da,0.0849,0.0,0.92,398.6
|
||||
rollout,after,29.31,sampler,565f94da69f4,0.0032,0.0,0.02,1.6
|
||||
rollout,after,29.31,vm,,0.986,,2.93,
|
||||
rollout,after,30.31,api-new,2ec1a8170637,0.2747,0.0,1.91,123.4
|
||||
rollout,after,30.31,api-new,9128897910ef,0.2739,0.0,1.64,122.8
|
||||
rollout,after,30.31,postgres,2708a61ed609,0.0853,0.0,0.67,191.6
|
||||
rollout,after,30.31,haproxy,e1726165eec2,0.0218,0.0,0.5,37.1
|
||||
rollout,after,30.31,valkey,dbe0fec0557d,0.0072,0.0,0.45,12.5
|
||||
rollout,after,30.31,k6,d3da387bc8da,0.0842,0.0,0.94,398.4
|
||||
rollout,after,30.31,sampler,565f94da69f4,0.003,0.0,0.01,1.7
|
||||
rollout,after,30.31,vm,,0.8365,,2.49,
|
||||
rollout,after,31.32,api-new,2ec1a8170637,0.2756,0.0,0.88,123.2
|
||||
rollout,after,31.32,api-new,9128897910ef,0.2772,0.0,0.85,123.2
|
||||
rollout,after,31.32,postgres,2708a61ed609,0.0873,0.0,0.64,191.6
|
||||
rollout,after,31.32,haproxy,e1726165eec2,0.0218,0.0,0.61,37.2
|
||||
rollout,after,31.32,valkey,dbe0fec0557d,0.0073,0.0,0.54,12.2
|
||||
rollout,after,31.32,k6,d3da387bc8da,0.0858,0.0,1.06,398.2
|
||||
rollout,after,31.32,sampler,565f94da69f4,0.0036,0.0,0.02,1.9
|
||||
rollout,after,31.32,vm,,0.9362,,4.24,
|
||||
rollout,after,32.32,api-new,2ec1a8170637,0.264,0.0,1.23,122.9
|
||||
rollout,after,32.32,api-new,9128897910ef,0.3366,0.0,1.31,123.7
|
||||
rollout,after,32.32,postgres,2708a61ed609,0.0817,0.0,0.78,191.9
|
||||
rollout,after,32.32,haproxy,e1726165eec2,0.0211,0.0,0.37,37.0
|
||||
rollout,after,32.32,valkey,dbe0fec0557d,0.0072,0.0,0.34,12.6
|
||||
rollout,after,32.32,k6,d3da387bc8da,0.0814,0.0,1.02,398.6
|
||||
rollout,after,32.32,sampler,565f94da69f4,0.0034,0.0,0.03,1.6
|
||||
rollout,after,32.32,vm,,1.1249,,5.28,
|
||||
rollout,after,33.32,api-new,2ec1a8170637,0.2753,0.0,1.44,123.2
|
||||
rollout,after,33.32,api-new,9128897910ef,0.2809,0.0,0.87,123.7
|
||||
rollout,after,33.32,postgres,2708a61ed609,0.0863,0.0,0.71,192.3
|
||||
rollout,after,33.32,haproxy,e1726165eec2,0.0337,0.0,0.28,37.2
|
||||
rollout,after,33.32,valkey,dbe0fec0557d,0.019,0.0,0.58,12.9
|
||||
rollout,after,33.32,k6,d3da387bc8da,0.083,0.0,1.24,398.4
|
||||
rollout,after,33.32,sampler,565f94da69f4,0.003,0.0,0.01,1.7
|
||||
rollout,after,33.32,vm,,0.9466,,3.38,
|
||||
rollout,after,34.33,api-new,2ec1a8170637,0.2812,0.0,1.19,122.8
|
||||
rollout,after,34.33,api-new,9128897910ef,0.2829,0.0,1.36,123.5
|
||||
rollout,after,34.33,postgres,2708a61ed609,0.1123,0.0,0.8,192.1
|
||||
rollout,after,34.33,haproxy,e1726165eec2,0.0214,0.0,0.51,37.3
|
||||
rollout,after,34.33,valkey,dbe0fec0557d,0.0076,0.0,0.46,12.4
|
||||
rollout,after,34.33,k6,d3da387bc8da,0.0799,0.0,1.07,398.8
|
||||
rollout,after,34.33,sampler,565f94da69f4,0.0032,0.0,0.02,1.5
|
||||
rollout,after,34.33,vm,,0.9366,,3.38,
|
||||
rollout,after,35.33,api-new,2ec1a8170637,0.2787,0.0,1.08,123.2
|
||||
rollout,after,35.33,api-new,9128897910ef,0.2843,0.0,1.04,122.7
|
||||
rollout,after,35.33,postgres,2708a61ed609,0.0871,0.0,0.61,192.1
|
||||
rollout,after,35.33,haproxy,e1726165eec2,0.0212,0.0,0.77,37.1
|
||||
rollout,after,35.33,valkey,dbe0fec0557d,0.0077,0.0,0.51,12.1
|
||||
rollout,after,35.33,k6,d3da387bc8da,0.078,0.0,0.98,398.7
|
||||
rollout,after,35.33,sampler,565f94da69f4,0.0036,0.0,0.01,1.6
|
||||
rollout,after,35.33,vm,,0.8956,,2.61,
|
||||
rollout,after,36.34,api-new,2ec1a8170637,0.2782,0.0,2.05,123.3
|
||||
rollout,after,36.34,api-new,9128897910ef,0.2812,0.0,1.59,122.6
|
||||
rollout,after,36.34,postgres,2708a61ed609,0.0867,0.0,0.78,192.6
|
||||
rollout,after,36.34,haproxy,e1726165eec2,0.0216,0.0,0.81,37.2
|
||||
rollout,after,36.34,valkey,dbe0fec0557d,0.0077,0.0,0.38,12.1
|
||||
rollout,after,36.34,k6,d3da387bc8da,0.0779,0.0,1.14,398.4
|
||||
rollout,after,36.34,sampler,565f94da69f4,0.0038,0.0,0.01,1.6
|
||||
rollout,after,36.34,vm,,0.906,,2.7,
|
||||
rollout,after,37.34,api-new,2ec1a8170637,0.2788,0.0,1.16,123.3
|
||||
rollout,after,37.34,api-new,9128897910ef,0.2808,0.0,0.77,123.1
|
||||
rollout,after,37.34,postgres,2708a61ed609,0.0861,0.0,0.98,192.4
|
||||
rollout,after,37.34,haproxy,e1726165eec2,0.0218,0.0,0.37,37.9
|
||||
rollout,after,37.34,valkey,dbe0fec0557d,0.0073,0.0,0.36,11.8
|
||||
rollout,after,37.34,k6,d3da387bc8da,0.0769,0.0,1.0,398.7
|
||||
rollout,after,37.34,sampler,565f94da69f4,0.0036,0.0,0.01,1.9
|
||||
rollout,after,37.34,vm,,0.8761,,2.6,
|
||||
rollout,after,38.35,api-new,2ec1a8170637,0.282,0.0,1.22,123.7
|
||||
rollout,after,38.35,api-new,9128897910ef,0.2861,0.0,1.53,123.1
|
||||
rollout,after,38.35,postgres,2708a61ed609,0.0873,0.0,0.94,192.9
|
||||
rollout,after,38.35,haproxy,e1726165eec2,0.0335,0.0,0.39,37.4
|
||||
rollout,after,38.35,valkey,dbe0fec0557d,0.0195,0.0,0.35,12.8
|
||||
rollout,after,38.35,k6,d3da387bc8da,0.0777,0.0,1.02,398.4
|
||||
rollout,after,38.35,sampler,565f94da69f4,0.0035,0.0,0.02,1.8
|
||||
rollout,after,38.35,vm,,0.9362,,3.12,
|
||||
rollout,after,39.35,api-new,2ec1a8170637,0.2757,0.0,2.13,124.1
|
||||
rollout,after,39.35,api-new,9128897910ef,0.2794,0.0,1.87,123.5
|
||||
rollout,after,39.35,postgres,2708a61ed609,0.1126,0.0,0.82,192.4
|
||||
rollout,after,39.35,haproxy,e1726165eec2,0.021,0.0,0.28,37.6
|
||||
rollout,after,39.35,valkey,dbe0fec0557d,0.0075,0.0,0.35,12.7
|
||||
rollout,after,39.35,k6,d3da387bc8da,0.077,0.0,1.24,399.0
|
||||
rollout,after,39.35,sampler,565f94da69f4,0.0032,0.0,0.1,2.0
|
||||
rollout,after,39.35,vm,,0.9165,,2.82,
|
||||
rollout,after,40.35,api-new,2ec1a8170637,0.2777,0.0,1.31,123.6
|
||||
rollout,after,40.35,api-new,9128897910ef,0.2852,0.0,1.28,123.4
|
||||
rollout,after,40.35,postgres,2708a61ed609,0.0876,0.0,1.0,192.7
|
||||
rollout,after,40.35,haproxy,e1726165eec2,0.0207,0.0,0.39,37.7
|
||||
rollout,after,40.35,valkey,dbe0fec0557d,0.0073,0.0,0.48,12.4
|
||||
rollout,after,40.35,k6,d3da387bc8da,0.0776,0.0,1.24,398.6
|
||||
rollout,after,40.35,sampler,565f94da69f4,0.0034,0.0,0.01,2.1
|
||||
rollout,after,40.35,vm,,0.8566,,2.55,
|
||||
rollout,after,41.36,api-new,2ec1a8170637,0.2786,0.0,1.45,123.8
|
||||
rollout,after,41.36,api-new,9128897910ef,0.2843,0.0,1.4,123.7
|
||||
rollout,after,41.36,postgres,2708a61ed609,0.0866,0.0,0.93,192.7
|
||||
rollout,after,41.36,haproxy,e1726165eec2,0.0208,0.0,0.53,37.3
|
||||
rollout,after,41.36,valkey,dbe0fec0557d,0.0074,0.0,0.22,12.9
|
||||
rollout,after,41.36,k6,d3da387bc8da,0.0773,0.0,1.21,398.5
|
||||
rollout,after,41.36,sampler,565f94da69f4,0.0034,0.0,0.02,2.0
|
||||
rollout,after,41.36,vm,,0.8369,,2.56,
|
||||
rollout,after,42.36,api-new,2ec1a8170637,0.2794,0.0,1.23,124.3
|
||||
rollout,after,42.36,api-new,9128897910ef,0.2878,0.0,1.76,123.7
|
||||
rollout,after,42.36,postgres,2708a61ed609,0.088,0.0,1.43,193.0
|
||||
rollout,after,42.36,haproxy,e1726165eec2,0.0215,0.0,0.45,37.8
|
||||
rollout,after,42.36,valkey,dbe0fec0557d,0.0085,0.0,0.36,12.6
|
||||
rollout,after,42.36,k6,d3da387bc8da,0.081,0.0,1.27,398.9
|
||||
rollout,after,42.36,sampler,565f94da69f4,0.0032,0.0,0.01,2.2
|
||||
rollout,after,42.36,vm,,1.085,,7.47,
|
||||
rollout,after,43.37,api-new,2ec1a8170637,0.2767,0.0,2.27,124.2
|
||||
rollout,after,43.37,api-new,9128897910ef,0.2842,0.0,2.88,123.9
|
||||
rollout,after,43.37,postgres,2708a61ed609,0.0916,0.0,0.92,194.0
|
||||
rollout,after,43.37,haproxy,e1726165eec2,0.0339,0.0,0.41,37.5
|
||||
rollout,after,43.37,valkey,dbe0fec0557d,0.0192,0.0,1.18,13.1
|
||||
rollout,after,43.37,k6,d3da387bc8da,0.3786,0.0,2.24,398.3
|
||||
rollout,after,43.37,sampler,565f94da69f4,0.0036,0.0,0.05,1.6
|
||||
rollout,after,43.37,vm,,1.2549,,6.3,
|
||||
rollout,after,44.37,api-new,2ec1a8170637,0.2697,0.0,1.39,124.1
|
||||
rollout,after,44.37,api-new,9128897910ef,0.2715,0.0,1.04,123.9
|
||||
rollout,after,44.37,postgres,2708a61ed609,0.1109,0.0,0.81,194.2
|
||||
rollout,after,44.37,haproxy,e1726165eec2,0.0203,0.0,0.71,37.5
|
||||
rollout,after,44.37,valkey,dbe0fec0557d,0.0072,0.0,0.25,13.3
|
||||
rollout,after,44.37,k6,d3da387bc8da,0.0845,0.0,1.17,398.2
|
||||
rollout,after,44.37,sampler,565f94da69f4,0.0033,0.0,0.01,1.6
|
||||
rollout,after,44.37,vm,,0.9163,,3.06,
|
||||
rollout,after,45.37,api-new,2ec1a8170637,0.3497,0.0,1.24,124.2
|
||||
rollout,after,45.37,api-new,9128897910ef,0.2751,0.0,1.22,124.4
|
||||
rollout,after,45.37,postgres,2708a61ed609,0.0817,0.0,0.64,194.2
|
||||
rollout,after,45.37,haproxy,e1726165eec2,0.0204,0.0,0.42,37.8
|
||||
rollout,after,45.37,valkey,dbe0fec0557d,0.0075,0.0,0.4,13.0
|
||||
rollout,after,45.37,k6,d3da387bc8da,0.0828,0.0,1.0,398.4
|
||||
rollout,after,45.37,sampler,565f94da69f4,0.0035,0.0,0.01,1.8
|
||||
rollout,after,45.37,vm,,0.9661,,2.4,
|
||||
rollout,after,46.38,api-new,2ec1a8170637,0.283,0.0,1.64,124.5
|
||||
rollout,after,46.38,api-new,9128897910ef,0.2851,0.0,1.46,124.2
|
||||
rollout,after,46.38,postgres,2708a61ed609,0.0856,0.0,0.66,194.6
|
||||
rollout,after,46.38,haproxy,e1726165eec2,0.0206,0.0,0.54,37.4
|
||||
rollout,after,46.38,valkey,dbe0fec0557d,0.0076,0.0,0.55,12.6
|
||||
rollout,after,46.38,k6,d3da387bc8da,0.0856,0.0,1.64,398.9
|
||||
rollout,after,46.38,sampler,565f94da69f4,0.0034,0.0,0.02,1.9
|
||||
rollout,after,46.38,vm,,0.926,,2.79,
|
||||
rollout,after,47.38,api-new,2ec1a8170637,0.282,0.0,1.24,124.4
|
||||
rollout,after,47.38,api-new,9128897910ef,0.3005,0.0,1.27,124.3
|
||||
rollout,after,47.38,postgres,2708a61ed609,0.0856,0.0,0.83,194.9
|
||||
rollout,after,47.38,haproxy,e1726165eec2,0.0204,0.0,0.6,38.0
|
||||
rollout,after,47.38,valkey,dbe0fec0557d,0.0077,0.0,0.3,12.8
|
||||
rollout,after,47.38,k6,d3da387bc8da,0.0852,0.0,1.34,398.8
|
||||
rollout,after,47.38,sampler,565f94da69f4,0.0039,0.0,0.01,1.8
|
||||
rollout,after,47.38,vm,,0.9056,,2.69,
|
||||
rollout,after,48.39,api-new,2ec1a8170637,0.2842,0.0,1.07,124.4
|
||||
rollout,after,48.39,api-new,9128897910ef,0.3424,0.0,1.64,124.5
|
||||
rollout,after,48.39,postgres,2708a61ed609,0.0861,0.0,0.97,194.5
|
||||
rollout,after,48.39,haproxy,e1726165eec2,0.033,0.0,0.48,38.2
|
||||
rollout,after,48.39,valkey,dbe0fec0557d,0.0197,0.0,0.6,13.2
|
||||
rollout,after,48.39,k6,d3da387bc8da,0.0853,0.0,1.58,398.3
|
||||
rollout,after,48.39,sampler,565f94da69f4,0.0032,0.0,0.03,1.8
|
||||
rollout,after,48.39,vm,,1.0362,,3.2,
|
||||
rollout,after,49.39,api-new,2ec1a8170637,0.2748,0.0,1.22,124.0
|
||||
rollout,after,49.39,api-new,9128897910ef,0.2794,0.0,1.45,123.9
|
||||
rollout,after,49.39,postgres,2708a61ed609,0.1111,0.0,0.62,194.4
|
||||
rollout,after,49.39,haproxy,e1726165eec2,0.02,0.0,0.55,37.6
|
||||
rollout,after,49.39,valkey,dbe0fec0557d,0.0074,0.0,0.24,13.3
|
||||
rollout,after,49.39,k6,d3da387bc8da,0.0775,0.0,1.8,399.2
|
||||
rollout,after,49.39,sampler,565f94da69f4,0.003,0.0,0.01,2.0
|
||||
rollout,after,49.39,vm,,0.9165,,2.94,
|
||||
rollout,after,50.4,api-new,2ec1a8170637,0.2806,0.0,1.98,124.7
|
||||
rollout,after,50.4,api-new,9128897910ef,0.2862,0.0,1.13,124.6
|
||||
rollout,after,50.4,postgres,2708a61ed609,0.0864,0.0,0.73,194.6
|
||||
rollout,after,50.4,haproxy,e1726165eec2,0.0202,0.0,0.27,37.6
|
||||
rollout,after,50.4,valkey,dbe0fec0557d,0.0074,0.0,0.4,13.0
|
||||
rollout,after,50.4,k6,d3da387bc8da,0.0774,0.0,1.46,399.1
|
||||
rollout,after,50.4,sampler,565f94da69f4,0.0032,0.0,0.01,1.9
|
||||
rollout,after,50.4,vm,,0.9164,,2.83,
|
||||
rollout,after,51.4,api-new,2ec1a8170637,0.2853,0.0,1.4,124.9
|
||||
rollout,after,51.4,api-new,9128897910ef,0.2859,0.0,1.49,124.2
|
||||
rollout,after,51.4,postgres,2708a61ed609,0.0872,0.0,0.71,194.8
|
||||
rollout,after,51.4,haproxy,e1726165eec2,0.0204,0.0,0.36,37.9
|
||||
rollout,after,51.4,valkey,dbe0fec0557d,0.0074,0.0,0.33,12.9
|
||||
rollout,after,51.4,k6,d3da387bc8da,0.0769,0.0,1.58,398.5
|
||||
rollout,after,51.4,sampler,565f94da69f4,0.0035,0.0,0.04,2.2
|
||||
rollout,after,51.4,vm,,0.8965,,2.84,
|
||||
rollout,after,52.4,api-new,2ec1a8170637,0.2816,0.0,1.37,124.6
|
||||
rollout,after,52.4,api-new,9128897910ef,0.2849,0.0,1.92,124.9
|
||||
rollout,after,52.4,postgres,2708a61ed609,0.0885,0.0,1.04,195.2
|
||||
rollout,after,52.4,haproxy,e1726165eec2,0.021,0.0,0.38,38.4
|
||||
rollout,after,52.4,valkey,dbe0fec0557d,0.0076,0.0,0.36,12.4
|
||||
rollout,after,52.4,k6,d3da387bc8da,0.0775,0.0,2.04,399.3
|
||||
rollout,after,52.4,sampler,565f94da69f4,0.0036,0.0,0.01,1.9
|
||||
rollout,after,52.4,vm,,1.1341,,6.41,
|
||||
rollout,after,53.41,api-new,2ec1a8170637,0.2812,0.0,2.23,124.4
|
||||
rollout,after,53.41,api-new,9128897910ef,0.2834,0.0,1.17,124.8
|
||||
rollout,after,53.41,postgres,2708a61ed609,0.0867,0.0,0.79,195.3
|
||||
rollout,after,53.41,haproxy,e1726165eec2,0.0326,0.0,0.42,37.7
|
||||
rollout,after,53.41,valkey,dbe0fec0557d,0.0189,0.0,0.29,12.8
|
||||
rollout,after,53.41,k6,d3da387bc8da,0.0776,0.0,1.45,399.2
|
||||
rollout,after,53.41,sampler,565f94da69f4,0.0034,0.0,0.02,1.6
|
||||
rollout,after,53.41,vm,,0.9562,,3.43,
|
||||
rollout,after,54.41,api-new,2ec1a8170637,0.275,0.0,1.87,124.8
|
||||
rollout,after,54.41,api-new,9128897910ef,0.2784,0.0,1.24,124.1
|
||||
rollout,after,54.41,postgres,2708a61ed609,0.1123,0.0,0.74,195.2
|
||||
rollout,after,54.41,haproxy,e1726165eec2,0.0205,0.0,0.61,38.0
|
||||
rollout,after,54.41,valkey,dbe0fec0557d,0.0075,0.0,0.4,12.5
|
||||
rollout,after,54.41,k6,d3da387bc8da,0.0768,0.0,1.01,399.1
|
||||
rollout,after,54.41,sampler,565f94da69f4,0.003,0.0,0.01,1.7
|
||||
rollout,after,54.41,vm,,0.976,,3.58,
|
||||
rollout,after,55.42,api-new,2ec1a8170637,0.2889,0.0,2.04,124.5
|
||||
rollout,after,55.42,api-new,9128897910ef,0.2922,0.0,1.41,124.0
|
||||
rollout,after,55.42,postgres,2708a61ed609,0.0885,0.0,0.81,195.3
|
||||
rollout,after,55.42,haproxy,e1726165eec2,0.0208,0.0,0.38,37.8
|
||||
rollout,after,55.42,valkey,dbe0fec0557d,0.0078,0.0,0.42,12.3
|
||||
rollout,after,55.42,k6,d3da387bc8da,0.0778,0.0,1.64,399.1
|
||||
rollout,after,55.42,sampler,565f94da69f4,0.0037,0.0,0.02,1.7
|
||||
rollout,after,55.42,vm,,0.9164,,2.94,
|
||||
rollout,after,56.42,api-new,2ec1a8170637,0.2838,0.0,1.06,124.3
|
||||
rollout,after,56.42,api-new,9128897910ef,0.2862,0.0,1.24,124.4
|
||||
rollout,after,56.42,postgres,2708a61ed609,0.0866,0.0,0.7,195.5
|
||||
rollout,after,56.42,haproxy,e1726165eec2,0.0203,0.0,0.32,38.1
|
||||
rollout,after,56.42,valkey,dbe0fec0557d,0.0074,0.0,0.28,12.3
|
||||
rollout,after,56.42,k6,d3da387bc8da,0.0767,0.0,1.02,398.5
|
||||
rollout,after,56.42,sampler,565f94da69f4,0.0034,0.0,0.01,2.0
|
||||
rollout,after,56.42,vm,,0.8861,,2.67,
|
||||
rollout,after,57.42,api-new,2ec1a8170637,0.2853,0.0,1.72,124.6
|
||||
rollout,after,57.42,api-new,9128897910ef,0.2892,0.0,2.14,125.0
|
||||
rollout,after,57.42,postgres,2708a61ed609,0.0875,0.0,0.67,195.4
|
||||
rollout,after,57.42,haproxy,e1726165eec2,0.0213,0.0,0.47,37.9
|
||||
rollout,after,57.42,valkey,dbe0fec0557d,0.0078,0.0,0.31,12.2
|
||||
rollout,after,57.42,k6,d3da387bc8da,0.078,0.0,1.43,399.0
|
||||
rollout,after,57.42,sampler,565f94da69f4,0.003,0.0,0.03,2.0
|
||||
rollout,after,57.42,vm,,0.897,,2.67,
|
||||
rollout,after,58.43,api-new,2ec1a8170637,0.2859,0.0,1.35,124.6
|
||||
rollout,after,58.43,api-new,9128897910ef,0.2904,0.0,2.07,124.5
|
||||
rollout,after,58.43,postgres,2708a61ed609,0.0882,0.0,0.78,195.5
|
||||
rollout,after,58.43,haproxy,e1726165eec2,0.0328,0.0,0.65,38.2
|
||||
rollout,after,58.43,valkey,dbe0fec0557d,0.0205,0.0,0.39,13.0
|
||||
rollout,after,58.43,k6,d3da387bc8da,0.0778,0.0,2.21,398.9
|
||||
rollout,after,58.43,sampler,565f94da69f4,0.0031,0.0,0.02,1.8
|
||||
rollout,after,58.43,vm,,0.9665,,3.56,
|
||||
rollout,after,59.43,api-new,2ec1a8170637,0.2716,0.0,2.23,124.9
|
||||
rollout,after,59.43,api-new,9128897910ef,0.278,0.0,1.34,124.7
|
||||
rollout,after,59.43,postgres,2708a61ed609,0.1119,0.0,0.66,195.5
|
||||
rollout,after,59.43,haproxy,e1726165eec2,0.0202,0.0,0.33,38.4
|
||||
rollout,after,59.43,valkey,dbe0fec0557d,0.0072,0.0,0.24,12.4
|
||||
rollout,after,59.43,k6,d3da387bc8da,0.0754,0.0,1.58,407.1
|
||||
rollout,after,59.43,sampler,565f94da69f4,0.0033,0.0,0.01,1.8
|
||||
rollout,after,59.43,vm,,0.8863,,2.8,
|
||||
rollout,after,60.44,api-new,2ec1a8170637,0.2876,0.0,2.16,124.9
|
||||
rollout,after,60.44,api-new,9128897910ef,0.2938,0.0,1.5,124.7
|
||||
rollout,after,60.44,postgres,2708a61ed609,0.0889,0.0,0.91,195.8
|
||||
rollout,after,60.44,haproxy,e1726165eec2,0.0205,0.0,1.02,37.8
|
||||
rollout,after,60.44,valkey,dbe0fec0557d,0.0076,0.0,0.5,12.2
|
||||
rollout,after,60.44,k6,d3da387bc8da,0.0775,0.0,1.6,415.2
|
||||
rollout,after,60.44,sampler,565f94da69f4,0.004,0.0,0.09,1.9
|
||||
rollout,after,60.44,vm,,0.896,,2.73,
|
||||
rollout,after,61.44,api-new,2ec1a8170637,0.3481,0.0,1.31,124.7
|
||||
rollout,after,61.44,api-new,9128897910ef,0.2754,0.0,1.53,124.6
|
||||
rollout,after,61.44,postgres,2708a61ed609,0.0831,0.0,0.8,195.5
|
||||
rollout,after,61.44,haproxy,e1726165eec2,0.0199,0.0,0.56,37.3
|
||||
rollout,after,61.44,valkey,dbe0fec0557d,0.0075,0.0,0.52,12.1
|
||||
rollout,after,61.44,k6,d3da387bc8da,0.0744,0.0,1.32,425.3
|
||||
rollout,after,61.44,sampler,565f94da69f4,0.0038,0.0,0.01,1.9
|
||||
rollout,after,61.44,vm,,0.9353,,2.42,
|
||||
rollout,after,62.45,api-new,2ec1a8170637,0.2878,0.0,1.13,124.6
|
||||
rollout,after,62.45,api-new,9128897910ef,0.2923,0.0,1.58,125.0
|
||||
rollout,after,62.45,postgres,2708a61ed609,0.0882,0.0,1.1,196.4
|
||||
rollout,after,62.45,haproxy,e1726165eec2,0.0211,0.0,0.25,37.9
|
||||
rollout,after,62.45,valkey,dbe0fec0557d,0.0077,0.0,0.62,12.1
|
||||
rollout,after,62.45,k6,d3da387bc8da,0.0783,0.0,1.76,433.9
|
||||
rollout,after,62.45,sampler,565f94da69f4,0.0037,0.0,0.01,2.1
|
||||
rollout,after,62.45,vm,,1.1553,,7.23,
|
||||
rollout,after,63.45,api-new,2ec1a8170637,0.2767,0.0,1.0,124.4
|
||||
rollout,after,63.45,api-new,9128897910ef,0.3584,0.0,0.99,125.1
|
||||
rollout,after,63.45,postgres,2708a61ed609,0.1443,0.0,0.64,196.8
|
||||
rollout,after,63.45,haproxy,e1726165eec2,0.0307,0.0,0.6,37.7
|
||||
rollout,after,63.45,valkey,dbe0fec0557d,0.0193,0.0,0.33,13.1
|
||||
rollout,after,63.45,k6,d3da387bc8da,0.0749,0.0,1.69,441.8
|
||||
rollout,after,63.45,sampler,565f94da69f4,0.0033,0.0,0.1,2.0
|
||||
rollout,after,63.45,vm,,1.1152,,2.91,
|
||||
rollout,after,64.45,api-new,2ec1a8170637,0.2632,0.0,4.09,124.3
|
||||
rollout,after,64.45,api-new,9128897910ef,0.2692,0.0,3.25,125.1
|
||||
rollout,after,64.45,postgres,2708a61ed609,0.1106,0.0,0.59,196.2
|
||||
rollout,after,64.45,haproxy,e1726165eec2,0.019,0.0,0.46,38.2
|
||||
rollout,after,64.45,valkey,dbe0fec0557d,0.0071,0.0,0.29,12.8
|
||||
rollout,after,64.45,k6,d3da387bc8da,0.3188,0.0,2.38,454.5
|
||||
rollout,after,64.45,sampler,565f94da69f4,0.003,0.0,0.01,1.7
|
||||
rollout,after,64.45,vm,,1.1557,,6.56,
|
||||
rollout,after,65.46,api-new,2ec1a8170637,0.2813,0.0,2.18,124.8
|
||||
rollout,after,65.46,api-new,9128897910ef,0.2855,0.0,0.73,124.9
|
||||
rollout,after,65.46,postgres,2708a61ed609,0.0926,0.0,0.76,196.6
|
||||
rollout,after,65.46,haproxy,e1726165eec2,0.0209,0.0,0.18,37.7
|
||||
rollout,after,65.46,valkey,dbe0fec0557d,0.0076,0.0,0.32,12.8
|
||||
rollout,after,65.46,k6,d3da387bc8da,0.0868,0.0,1.33,454.7
|
||||
rollout,after,65.46,sampler,565f94da69f4,0.0033,0.0,0.03,2.0
|
||||
rollout,after,65.46,vm,,0.8866,,2.53,
|
||||
rollout,after,66.46,api-new,2ec1a8170637,0.2779,0.0,1.86,124.8
|
||||
rollout,after,66.46,api-new,9128897910ef,0.2798,0.0,1.6,125.1
|
||||
rollout,after,66.46,postgres,2708a61ed609,0.0902,0.0,0.79,197.3
|
||||
rollout,after,66.46,haproxy,e1726165eec2,0.0201,0.0,0.42,38.0
|
||||
rollout,after,66.46,valkey,dbe0fec0557d,0.0072,0.0,0.38,12.8
|
||||
rollout,after,66.46,k6,d3da387bc8da,0.0855,0.0,1.35,454.5
|
||||
rollout,after,66.46,sampler,565f94da69f4,0.0029,0.0,0.01,1.8
|
||||
rollout,after,66.46,vm,,0.8867,,2.79,
|
||||
rollout,after,67.46,api-new,2ec1a8170637,0.2841,0.0,1.3,124.8
|
||||
rollout,after,67.46,api-new,9128897910ef,0.2902,0.0,2.24,124.9
|
||||
rollout,after,67.46,postgres,2708a61ed609,0.0932,0.0,0.74,197.2
|
||||
rollout,after,67.46,haproxy,e1726165eec2,0.0211,0.0,0.62,37.8
|
||||
rollout,after,67.46,valkey,dbe0fec0557d,0.0076,0.0,0.28,12.8
|
||||
rollout,after,67.46,k6,d3da387bc8da,0.0867,0.0,1.77,454.2
|
||||
rollout,after,67.46,sampler,565f94da69f4,0.0033,0.0,0.01,1.8
|
||||
rollout,after,67.46,vm,,0.8969,,2.88,
|
||||
rollout,after,68.47,api-new,2ec1a8170637,0.2878,0.0,2.08,124.3
|
||||
rollout,after,68.47,api-new,9128897910ef,0.2904,0.0,1.79,125.2
|
||||
rollout,after,68.47,postgres,2708a61ed609,0.0944,0.0,0.94,197.3
|
||||
rollout,after,68.47,haproxy,e1726165eec2,0.0313,0.0,0.77,45.3
|
||||
rollout,after,68.47,valkey,dbe0fec0557d,0.0207,0.0,0.42,12.8
|
||||
rollout,after,68.47,k6,d3da387bc8da,0.0855,0.0,1.85,454.3
|
||||
rollout,after,68.47,sampler,565f94da69f4,0.0033,0.0,0.03,1.5
|
||||
rollout,after,68.47,vm,,0.9961,,3.39,
|
||||
rollout,after,69.47,api-new,2ec1a8170637,0.2714,0.0,1.41,124.9
|
||||
rollout,after,69.47,api-new,9128897910ef,0.2735,0.0,1.47,125.5
|
||||
rollout,after,69.47,postgres,2708a61ed609,0.1145,0.0,0.67,197.0
|
||||
rollout,after,69.47,haproxy,e1726165eec2,0.0207,0.0,0.59,38.2
|
||||
rollout,after,69.47,valkey,dbe0fec0557d,0.0071,0.0,0.53,12.8
|
||||
rollout,after,69.47,k6,d3da387bc8da,0.0812,0.0,1.49,454.5
|
||||
rollout,after,69.47,sampler,565f94da69f4,0.0032,0.0,0.01,1.7
|
||||
rollout,after,69.47,vm,,0.9262,,3.21,
|
||||
rollout,after,70.48,api-new,2ec1a8170637,0.3366,0.0,1.7,125.0
|
||||
rollout,after,70.48,api-new,9128897910ef,0.3351,0.0,1.47,125.2
|
||||
rollout,after,70.48,postgres,2708a61ed609,0.1058,0.0,0.95,197.2
|
||||
rollout,after,70.48,haproxy,e1726165eec2,0.027,0.0,0.84,37.8
|
||||
rollout,after,70.48,valkey,dbe0fec0557d,0.0093,0.0,0.25,12.8
|
||||
rollout,after,70.48,k6,d3da387bc8da,0.0969,0.0,1.39,454.0
|
||||
rollout,after,70.48,sampler,565f94da69f4,0.0033,0.0,0.09,1.7
|
||||
rollout,after,70.48,vm,,1.0357,,2.73,
|
||||
rollout,after,71.48,api-new,2ec1a8170637,0.3216,0.0,0.9,125.0
|
||||
rollout,after,71.48,api-new,9128897910ef,0.3214,0.0,1.08,125.0
|
||||
rollout,after,71.48,postgres,2708a61ed609,0.1001,0.0,0.82,197.5
|
||||
rollout,after,71.48,haproxy,e1726165eec2,0.0256,0.0,0.4,38.1
|
||||
rollout,after,71.48,valkey,dbe0fec0557d,0.0091,0.0,0.2,12.7
|
||||
rollout,after,71.48,k6,d3da387bc8da,0.0916,0.0,1.22,454.2
|
||||
rollout,after,71.48,sampler,565f94da69f4,0.003,0.0,0.04,1.6
|
||||
rollout,after,71.48,vm,,1.0161,,2.37,
|
||||
rollout,after,72.48,api-new,2ec1a8170637,0.2923,0.0,1.69,125.0
|
||||
rollout,after,72.48,api-new,9128897910ef,0.29,0.0,1.12,125.2
|
||||
rollout,after,72.48,postgres,2708a61ed609,0.094,0.0,1.02,197.8
|
||||
rollout,after,72.48,haproxy,e1726165eec2,0.0215,0.0,0.76,37.8
|
||||
rollout,after,72.48,valkey,dbe0fec0557d,0.0077,0.0,0.51,12.6
|
||||
rollout,after,72.48,k6,d3da387bc8da,0.081,0.0,1.96,454.4
|
||||
rollout,after,72.48,sampler,565f94da69f4,0.0034,0.0,0.07,2.0
|
||||
rollout,after,72.48,vm,,1.1956,,8.05,
|
||||
rollout,after,73.49,api-new,2ec1a8170637,0.283,0.0,1.75,125.2
|
||||
rollout,after,73.49,api-new,9128897910ef,0.2872,0.0,1.73,124.8
|
||||
rollout,after,73.49,postgres,2708a61ed609,0.0918,0.0,0.72,197.8
|
||||
rollout,after,73.49,haproxy,e1726165eec2,0.0207,0.0,0.91,38.2
|
||||
rollout,after,73.49,valkey,dbe0fec0557d,0.0193,0.0,0.4,12.6
|
||||
rollout,after,73.49,k6,d3da387bc8da,0.0782,0.0,1.77,454.5
|
||||
rollout,after,73.49,sampler,565f94da69f4,0.0032,0.0,0.01,1.6
|
||||
rollout,after,73.49,vm,,1.0058,,3.09,
|
||||
rollout,after,74.49,api-new,2ec1a8170637,0.277,0.0,1.33,124.7
|
||||
rollout,after,74.49,api-new,9128897910ef,0.2788,0.0,1.47,125.3
|
||||
rollout,after,74.49,postgres,2708a61ed609,0.116,0.0,0.68,197.2
|
||||
rollout,after,74.49,haproxy,e1726165eec2,0.0315,0.0,0.37,38.0
|
||||
rollout,after,74.49,valkey,dbe0fec0557d,0.0074,0.0,0.48,12.5
|
||||
rollout,after,74.49,k6,d3da387bc8da,0.0769,0.0,1.7,454.3
|
||||
rollout,after,74.49,sampler,565f94da69f4,0.003,0.0,0.01,1.7
|
||||
rollout,after,74.49,vm,,0.9962,,3.44,
|
||||
rollout,after,75.5,api-new,2ec1a8170637,0.2921,0.0,2.35,124.5
|
||||
rollout,after,75.5,api-new,9128897910ef,0.2977,0.0,1.42,124.9
|
||||
rollout,after,75.5,postgres,2708a61ed609,0.095,0.0,0.82,197.1
|
||||
rollout,after,75.5,haproxy,e1726165eec2,0.0215,0.0,0.44,37.9
|
||||
rollout,after,75.5,valkey,dbe0fec0557d,0.0079,0.0,0.56,12.7
|
||||
rollout,after,75.5,k6,d3da387bc8da,0.0789,0.0,1.64,454.4
|
||||
rollout,after,75.5,sampler,565f94da69f4,0.0036,0.0,0.02,1.8
|
||||
rollout,after,75.5,vm,,0.9261,,2.86,
|
||||
rollout,after,76.5,api-new,2ec1a8170637,0.2854,0.0,2.29,124.7
|
||||
rollout,after,76.5,api-new,9128897910ef,0.2883,0.0,2.4,124.9
|
||||
rollout,after,76.5,postgres,2708a61ed609,0.0927,0.0,1.05,197.4
|
||||
rollout,after,76.5,haproxy,e1726165eec2,0.0205,0.0,0.63,38.1
|
||||
rollout,after,76.5,valkey,dbe0fec0557d,0.0074,0.0,0.42,12.7
|
||||
rollout,after,76.5,k6,d3da387bc8da,0.0773,0.0,1.48,454.2
|
||||
rollout,after,76.5,sampler,565f94da69f4,0.0035,0.0,0.01,1.7
|
||||
rollout,after,76.5,vm,,0.9358,,2.85,
|
||||
rollout,after,77.51,api-new,2ec1a8170637,0.354,0.0,1.0,125.4
|
||||
rollout,after,77.51,api-new,9128897910ef,0.2845,0.0,1.15,124.9
|
||||
rollout,after,77.51,postgres,2708a61ed609,0.0909,0.0,0.76,198.2
|
||||
rollout,after,77.51,haproxy,e1726165eec2,0.0203,0.0,0.36,37.9
|
||||
rollout,after,77.51,valkey,dbe0fec0557d,0.0075,0.0,0.72,12.4
|
||||
rollout,after,77.51,k6,d3da387bc8da,0.0765,0.0,1.13,454.5
|
||||
rollout,after,77.51,sampler,565f94da69f4,0.0036,0.0,0.02,1.7
|
||||
rollout,after,77.51,vm,,1.0059,,2.95,
|
||||
rollout,after,78.51,api-new,2ec1a8170637,0.2848,0.0,1.52,124.5
|
||||
rollout,after,78.51,api-new,9128897910ef,0.2876,0.0,1.78,124.4
|
||||
rollout,after,78.51,postgres,2708a61ed609,0.0922,0.0,0.85,198.4
|
||||
rollout,after,78.51,haproxy,e1726165eec2,0.0202,0.0,0.7,37.6
|
||||
rollout,after,78.51,valkey,dbe0fec0557d,0.0194,0.0,0.37,13.1
|
||||
rollout,after,78.51,k6,d3da387bc8da,0.0775,0.0,1.4,454.6
|
||||
rollout,after,78.51,sampler,565f94da69f4,0.0036,0.0,0.04,1.8
|
||||
rollout,after,78.51,vm,,0.906,,3.14,
|
||||
rollout,after,79.51,api-new,2ec1a8170637,0.2761,0.0,1.04,124.9
|
||||
rollout,after,79.51,api-new,9128897910ef,0.3482,0.0,0.98,125.2
|
||||
rollout,after,79.51,postgres,2708a61ed609,0.114,0.0,0.87,198.1
|
||||
rollout,after,79.51,haproxy,e1726165eec2,0.0319,0.0,0.37,37.7
|
||||
rollout,after,79.51,valkey,dbe0fec0557d,0.0075,0.0,0.26,12.7
|
||||
rollout,after,79.51,k6,d3da387bc8da,0.0762,0.0,2.44,454.9
|
||||
rollout,after,79.51,sampler,565f94da69f4,0.003,0.0,0.02,1.7
|
||||
rollout,after,79.51,vm,,1.0163,,3.12,
|
||||
rollout,after,80.52,api-new,2ec1a8170637,0.277,0.0,1.47,125.2
|
||||
rollout,after,80.52,api-new,9128897910ef,0.2853,0.0,1.59,124.8
|
||||
rollout,after,80.52,postgres,2708a61ed609,0.0915,0.0,0.96,198.6
|
||||
rollout,after,80.52,haproxy,e1726165eec2,0.02,0.0,0.57,38.1
|
||||
rollout,after,80.52,valkey,dbe0fec0557d,0.007,0.0,0.54,12.8
|
||||
rollout,after,80.52,k6,d3da387bc8da,0.0767,0.0,1.04,454.9
|
||||
rollout,after,80.52,sampler,565f94da69f4,0.0032,0.0,0.02,1.7
|
||||
rollout,after,80.52,vm,,0.837,,2.51,
|
||||
rollout,after,81.52,api-new,2ec1a8170637,0.2831,0.0,1.23,124.3
|
||||
rollout,after,81.52,api-new,9128897910ef,0.291,0.0,1.05,124.7
|
||||
rollout,after,81.52,postgres,2708a61ed609,0.0928,0.0,0.68,198.7
|
||||
rollout,after,81.52,haproxy,e1726165eec2,0.0208,0.0,0.56,37.9
|
||||
rollout,after,81.52,valkey,dbe0fec0557d,0.0074,0.0,0.19,12.8
|
||||
rollout,after,81.52,k6,d3da387bc8da,0.0781,0.0,0.95,454.3
|
||||
rollout,after,81.52,sampler,565f94da69f4,0.0036,0.0,0.01,1.8
|
||||
rollout,after,81.52,vm,,0.8559,,2.79,
|
||||
rollout,after,82.53,api-new,2ec1a8170637,0.2927,0.0,1.38,125.1
|
||||
rollout,after,82.53,api-new,9128897910ef,0.2925,0.0,1.85,124.9
|
||||
rollout,after,82.53,postgres,2708a61ed609,0.0949,0.0,0.83,199.2
|
||||
rollout,after,82.53,haproxy,e1726165eec2,0.0215,0.0,0.57,37.8
|
||||
rollout,after,82.53,valkey,dbe0fec0557d,0.0081,0.0,0.45,13.1
|
||||
rollout,after,82.53,k6,d3da387bc8da,0.0811,0.0,1.04,454.6
|
||||
rollout,after,82.53,sampler,565f94da69f4,0.0036,0.0,0.02,1.4
|
||||
rollout,after,82.53,vm,,1.1355,,7.43,
|
||||
rollout,after,83.53,api-new,2ec1a8170637,0.2815,0.0,1.77,125.2
|
||||
rollout,after,83.53,api-new,9128897910ef,0.2852,0.0,1.19,124.5
|
||||
rollout,after,83.53,postgres,2708a61ed609,0.0982,0.0,0.81,199.0
|
||||
rollout,after,83.53,haproxy,e1726165eec2,0.0201,0.0,0.5,38.0
|
||||
rollout,after,83.53,valkey,dbe0fec0557d,0.0194,0.0,0.43,13.5
|
||||
rollout,after,83.53,k6,d3da387bc8da,0.0765,0.0,1.14,454.6
|
||||
rollout,after,83.53,sampler,565f94da69f4,0.0034,0.0,0.03,1.6
|
||||
rollout,after,83.53,vm,,0.8962,,3.39,
|
||||
rollout,after,84.53,api-new,2ec1a8170637,0.272,0.0,1.67,125.5
|
||||
rollout,after,84.53,api-new,9128897910ef,0.2728,0.0,1.06,125.0
|
||||
rollout,after,84.53,postgres,2708a61ed609,0.1155,0.0,0.81,198.8
|
||||
rollout,after,84.53,haproxy,e1726165eec2,0.0317,0.0,0.58,37.6
|
||||
rollout,after,84.53,valkey,dbe0fec0557d,0.0071,0.0,0.47,13.3
|
||||
rollout,after,84.53,k6,d3da387bc8da,0.0764,0.0,1.36,454.7
|
||||
rollout,after,84.53,sampler,565f94da69f4,0.0032,0.0,0.01,1.8
|
||||
rollout,after,84.53,vm,,0.9565,,3.68,
|
||||
rollout,after,85.54,api-new,2ec1a8170637,0.2936,0.0,1.31,125.1
|
||||
rollout,after,85.54,api-new,9128897910ef,0.2963,0.0,0.51,125.1
|
||||
rollout,after,85.54,postgres,2708a61ed609,0.092,0.0,1.21,199.0
|
||||
rollout,after,85.54,haproxy,e1726165eec2,0.0214,0.0,0.58,38.3
|
||||
rollout,after,85.54,valkey,dbe0fec0557d,0.0079,0.0,0.42,13.3
|
||||
rollout,after,85.54,k6,d3da387bc8da,0.3692,0.0,6.91,454.8
|
||||
rollout,after,85.54,sampler,565f94da69f4,0.0036,0.0,0.02,1.6
|
||||
rollout,after,85.54,vm,,1.1851,,8.52,
|
||||
rollout,after,86.54,api-new,2ec1a8170637,0.2753,0.0,1.42,124.8
|
||||
rollout,after,86.54,api-new,9128897910ef,0.2822,0.0,1.74,124.7
|
||||
rollout,after,86.54,postgres,2708a61ed609,0.0905,0.0,0.78,199.2
|
||||
rollout,after,86.54,haproxy,e1726165eec2,0.0205,0.0,0.61,38.0
|
||||
rollout,after,86.54,valkey,dbe0fec0557d,0.0075,0.0,0.51,13.3
|
||||
rollout,after,86.54,k6,d3da387bc8da,0.0851,0.0,0.99,454.3
|
||||
rollout,after,86.54,sampler,565f94da69f4,0.0032,0.0,0.01,1.7
|
||||
rollout,after,86.54,vm,,0.8861,,2.47,
|
||||
rollout,after,87.55,api-new,2ec1a8170637,0.2739,0.0,1.67,125.2
|
||||
rollout,after,87.55,api-new,9128897910ef,0.2791,0.0,0.92,124.5
|
||||
rollout,after,87.55,postgres,2708a61ed609,0.0899,0.0,0.78,199.4
|
||||
rollout,after,87.55,haproxy,e1726165eec2,0.0204,0.0,0.39,37.9
|
||||
rollout,after,87.55,valkey,dbe0fec0557d,0.0074,0.0,0.49,12.9
|
||||
rollout,after,87.55,k6,d3da387bc8da,0.0834,0.0,0.94,454.9
|
||||
rollout,after,87.55,sampler,565f94da69f4,0.0035,0.0,0.1,1.7
|
||||
rollout,after,87.55,vm,,0.8467,,2.91,
|
||||
rollout,after,88.55,api-new,2ec1a8170637,0.2885,0.0,1.48,124.6
|
||||
rollout,after,88.55,api-new,9128897910ef,0.2908,0.0,1.29,124.8
|
||||
rollout,after,88.55,postgres,2708a61ed609,0.0942,0.0,1.1,199.1
|
||||
rollout,after,88.55,haproxy,e1726165eec2,0.0211,0.0,0.62,37.9
|
||||
rollout,after,88.55,valkey,dbe0fec0557d,0.0206,0.0,0.39,12.6
|
||||
rollout,after,88.55,k6,d3da387bc8da,0.0885,0.0,0.93,455.0
|
||||
rollout,after,88.55,sampler,565f94da69f4,0.0034,0.0,0.02,1.7
|
||||
rollout,after,88.55,vm,,0.9262,,2.92,
|
||||
rollout,after,89.55,api-new,2ec1a8170637,0.2781,0.0,1.73,124.7
|
||||
rollout,after,89.55,api-new,9128897910ef,0.2851,0.0,0.75,125.1
|
||||
rollout,after,89.55,postgres,2708a61ed609,0.117,0.0,0.92,199.5
|
||||
rollout,after,89.55,haproxy,e1726165eec2,0.0327,0.0,0.32,37.6
|
||||
rollout,after,89.55,valkey,dbe0fec0557d,0.0077,0.0,0.29,12.5
|
||||
rollout,after,89.55,k6,d3da387bc8da,0.0849,0.0,0.95,455.2
|
||||
rollout,after,89.55,sampler,565f94da69f4,0.0034,0.0,0.01,1.7
|
||||
rollout,after,89.55,vm,,0.9565,,3.05,
|
||||
rollout,after,90.56,api-new,2ec1a8170637,0.2869,0.0,1.4,125.2
|
||||
rollout,after,90.56,api-new,9128897910ef,0.2909,0.0,1.96,125.1
|
||||
rollout,after,90.56,postgres,2708a61ed609,0.095,0.0,1.06,200.4
|
||||
rollout,after,90.56,haproxy,e1726165eec2,0.0211,0.0,0.68,37.9
|
||||
rollout,after,90.56,valkey,dbe0fec0557d,0.0076,0.0,0.56,12.4
|
||||
rollout,after,90.56,k6,d3da387bc8da,0.0848,0.0,1.18,455.1
|
||||
rollout,after,90.56,sampler,565f94da69f4,0.0033,0.0,0.02,1.8
|
||||
rollout,after,90.56,vm,,0.9064,,2.79,
|
||||
rollout,after,91.56,api-new,2ec1a8170637,0.2866,0.0,1.46,125.2
|
||||
rollout,after,91.56,api-new,9128897910ef,0.2909,0.0,1.14,124.6
|
||||
rollout,after,91.56,postgres,2708a61ed609,0.0923,0.0,0.89,200.1
|
||||
rollout,after,91.56,haproxy,e1726165eec2,0.021,0.0,0.4,38.0
|
||||
rollout,after,91.56,valkey,dbe0fec0557d,0.0077,0.0,0.53,12.4
|
||||
rollout,after,91.56,k6,d3da387bc8da,0.0806,0.0,0.98,454.8
|
||||
rollout,after,91.56,sampler,565f94da69f4,0.0032,0.0,0.01,1.8
|
||||
rollout,after,91.56,vm,,0.8964,,2.59,
|
||||
rollout,after,92.57,api-new,2ec1a8170637,0.2868,0.0,0.97,125.1
|
||||
rollout,after,92.57,api-new,9128897910ef,0.292,0.0,1.1,125.1
|
||||
rollout,after,92.57,postgres,2708a61ed609,0.0936,0.0,0.77,200.2
|
||||
rollout,after,92.57,haproxy,e1726165eec2,0.0214,0.0,0.44,38.0
|
||||
rollout,after,92.57,valkey,dbe0fec0557d,0.0078,0.0,0.31,12.8
|
||||
rollout,after,92.57,k6,d3da387bc8da,0.0807,0.0,1.24,454.6
|
||||
rollout,after,92.57,sampler,565f94da69f4,0.0035,0.0,0.02,1.9
|
||||
rollout,after,92.57,vm,,1.1359,,6.49,
|
||||
rollout,after,93.57,api-new,2ec1a8170637,0.3489,0.0,1.95,124.8
|
||||
rollout,after,93.57,api-new,9128897910ef,0.2763,0.0,1.09,124.9
|
||||
rollout,after,93.57,postgres,2708a61ed609,0.0885,0.0,0.64,200.4
|
||||
rollout,after,93.57,haproxy,e1726165eec2,0.0202,0.0,0.43,38.2
|
||||
rollout,after,93.57,valkey,dbe0fec0557d,0.0201,0.0,0.32,12.5
|
||||
rollout,after,93.57,k6,d3da387bc8da,0.0755,0.0,1.12,454.5
|
||||
rollout,after,93.57,sampler,565f94da69f4,0.0033,0.0,0.02,1.7
|
||||
rollout,after,93.57,vm,,0.9561,,3.01,
|
||||
rollout,after,94.57,api-new,2ec1a8170637,0.2754,0.0,1.58,124.9
|
||||
rollout,after,94.57,api-new,9128897910ef,0.2793,0.0,1.4,125.0
|
||||
rollout,after,94.57,postgres,2708a61ed609,0.1146,0.0,0.74,200.3
|
||||
rollout,after,94.57,haproxy,e1726165eec2,0.0336,0.0,0.37,38.0
|
||||
rollout,after,94.57,valkey,dbe0fec0557d,0.0077,0.0,0.31,12.6
|
||||
rollout,after,94.57,k6,d3da387bc8da,0.0777,0.0,1.09,454.5
|
||||
rollout,after,94.57,sampler,565f94da69f4,0.0033,0.0,0.01,1.8
|
||||
rollout,after,94.57,vm,,0.9855,,3.28,
|
||||
rollout,after,95.58,api-new,2ec1a8170637,0.288,0.0,2.02,125.1
|
||||
rollout,after,95.58,api-new,9128897910ef,0.3642,0.0,1.21,125.3
|
||||
rollout,after,95.58,postgres,2708a61ed609,0.0941,0.0,0.68,200.2
|
||||
rollout,after,95.58,haproxy,e1726165eec2,0.0218,0.0,0.72,37.9
|
||||
rollout,after,95.58,valkey,dbe0fec0557d,0.0075,0.0,0.38,12.6
|
||||
rollout,after,95.58,k6,d3da387bc8da,0.0795,0.0,0.97,454.9
|
||||
rollout,after,95.58,sampler,565f94da69f4,0.0032,0.0,0.04,1.8
|
||||
rollout,after,95.58,vm,,0.9761,,2.5,
|
||||
rollout,after,96.58,api-new,2ec1a8170637,0.2709,0.0,2.03,124.7
|
||||
rollout,after,96.58,api-new,9128897910ef,0.2758,0.0,1.46,125.1
|
||||
rollout,after,96.58,postgres,2708a61ed609,0.0904,0.0,0.74,200.7
|
||||
rollout,after,96.58,haproxy,e1726165eec2,0.0198,0.0,0.65,38.2
|
||||
rollout,after,96.58,valkey,dbe0fec0557d,0.0071,0.0,0.46,12.3
|
||||
rollout,after,96.58,k6,d3da387bc8da,0.0763,0.0,1.12,455.2
|
||||
rollout,after,96.58,sampler,565f94da69f4,0.0028,0.0,0.01,2.1
|
||||
rollout,after,96.58,vm,,0.8671,,2.64,
|
||||
rollout,after,97.58,api-new,2ec1a8170637,0.285,0.0,1.27,124.4
|
||||
rollout,after,97.58,api-new,9128897910ef,0.2902,0.0,1.79,124.7
|
||||
rollout,after,97.58,postgres,2708a61ed609,0.0932,0.0,1.02,200.6
|
||||
rollout,after,97.58,haproxy,e1726165eec2,0.0207,0.0,0.79,37.8
|
||||
rollout,after,97.58,valkey,dbe0fec0557d,0.0074,0.0,0.31,12.6
|
||||
rollout,after,97.58,k6,d3da387bc8da,0.0784,0.0,1.21,454.9
|
||||
rollout,after,97.58,sampler,565f94da69f4,0.0034,0.0,0.01,2.1
|
||||
rollout,after,97.58,vm,,0.9067,,2.77,
|
||||
rollout,after,98.59,api-new,2ec1a8170637,0.2854,0.0,1.81,124.6
|
||||
rollout,after,98.59,api-new,9128897910ef,0.2892,0.0,1.61,124.9
|
||||
rollout,after,98.59,postgres,2708a61ed609,0.0933,0.0,1.03,200.7
|
||||
rollout,after,98.59,haproxy,e1726165eec2,0.0215,0.0,0.74,37.6
|
||||
rollout,after,98.59,valkey,dbe0fec0557d,0.0197,0.0,0.32,12.9
|
||||
rollout,after,98.59,k6,d3da387bc8da,0.0789,0.0,1.0,454.8
|
||||
rollout,after,98.59,sampler,565f94da69f4,0.0035,0.0,0.02,1.3
|
||||
rollout,after,98.59,vm,,0.8964,,3.11,
|
||||
rollout,after,99.59,api-new,2ec1a8170637,0.272,0.0,0.92,125.2
|
||||
rollout,after,99.59,api-new,9128897910ef,0.2763,0.0,1.14,125.2
|
||||
rollout,after,99.59,postgres,2708a61ed609,0.1152,0.0,0.75,200.5
|
||||
rollout,after,99.59,haproxy,e1726165eec2,0.0313,0.0,0.36,37.9
|
||||
rollout,after,99.59,valkey,dbe0fec0557d,0.007,0.0,0.36,12.4
|
||||
rollout,after,99.59,k6,d3da387bc8da,0.076,0.0,0.99,454.9
|
||||
rollout,after,99.59,sampler,565f94da69f4,0.0031,0.0,0.01,1.6
|
||||
rollout,after,99.59,vm,,0.916,,3.23,
|
||||
rollout,after,100.6,api-new,2ec1a8170637,0.2827,0.0,2.19,124.9
|
||||
rollout,after,100.6,api-new,9128897910ef,0.2847,0.0,1.85,125.1
|
||||
rollout,after,100.6,postgres,2708a61ed609,0.0914,0.0,0.83,200.5
|
||||
rollout,after,100.6,haproxy,e1726165eec2,0.0208,0.0,0.55,38.1
|
||||
rollout,after,100.6,valkey,dbe0fec0557d,0.0077,0.0,0.56,12.3
|
||||
rollout,after,100.6,k6,d3da387bc8da,0.0778,0.0,1.09,455.1
|
||||
rollout,after,100.6,sampler,565f94da69f4,0.0036,0.0,0.02,1.9
|
||||
rollout,after,100.6,vm,,0.8666,,2.62,
|
||||
rollout,after,101.6,api-new,2ec1a8170637,0.2876,0.0,2.05,125.1
|
||||
rollout,after,101.6,api-new,9128897910ef,0.2892,0.0,1.28,125.1
|
||||
rollout,after,101.6,postgres,2708a61ed609,0.092,0.0,0.81,200.5
|
||||
rollout,after,101.6,haproxy,e1726165eec2,0.0214,0.0,0.39,37.8
|
||||
rollout,after,101.6,valkey,dbe0fec0557d,0.0077,0.0,0.37,12.6
|
||||
rollout,after,101.6,k6,d3da387bc8da,0.0781,0.0,1.1,455.0
|
||||
rollout,after,101.6,sampler,565f94da69f4,0.0035,0.0,0.01,1.7
|
||||
rollout,after,101.6,vm,,0.8962,,2.94,
|
||||
rollout,after,102.61,api-new,2ec1a8170637,0.2891,0.0,1.33,125.1
|
||||
rollout,after,102.61,api-new,9128897910ef,0.2891,0.0,1.46,125.3
|
||||
rollout,after,102.61,postgres,2708a61ed609,0.0941,0.0,1.35,201.4
|
||||
rollout,after,102.61,haproxy,e1726165eec2,0.0214,0.0,0.51,38.1
|
||||
rollout,after,102.61,valkey,dbe0fec0557d,0.0077,0.0,0.96,12.0
|
||||
rollout,after,102.61,k6,d3da387bc8da,0.0824,0.0,1.47,455.3
|
||||
rollout,after,102.61,sampler,565f94da69f4,0.0037,0.0,0.02,1.7
|
||||
rollout,after,102.61,vm,,1.1654,,8.66,
|
||||
rollout,after,103.61,api-new,2ec1a8170637,0.2806,0.0,1.91,125.1
|
||||
rollout,after,103.61,api-new,9128897910ef,0.2835,0.0,1.82,124.8
|
||||
rollout,after,103.61,postgres,2708a61ed609,0.095,0.0,0.85,201.9
|
||||
rollout,after,103.61,haproxy,e1726165eec2,0.021,0.0,0.91,37.7
|
||||
rollout,after,103.61,valkey,dbe0fec0557d,0.02,0.0,0.47,13.1
|
||||
rollout,after,103.61,k6,d3da387bc8da,0.0765,0.0,1.27,454.9
|
||||
rollout,after,103.61,sampler,565f94da69f4,0.0036,0.0,0.03,1.9
|
||||
rollout,after,103.61,vm,,0.9456,,3.23,
|
||||
|
@@ -0,0 +1,94 @@
|
||||
# Поочерёдная замена реплик под нагрузкой: метка `attachments`
|
||||
|
||||
- Статус: **complete**
|
||||
- Commit: `d56b892` + незакоммиченные изменения
|
||||
- Профиль `rollout`: 2 реплики за HAProxy; замена с образа commit `e1161f7` на образ текущего checkout (`scripts/rolling-update.sh`); 100 записей с ключом и 100 чтений страницы истории в секунду (k6, постоянная интенсивность); окна: 15 с до запуска скрипта, бюджет замены 90 с, ещё 15 с; порог доли успешных 99%
|
||||
- Лимиты без swap: API 1 CPU / 512m на реплику, HAProxy 1 CPU / 256m, PostgreSQL 2 CPU / 1g, Valkey 0.5 CPU / 128m, сэмплер 0.25 CPU / 128m, k6 без лимита
|
||||
- Посторонние контейнеры в VM во время замера: нет
|
||||
|
||||
Время — секунды на часах ВМ Docker от начала замены (создание контейнера миграции). Шаги — события Docker (создание и старт новой реплики, SIGTERM, выход и удаление старой) и журнал HAProxy (`UP` новой, `drain` и `maint` старой, возврат слота в `ready`). Ответы — отметки k6 при отправке; «старые» и «новые» — по `X-Instance-Id` контейнеров до и после замены. Окно замены — от начала до удаления последней старой реплики. Сверка — после повтора неуспешных записей с тем же ключом: сообщений в чате записи и разных текстов столько, сколько отправлено ключей.
|
||||
|
||||
| Замена, с | Сборка / миграция, с | Ревизия до → после | Ошибки до / во время / после | Мин. доля успешных | Секунд ниже порога | Успешных в секунду: до / мин. во время | p95: до / макс. во время, мс | Первый ответ новой / последний старой, с | Ответов старых после дренажа | Повтор записей | Сообщений = ключей | Пометки |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 8.26 | 0 / 2 | `e656d6bbe1a4` → `ef25d50b01a8` | 0 / 0 / 0 | 1 | 0 | 200 / 200 | 4 / 43 | 4.18 / 7.61 | 0 | 0 → 0 | 12001 = 12001 | — |
|
||||
|
||||
## Шаги по репликам
|
||||
|
||||
| Шаг | Новая (слот) | Старая (слот) | Новая создана, с | Новая `UP`, с | Старт → `UP`, с | Дренаж, с | Остановка, с | Старая удалена, с | Слот `ready`, с | Замена целиком, с | Ошибок за шаг | Ответов старой после дренажа |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| 1 | `2ec1a8170637` (api-3) | `9c7a7d981f5d` (api-2) | 1.81 | 4.17 | 2.26 | 0.14 | 0.35 | 5.28 | 5.79 | 3.47 | 0 | 0 |
|
||||
| 2 | `9128897910ef` (api-2) | `836903d5ebd9` (api-1) | 5.99 | 7.15 | 1.07 | 0.14 | 0.39 | 8.26 | 8.78 | 2.28 | 0 | 0 |
|
||||
|
||||
## Ресурсы по окнам замены
|
||||
|
||||
Ресурсы — сэмплер `bench/sampler.sh` в режиме `scan` (свой контейнер без сети, лимит 0,25 CPU / 128 МиБ): раз в секунду счётчики cgroup каждого контейнера стенда, k6 и самого сэмплера и `/proc/stat` ВМ, на тех же часах ВМ, что шкала. Ячейка — среднее CPU за окно (CPU-секунды / длина окна, ядер) / наибольшее посекундное: CPU интервала между отсчётами / его длина по всем интервалам, задевающим окно, включая граничный (у заморозки — только с CPU в окне); строка граничного интервала в `samples.csv` помечена соседним окном (окно строки — по концу интервала, у заморозки — по концу незамороженной части); пик памяти cgroup (`memory.current`, с кэшем файлов) по отсчётам, МиБ; «(N с)» — сколько секунд окна контейнер существовал, если меньше окна; «нет» — контейнера не было ни в одном отсчёте. API всего — сумма по репликам, пик — наибольшая сумма памяти на одном отсчёте. ВМ — занятые ядра ВМ Docker целиком (в них и k6, и сэмплер). Полные значения — `resources.csv`, посекундные — `samples.csv`.
|
||||
|
||||
Окна (секунды от начала замены): до — от первого запроса + 2 с (разгон k6) до первого обращения скрипта к стенду (`exec_start` в событиях проекта, кроме проверок здоровья контейнеров); подготовка — проверки скрипта и сборка образа до создания контейнера миграции; замена — до удаления последней старой реплики; после — до отправки последнего запроса. Старые и новые реплики — каждая отдельно со своим `X-Instance-Id`; на пике замены работают три реплики.
|
||||
|
||||
| Компонент | до (-14…-1.02 с) | подготовка (-1.02…0 с) | замена (0…8.26 с) | после (8.26…103.99 с) |
|
||||
|---|---|---|---|---|
|
||||
| старая `836903d5ebd9` | 0.28 / 0.34; 120 | 0.29 / 0.29; 120 | 0.22 / 0.30; 123 (7.22 с) | 0.00 / — (0 с) |
|
||||
| старая `9c7a7d981f5d` | 0.28 / 0.35; 120 | 0.29 / 0.29; 121 | 0.13 / 0.29; 122 (4.21 с) | 0.00 / — (0 с) |
|
||||
| новая `2ec1a8170637` | 0.00 / — (0 с) | 0.00 / — (0 с) | 0.25 / 1.04; 111 (6.36 с) | 0.29 / 0.36; 126 |
|
||||
| новая `9128897910ef` | 0.00 / — (0 с) | 0.00 / — (0 с) | 0.15 / 1.09; 106 (2.18 с) | 0.29 / 0.38; 126 |
|
||||
| API всего | 0.55 / 0.69; 240 | 0.57 / 0.57; 240 | 0.75 / 1.77; 349 | 0.57 / 0.67; 250 |
|
||||
| миграция `87f513915217` | 0.00 / — (0 с) | 0.00 / — (0 с) | 0.07 / 0.77; 65 (1.1 с) | 0.00 / — (0 с) |
|
||||
| PostgreSQL `2708a61ed609` | 0.08 / 0.10; 141 | 0.09 / 0.10; 138 | 0.09 / 0.14; 158 | 0.10 / 0.14; 202 |
|
||||
| HAProxy `e1726165eec2` | 0.03 / 0.04; 36 | 0.05 / 0.06; 36 | 0.05 / 0.10; 37 | 0.03 / 0.08; 45 |
|
||||
| Valkey `dbe0fec0557d` | 0.01 / 0.02; 13 | 0.01 / 0.01; 12 | 0.01 / 0.02; 13 | 0.01 / 0.02; 14 |
|
||||
| k6 `d3da387bc8da` | 0.13 / 0.36; 363 | 0.09 / 0.09; 363 | 0.07 / 0.09; 364 | 0.10 / 0.39; 455 |
|
||||
| сэмплер `565f94da69f4` | 0.00 / 0.00; 2 | 0.00 / 0.00; 2 | 0.00 / 0.00; 2 | 0.00 / 0.00; 2 |
|
||||
| ВМ | 0.96 / 1.26 | 2.04 / 2.31 | 1.70 / 2.38 | 0.97 / 1.61 |
|
||||
|
||||
### CPU на успешный запрос до и после замены, мкс
|
||||
|
||||
CPU компонента × 10⁶ / успешных в секунду в окнах до и после: две реплики одной версии, скрипт не работает. В подготовке и замене CPU на запрос не считается: CPU там идёт на сборку образа, миграцию, запуск третьей реплики, дренаж и остановку старой — цену процедуры, а не запросов; там — только абсолютные числа выше. Смесь (50 % записи, 50 % чтения страницы чата из 1 000 сообщений) другая, чем у серий трека 1: с их CPU на запрос числа напрямую не сравниваются.
|
||||
|
||||
| Компонент | до | после |
|
||||
|---|---|---|
|
||||
| API всего | 2773 | 2874 |
|
||||
| PostgreSQL | 394 | 477 |
|
||||
| HAProxy | 150 | 126 |
|
||||
| Valkey | 50 | 50 |
|
||||
| k6 | 640 | 476 |
|
||||
| сэмплер | 16 | 16 |
|
||||
| ВМ | 4799 | 4842 |
|
||||
|
||||
Успешных в секунду по окнам: до 199.9, подготовка 200.2, замена 200.1, после 200.0.
|
||||
|
||||
## Контейнеры `api` до замены
|
||||
|
||||
| Контейнер | `X-Instance-Id` | Образ (тег) | Конфигурация образа |
|
||||
|---|---|---|---|
|
||||
| mtusi-bench-api-1 | `9c7a7d981f5d` | `mtusi-messenger:bench` | `sha256:ba551a152f2c…` |
|
||||
| mtusi-bench-api-2 | `836903d5ebd9` | `mtusi-messenger:bench` | `sha256:ba551a152f2c…` |
|
||||
|
||||
## Контейнеры `api` после замены
|
||||
|
||||
| Контейнер | `X-Instance-Id` | Образ (тег) | Конфигурация образа |
|
||||
|---|---|---|---|
|
||||
| mtusi-bench-api-3 | `2ec1a8170637` | `mtusi-messenger:bench` | `sha256:ce2cce1235d3…` |
|
||||
| mtusi-bench-api-4 | `9128897910ef` | `mtusi-messenger:bench` | `sha256:ce2cce1235d3…` |
|
||||
|
||||
## Шкала вокруг замены
|
||||
|
||||
Полная шкала — `timeline.csv`.
|
||||
|
||||
| с от начала | Записи ok / ош | Чтения ok / ош | Коды ошибок | Доля успешных | p50 / p95, мс | Старые / новые ok | Ответы реплик |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| -3 | 100 / 0 | 100 / 0 | — | 1 | 3 / 4 | 200 / 0 | 836903d5ebd9:100 9c7a7d981f5d:100 |
|
||||
| -2 | 100 / 0 | 100 / 0 | — | 1 | 3 / 5 | 200 / 0 | 836903d5ebd9:100 9c7a7d981f5d:100 |
|
||||
| -1 | 100 / 0 | 100 / 0 | — | 1 | 3 / 5 | 200 / 0 | 836903d5ebd9:100 9c7a7d981f5d:100 |
|
||||
| 0 | 100 / 0 | 100 / 0 | — | 1 | 3 / 5 | 200 / 0 | 836903d5ebd9:100 9c7a7d981f5d:100 |
|
||||
| 1 | 100 / 0 | 100 / 0 | — | 1 | 3 / 5 | 200 / 0 | 836903d5ebd9:100 9c7a7d981f5d:100 |
|
||||
| 2 | 100 / 0 | 100 / 0 | — | 1 | 3 / 5 | 200 / 0 | 836903d5ebd9:100 9c7a7d981f5d:100 |
|
||||
| 3 | 100 / 0 | 100 / 0 | — | 1 | 3 / 5 | 200 / 0 | 836903d5ebd9:100 9c7a7d981f5d:100 |
|
||||
| 4 | 100 / 0 | 100 / 0 | — | 1 | 3 / 5 | 134 / 66 | 2ec1a8170637:66 836903d5ebd9:83 9c7a7d981f5d:51 |
|
||||
| 5 | 100 / 0 | 100 / 0 | — | 1 | 3 / 7 | 100 / 100 | 2ec1a8170637:100 836903d5ebd9:100 |
|
||||
| 6 | 100 / 0 | 100 / 0 | — | 1 | 3 / 6 | 100 / 100 | 2ec1a8170637:100 836903d5ebd9:100 |
|
||||
| 7 | 100 / 0 | 100 / 0 | — | 1 | 3 / 6 | 47 / 153 | 2ec1a8170637:84 836903d5ebd9:47 9128897910ef:69 |
|
||||
| 8 | 100 / 0 | 100 / 0 | — | 1 | 3 / 43 | 0 / 200 | 2ec1a8170637:100 9128897910ef:100 |
|
||||
| 9 | 100 / 0 | 100 / 0 | — | 1 | 3 / 5 | 0 / 200 | 2ec1a8170637:100 9128897910ef:100 |
|
||||
| 10 | 100 / 0 | 100 / 0 | — | 1 | 4 / 5 | 0 / 200 | 2ec1a8170637:100 9128897910ef:100 |
|
||||
| 11 | 100 / 0 | 100 / 0 | — | 1 | 4 / 5 | 0 / 200 | 2ec1a8170637:100 9128897910ef:100 |
|
||||
| 12 | 100 / 0 | 100 / 0 | — | 1 | 3 / 7 | 0 / 200 | 2ec1a8170637:100 9128897910ef:100 |
|
||||
@@ -0,0 +1,122 @@
|
||||
second,writes_ok,writes_failed,reads_ok,reads_failed,codes,success,p50_ms,p95_ms,old_ok,new_ok,instances
|
||||
-17,1,0,1,0,,1.0,5,6,2,0,836903d5ebd9:1 9c7a7d981f5d:1
|
||||
-16,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-15,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-14,100,0,100,0,,1.0,3,5,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-13,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-12,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-11,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-10,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-9,100,0,100,0,,1.0,3,27,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-8,100,0,100,0,,1.0,3,6,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-7,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-6,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-5,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-4,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-3,100,0,100,0,,1.0,3,4,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-2,100,0,100,0,,1.0,3,5,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
-1,100,0,100,0,,1.0,3,5,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
0,100,0,100,0,,1.0,3,5,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
1,100,0,100,0,,1.0,3,5,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
2,100,0,100,0,,1.0,3,5,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
3,100,0,100,0,,1.0,3,5,200,0,836903d5ebd9:100 9c7a7d981f5d:100
|
||||
4,100,0,100,0,,1.0,3,5,134,66,2ec1a8170637:66 836903d5ebd9:83 9c7a7d981f5d:51
|
||||
5,100,0,100,0,,1.0,3,7,100,100,2ec1a8170637:100 836903d5ebd9:100
|
||||
6,100,0,100,0,,1.0,3,6,100,100,2ec1a8170637:100 836903d5ebd9:100
|
||||
7,100,0,100,0,,1.0,3,6,47,153,2ec1a8170637:84 836903d5ebd9:47 9128897910ef:69
|
||||
8,100,0,100,0,,1.0,3,43,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
9,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
10,100,0,100,0,,1.0,4,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
11,100,0,100,0,,1.0,4,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
12,100,0,100,0,,1.0,3,7,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
13,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
14,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
15,100,0,100,0,,1.0,3,7,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
16,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
17,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
18,100,0,100,0,,1.0,3,15,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
19,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
20,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
21,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
22,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
23,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
24,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
25,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
26,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
27,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
28,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
29,100,0,100,0,,1.0,3,30,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
30,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
31,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
32,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
33,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
34,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
35,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
36,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
37,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
38,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
39,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
40,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
41,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
42,100,0,100,0,,1.0,3,6,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
43,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
44,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
45,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
46,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
47,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
48,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
49,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
50,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
51,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
52,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
53,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
54,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
55,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
56,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
57,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
58,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
59,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
60,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
61,100,0,100,0,,1.0,3,7,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
62,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
63,100,0,100,0,,1.0,3,16,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
64,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
65,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
66,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
67,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
68,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
69,100,0,100,0,,1.0,4,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
70,100,0,100,0,,1.0,4,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
71,100,0,100,0,,1.0,4,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
72,100,0,100,0,,1.0,3,7,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
73,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
74,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
75,100,0,100,0,,1.0,4,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
76,100,0,100,0,,1.0,3,8,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
77,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
78,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
79,100,0,100,0,,1.0,3,10,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
80,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
81,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
82,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
83,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
84,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
85,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
86,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
87,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
88,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
89,100,0,100,0,,1.0,4,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
90,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
91,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
92,100,0,100,0,,1.0,3,12,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
93,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
94,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
95,100,0,100,0,,1.0,3,9,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
96,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
97,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
98,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
99,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
100,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
101,100,0,100,0,,1.0,3,4,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
102,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
103,100,0,100,0,,1.0,3,5,0,200,2ec1a8170637:100 9128897910ef:100
|
||||
|
@@ -1,33 +1,24 @@
|
||||
# `make failure_checks` со своим S3
|
||||
# `make failure_checks` со своим S3 и проверками отказов вложений
|
||||
|
||||
- Дата: 2026-10-20, 18:35:12–18:37:20 UTC.
|
||||
- Commit: `f57635a` (рабочее дерево — только `PLAN.md` вне кода).
|
||||
- Стенд отказов: проект `mtusi-lab4-failures` (Postgres, Valkey, S3 с
|
||||
bootstrap, без портов хранилищ), два API на `127.0.0.1`. Рабочий стенд
|
||||
`mtusi-lab4` в это время запущен. Docker Desktop 29.8.2, Apple M3 Pro.
|
||||
- Новая проверка change `s3-storage` —
|
||||
`test_readiness_without_object_storage[stop|pause]`: спек `health`,
|
||||
«Хранилище объектов недоступно». Вариант `pause` добавлен по находке
|
||||
verify S3. Проба hook после `unpause` и `start` — та же команда, что
|
||||
healthcheck `s3`, с регистрацией тома (находка W1).
|
||||
- Дата: 2026-10-20, 22:41:37–22:43:32 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд отказов: проект `mtusi-lab4-failures` (PostgreSQL, Valkey, S3 с
|
||||
bootstrap, без портов хранилищ), два API на `127.0.0.1`: обычный и
|
||||
«тесный» (сроки сессии и ключа 2 с, пул 1). Рабочий стенд `mtusi-lab4`
|
||||
в это время запущен. Docker Desktop 29.8.2, Apple M3 Pro.
|
||||
- Проверки отказов change `attachments` (спек `attachments`, design
|
||||
решение 12): «Запись в S3 не удалась»
|
||||
(`test_upload_with_object_storage_stopped`), «Отклонённая загрузка
|
||||
ничего не оставляет» (`test_rejected_uploads_leave_nothing`), «S3
|
||||
остановлен» (`test_object_storage_stopped`), «S3 завис»
|
||||
(`test_object_storage_paused`). Hook считает вложения по статусам
|
||||
(`count-attachments`, psql) и объекты вложений в bucket стенда
|
||||
(`count-objects`, `weed shell fs.ls`).
|
||||
- Прежняя пересъёмка этого файла (commit `f57635a`, change
|
||||
`s3-storage`) была красной на 5 проверках из-за схемы `Message` без
|
||||
`attachment_ids` — ожидаемо до `attachments`; теперь все зелёные.
|
||||
|
||||
## Итог: код 2, пять красных — ожидаемо до change `attachments`
|
||||
|
||||
Красны 5 проверок из уточнённого списка в шапке `tasks.md` change
|
||||
`s3-storage`, у всех одна причина:
|
||||
|
||||
```
|
||||
5 E AssertionError: Message schema: /: 'attachment_ids' is a required property
|
||||
```
|
||||
|
||||
Контракт лабы 4 требует поле `attachment_ids` в `Message`, а код лабы 3
|
||||
его не отдаёт (в `src/` поля нет). Эти тесты проверяют схему ответа на
|
||||
отправку сообщения. Чинит change `attachments` (ответ пользователя
|
||||
2026-10-20, `PLAN.md`, `s3-storage`, «Вопросы», п. 2), там этот файл
|
||||
переснимается зелёным. S3 и стенд отказов здесь ни при чём: остальные
|
||||
14 проверок, включая оба варианта новой, прошли.
|
||||
|
||||
## Вывод (строки оркестратора и pytest)
|
||||
## Итог: код 0, 23 из 23
|
||||
|
||||
```
|
||||
$ make failure_checks
|
||||
@@ -40,47 +31,34 @@ s3-bootstrap-1 | s3-bootstrap: bucket messenger created
|
||||
s3-bootstrap-1 | s3-bootstrap: user api created
|
||||
s3-bootstrap-1 | s3-bootstrap: checks passed (api key put/get/delete; api key list and bucket create, anonymous get and list denied)
|
||||
[failure_checks] start api containers
|
||||
[failure_checks] ready: api=http://127.0.0.1:54756 tight=http://127.0.0.1:54761
|
||||
[failure_checks] ready: api=http://127.0.0.1:54995 tight=http://127.0.0.1:54999
|
||||
[failure_checks] pytest -m fault
|
||||
student-tests/test_failures.py::test_expired_token PASSED [ 5%]
|
||||
student-tests/test_failures.py::test_use_does_not_extend_session PASSED [ 10%]
|
||||
student-tests/test_failures.py::test_token_while_database_stopped PASSED [ 15%]
|
||||
student-tests/test_failures.py::test_members_failure_rolls_back_chat PASSED [ 21%]
|
||||
student-tests/test_failures.py::test_database_stopped_and_back FAILED [ 26%]
|
||||
student-tests/test_failures.py::test_pool_exhausted PASSED [ 31%]
|
||||
student-tests/test_failures.py::test_connection_limit_reached PASSED [ 36%]
|
||||
student-tests/test_failures.py::test_pool_keeps_connections PASSED [ 42%]
|
||||
student-tests/test_failures.py::test_readiness_without_database PASSED [ 47%]
|
||||
student-tests/test_failures.py::test_readiness_without_cache PASSED [ 52%]
|
||||
student-tests/test_failures.py::test_readiness_without_object_storage[stop] PASSED [ 57%]
|
||||
student-tests/test_failures.py::test_readiness_without_object_storage[pause] PASSED [ 63%]
|
||||
student-tests/test_failures.py::test_flush_cache PASSED [ 68%]
|
||||
student-tests/test_failures.py::test_valkey_stopped FAILED [ 73%]
|
||||
student-tests/test_failures.py::test_valkey_paused PASSED [ 78%]
|
||||
student-tests/test_failures.py::test_valkey_back PASSED [ 84%]
|
||||
student-tests/test_failures.py::test_message_failure_keeps_key_free FAILED [ 89%]
|
||||
student-tests/test_failures.py::test_database_outage_keeps_key_free FAILED [ 94%]
|
||||
student-tests/test_failures.py::test_key_free_after_ttl FAILED [100%]
|
||||
=========== 5 failed, 14 passed, 142 deselected in 69.63s (0:01:09) ============
|
||||
[failure_checks] postgres log: 381 lines, crash markers 0; "too many clients" 26, "administrator command" 129
|
||||
[failure_checks] last 50 lines of mtusi-lab4-failures-api
|
||||
[failure_checks] last 50 lines of mtusi-lab4-failures-api-tight
|
||||
student-tests/test_failures.py::test_expired_token PASSED [ 4%]
|
||||
student-tests/test_failures.py::test_use_does_not_extend_session PASSED [ 8%]
|
||||
student-tests/test_failures.py::test_token_while_database_stopped PASSED [ 13%]
|
||||
student-tests/test_failures.py::test_members_failure_rolls_back_chat PASSED [ 17%]
|
||||
student-tests/test_failures.py::test_database_stopped_and_back PASSED [ 21%]
|
||||
student-tests/test_failures.py::test_pool_exhausted PASSED [ 26%]
|
||||
student-tests/test_failures.py::test_connection_limit_reached PASSED [ 30%]
|
||||
student-tests/test_failures.py::test_pool_keeps_connections PASSED [ 34%]
|
||||
student-tests/test_failures.py::test_readiness_without_database PASSED [ 39%]
|
||||
student-tests/test_failures.py::test_readiness_without_cache PASSED [ 43%]
|
||||
student-tests/test_failures.py::test_readiness_without_object_storage[stop] PASSED [ 47%]
|
||||
student-tests/test_failures.py::test_readiness_without_object_storage[pause] PASSED [ 52%]
|
||||
student-tests/test_failures.py::test_flush_cache PASSED [ 56%]
|
||||
student-tests/test_failures.py::test_valkey_stopped PASSED [ 60%]
|
||||
student-tests/test_failures.py::test_valkey_paused PASSED [ 65%]
|
||||
student-tests/test_failures.py::test_valkey_back PASSED [ 69%]
|
||||
student-tests/test_failures.py::test_message_failure_keeps_key_free PASSED [ 73%]
|
||||
student-tests/test_failures.py::test_database_outage_keeps_key_free PASSED [ 78%]
|
||||
student-tests/test_failures.py::test_key_free_after_ttl PASSED [ 82%]
|
||||
student-tests/test_failures.py::test_upload_with_object_storage_stopped PASSED [ 86%]
|
||||
student-tests/test_failures.py::test_rejected_uploads_leave_nothing PASSED [ 91%]
|
||||
student-tests/test_failures.py::test_object_storage_stopped PASSED [ 95%]
|
||||
student-tests/test_failures.py::test_object_storage_paused PASSED [100%]
|
||||
================ 23 passed, 196 deselected in 95.64s (0:01:35) =================
|
||||
[failure_checks] postgres log: 375 lines, crash markers 0; "too many clients" 20, "administrator command" 129
|
||||
[failure_checks] cleanup
|
||||
[failure_checks] results: {'fault': 1, 'postgres': 0}
|
||||
exit=2
|
||||
```
|
||||
|
||||
Журналы API стенда отказов, которые печатает оркестратор, здесь опущены.
|
||||
|
||||
## Только проверки S3
|
||||
|
||||
```
|
||||
$ make failure_checks FAULT_ARGS='-k object_storage'
|
||||
s3-bootstrap-1 | s3-bootstrap: checks passed (api key put/get/delete; api key list and bucket create, anonymous get and list denied)
|
||||
student-tests/test_failures.py::test_readiness_without_object_storage[stop] PASSED [ 50%]
|
||||
student-tests/test_failures.py::test_readiness_without_object_storage[pause] PASSED [100%]
|
||||
====================== 2 passed, 159 deselected in 7.71s =======================
|
||||
[failure_checks] postgres log: 57 lines, crash markers 0; "too many clients" 0, "administrator command" 0
|
||||
[failure_checks] results: {'fault': 0, 'postgres': 0}
|
||||
exit=0
|
||||
```
|
||||
@@ -88,20 +66,41 @@ exit=0
|
||||
## Уборка и рабочий стенд
|
||||
|
||||
```
|
||||
working stand StartedAt unchanged
|
||||
$ docker inspect -f '{{.Name}} {{.State.StartedAt}}' $(docker compose ps -q) # до
|
||||
/mtusi-lab4-api-5 2026-10-20T22:35:43.977516884Z
|
||||
/mtusi-lab4-api-6 2026-10-20T22:36:34.974230921Z
|
||||
/mtusi-lab4-haproxy-1 2026-10-20T22:32:07.873866047Z
|
||||
/mtusi-lab4-postgres-1 2026-10-20T22:31:42.768951925Z
|
||||
/mtusi-lab4-s3-1 2026-10-20T22:31:57.790816751Z
|
||||
/mtusi-lab4-valkey-1 2026-10-20T22:31:42.807267216Z
|
||||
$ docker inspect -f '{{.Name}} {{.State.StartedAt}}' $(docker compose ps -q) # после
|
||||
/mtusi-lab4-api-5 2026-10-20T22:35:43.977516884Z
|
||||
/mtusi-lab4-api-6 2026-10-20T22:36:34.974230921Z
|
||||
/mtusi-lab4-haproxy-1 2026-10-20T22:32:07.873866047Z
|
||||
/mtusi-lab4-postgres-1 2026-10-20T22:31:42.768951925Z
|
||||
/mtusi-lab4-s3-1 2026-10-20T22:31:57.790816751Z
|
||||
/mtusi-lab4-valkey-1 2026-10-20T22:31:42.807267216Z
|
||||
left: containers 0, volumes 0
|
||||
```
|
||||
|
||||
- `StartedAt` всех контейнеров `mtusi-lab4` до и после прогона совпали.
|
||||
- `StartedAt` всех контейнеров `mtusi-lab4` до и после совпали: рабочий
|
||||
стенд не тронут.
|
||||
- У проекта `mtusi-lab4-failures` не осталось ни контейнеров, ни
|
||||
volumes, включая volume S3.
|
||||
|
||||
## Новая проверка
|
||||
## Что проверяют проверки вложений
|
||||
|
||||
`test_readiness_without_object_storage[stop|pause]`: hook останавливает
|
||||
(`stop_grace_period: 2s`) или замораживает (`docker pause`) `s3` стенда
|
||||
отказов. `GET /health/ready` и `GET /health/live` отвечают 200
|
||||
`{"status": "ok"}` быстрее 1 с, затем `s3` возвращается (`start` или
|
||||
`unpause` с пробой healthcheck). При `pause` соединение с S3 есть, а
|
||||
ответа нет: проверка, которая всё же ходила бы в S3, ждала бы срока
|
||||
вызова (5 с), а не отказа соединения.
|
||||
- «Запись в S3 не удалась»: S3 остановлен → загрузка 503 `unavailable`
|
||||
быстрее срока вызова S3 (5 с) с запасом; в БД на одну строку
|
||||
`pending` больше, `ready` — столько же; текстовое сообщение — 201; после
|
||||
`start` та же загрузка — 201.
|
||||
- «Отклонённая загрузка ничего не оставляет»: 404 постороннему, 413 по
|
||||
`Content-Length`, 413 по счётчику (частями), 413 по пикселям, 415 по
|
||||
типу и 415 по целостности — счётчики вложений по статусам и число
|
||||
объектов в bucket не изменились (перед этим одна настоящая загрузка:
|
||||
счётчики ненулевые).
|
||||
- «S3 остановлен»: скачивание 503, метаданные 200 (равны ответу
|
||||
загрузки), сообщение с этим вложением 201, `/health/ready` 200; после
|
||||
`start` скачивание — те же байты.
|
||||
- «S3 завис» (`docker pause`): загрузка и скачивание — 503 не позже
|
||||
5 с + 0,6 с; после `unpause` скачивание — те же байты.
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# `make student_test` через балансировщик
|
||||
|
||||
- Дата: 2026-10-20, 22:37:05–22:37:42 UTC (второй прогон для подсчёта
|
||||
по модулям — сразу после, тот же итог).
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`, change `attachments`).
|
||||
- Стенд: рабочий `mtusi-lab4` — `BASE_URL=http://localhost:8080`
|
||||
(HAProxy), `EXPECT_REPLICAS=2` (`API_REPLICAS`): реплики
|
||||
`5f5d4ac50d9a` и `847ad74efd7c`, PostgreSQL, Valkey, S3. Тесты реплик
|
||||
выполняются, проверки отказов (`fault`) — нет (`-m 'not fault'`, их
|
||||
гоняет `make failure_checks`, `failure-checks.md`).
|
||||
- Перед прогоном сняты файлы исключений сверки покрытия на стенде лабы 4
|
||||
(`evidence/cache/hits.md`, `evidence/auth/passwords-and-tokens.md`,
|
||||
`evidence/idempotency/replay-after-restart.md`,
|
||||
`evidence/replicas/test-persistence.md`,
|
||||
`evidence/attachments/failover.md`, `evidence/attachments/storage.md`):
|
||||
`test_exclusion_evidence_exists` зелёный.
|
||||
|
||||
## Итог
|
||||
|
||||
```
|
||||
$ make student_test
|
||||
coverage: 201 scenarios in 12 specs, 186 tested (23 by make failure_checks), 15 excluded
|
||||
===================== 196 passed, 23 deselected in 36.41s ======================
|
||||
exit=0
|
||||
```
|
||||
|
||||
## По модулям
|
||||
|
||||
```
|
||||
21 student-tests/test_api_conventions.py PASSED
|
||||
32 student-tests/test_attachments.py PASSED
|
||||
4 student-tests/test_chat_cache.py PASSED
|
||||
13 student-tests/test_chats.py PASSED
|
||||
12 student-tests/test_contract.py PASSED
|
||||
9 student-tests/test_coverage.py PASSED
|
||||
3 student-tests/test_data_storage.py PASSED
|
||||
4 student-tests/test_health.py PASSED
|
||||
20 student-tests/test_idempotency.py PASSED
|
||||
10 student-tests/test_identification.py PASSED
|
||||
2 student-tests/test_load_balancing.py PASSED
|
||||
15 student-tests/test_messages.py PASSED
|
||||
14 student-tests/test_pagination.py PASSED
|
||||
4 student-tests/test_race.py PASSED
|
||||
12 student-tests/test_rubric.py PASSED
|
||||
10 student-tests/test_sessions.py PASSED
|
||||
11 student-tests/test_users.py PASSED
|
||||
```
|
||||
|
||||
## Строки тестов реплик и гонок (`-rsP`)
|
||||
|
||||
```
|
||||
uploaded via 847ad74efd7c, downloaded via ['5f5d4ac50d9a', '847ad74efd7c']
|
||||
sent via 847ad74efd7c, replayed via ['5f5d4ac50d9a']
|
||||
409 via ['5f5d4ac50d9a', '847ad74efd7c']
|
||||
one session, 20 requests: {'5f5d4ac50d9a': 10, '847ad74efd7c': 10}
|
||||
written via 847ad74efd7c, read via {'5f5d4ac50d9a': 3, '847ad74efd7c': 3}
|
||||
race: 10 clients x 10 messages, statuses {201: 100}, max in flight 10; history: 100 messages, 100 unique ids, 4 pages by 30
|
||||
username race: 10 clients, statuses {409: 9, 201: 1}, max in flight 10
|
||||
key race: 10 clients, statuses {201: 10}, max in flight 10; history: 1 messages
|
||||
key race: 10 clients, statuses {201: 10}, max in flight 10; history: 1 messages
|
||||
key race through replicas: {'5f5d4ac50d9a': 5, '847ad74efd7c': 5}
|
||||
```
|
||||
|
||||
- Вложения лабы 4: 32 теста спека `attachments` (`test_attachments.py`)
|
||||
и пять сценариев рубрики «4» лабы 4 в `test_rubric.py` (слишком
|
||||
большой файл, повреждённое изображение, подмена MIME, чужое
|
||||
`attachment_id`, вложение другого чата): каждая ошибка клиента — 4xx
|
||||
в формате контракта, не 500. «Скачивание через другую реплику»:
|
||||
загрузку принял `847ad74efd7c`, оригинал отдал `5f5d4ac50d9a`.
|
||||
- Уровень 4 лаб 2–3: 100 сообщений от 10 клиентов — 100 уникальных в
|
||||
истории; гонка регистрации — один 201; гонка одного ключа через
|
||||
реплики — одно сообщение.
|
||||
- Сверка покрытия: все 201 сценарий 12 спеков — тест (186, из них 23
|
||||
проверками отказов) или исключение с доказательством (15).
|
||||
@@ -1,15 +1,64 @@
|
||||
# `make test_isolated`: уборка объектов S3 временной базы
|
||||
|
||||
- Спек `student-tests`: «Прогон на временной базе» (уборка префикса
|
||||
`test_<db>/`, «Хвост прошлого прогона», «Сохранить базу для разбора»).
|
||||
Design change `s3-storage`, решение 9 и «Уточнения apply», п. 11.
|
||||
- Раздел 0 — пересъёмка после change `attachments` (зелёная). Разделы
|
||||
1–3 сняты на `f57635a` (change `s3-storage`): механизм уборки с тех
|
||||
пор не менялся (`student-tests/run_isolated.py`), а красное в них —
|
||||
ожидаемое до `attachments`, как сказано ниже.
|
||||
|
||||
## 0. Пересъёмка: код 0 (change `attachments`)
|
||||
|
||||
- Дата: 2026-10-20, 22:39:22–22:41:15 UTC.
|
||||
- Commit: `d56b892` (код приложения — `116d8d3`).
|
||||
- Стенд: рабочий `mtusi-lab4`; временная база `test_messenger`, один
|
||||
контейнер API `mtusi-lab4-test-api` на `127.0.0.1:18088` без LB
|
||||
(`EXPECT_REPLICAS=1`), Valkey и S3 стенда общие.
|
||||
|
||||
```
|
||||
$ make test_isolated
|
||||
[test_isolated] database=test_messenger objects=/buckets/messenger/test_messenger/ container=mtusi-lab4-test-api base_url=http://127.0.0.1:18088
|
||||
[test_isolated] remove objects /buckets/messenger/test_messenger/
|
||||
[test_isolated] build image
|
||||
[test_isolated] create database test_messenger
|
||||
[test_isolated] migrations
|
||||
[test_isolated] start api container
|
||||
[test_isolated] ready
|
||||
[test_isolated] make test
|
||||
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
|
||||
[test_isolated] make student_test
|
||||
SKIPPED [1] student-tests/test_attachments.py:610: one API process (EXPECT_REPLICAS=1): replica checks need the stand with the load balancer
|
||||
SKIPPED [1] student-tests/test_idempotency.py:372: one API process (EXPECT_REPLICAS=1): replica checks need the stand with the load balancer
|
||||
SKIPPED [1] student-tests/test_load_balancing.py:18: one API process (EXPECT_REPLICAS=1): replica checks need the stand with the load balancer
|
||||
SKIPPED [1] student-tests/test_load_balancing.py:34: one API process (EXPECT_REPLICAS=1): replica checks need the stand with the load balancer
|
||||
SKIPPED [1] student-tests/test_race.py:169: one API process (EXPECT_REPLICAS=1): replica checks need the stand with the load balancer
|
||||
========== 191 passed, 5 skipped, 23 deselected in 100.64s (0:01:40) ===========
|
||||
[test_isolated] cleanup
|
||||
[test_isolated] remove objects /buckets/messenger/test_messenger/
|
||||
[test_isolated] results: {'test': 0, 'student_test': 0}
|
||||
exit=0
|
||||
$ printf 'fs.ls /buckets/messenger/test_messenger\n' | docker compose exec -T s3 sh -c "exec weed shell -master=127.0.0.1:9333 2>&1" | wc -l
|
||||
0
|
||||
$ psql … -d postgres -c "SELECT count(*) FROM pg_database WHERE datname = 'test_messenger'"
|
||||
0
|
||||
$ docker ps -a --filter name=mtusi-lab4-test-api -q | wc -l
|
||||
0
|
||||
```
|
||||
|
||||
- Smoke и весь набор против одного контейнера прошли; пропущены с
|
||||
причиной только 5 тестов реплик (нужен LB).
|
||||
- Тестовые загрузки (вложения `test_attachments.py` и smoke) легли под
|
||||
префикс `test_messenger/` и убраны: `fs.ls` префикса пуст, временной
|
||||
базы и контейнера нет.
|
||||
|
||||
## Итог прогона на `f57635a` (до `attachments`): код 2, красное — ожидаемо
|
||||
|
||||
- Дата: 2026-10-20, 18:39:04–18:42:42 UTC.
|
||||
- Commit: `f57635a` (рабочее дерево — только `PLAN.md` вне кода).
|
||||
- Стенд: рабочий `mtusi-lab4` (`make run_docker` с этого commit).
|
||||
Временная база `test_messenger`, S3 и Valkey стенда общие. Раздел 3 —
|
||||
временный стенд (ниже).
|
||||
- Спек `student-tests`: «Прогон на временной базе» (уборка префикса
|
||||
`test_<db>/`, «Хвост прошлого прогона», «Сохранить базу для разбора»).
|
||||
Design change `s3-storage`, решение 9 и «Уточнения apply», п. 11.
|
||||
|
||||
## Итог прогона: код 2, красное — ожидаемо до change `attachments`
|
||||
|
||||
- Smoke: `FAIL: response: missing required fields`. Схема ответа
|
||||
контракта лабы 4 требует поле `attachment_ids`, которого в коде лабы 3
|
||||
|
||||
@@ -312,7 +312,7 @@
|
||||
|
||||
Затем показать изменения, предложить сообщение коммита и после
|
||||
подтверждения закоммитить.
|
||||
- [ ] 9.2 Evidence исключений с закоммиченного кода, без токенов,
|
||||
- [x] 9.2 Evidence исключений с закоммиченного кода, без токенов,
|
||||
паролей и секретов S3, в каждом файле дата, commit и стенд:
|
||||
- четыре файла перехода по AGENTS.md («Переход в следующую лабу»).
|
||||
`evidence/replicas/test-persistence.md` — `make test-persistence` с
|
||||
@@ -324,7 +324,7 @@
|
||||
Проверка: `make student_test` — `test_exclusion_evidence_exists`
|
||||
зелёный; `grep` по значениям секретов из `.env` в `evidence/` пуст.
|
||||
Коммит — после подтверждения.
|
||||
- [ ] 9.3 Evidence проверок по таблице решения 15:
|
||||
- [x] 9.3 Evidence проверок по таблице решения 15:
|
||||
- `evidence/tests/` — зелёные `make test_isolated`, `make
|
||||
student_test` через LB, `make failure_checks`;
|
||||
- `evidence/attachments/smoke.md`, `slow-transfer.md`;
|
||||
@@ -333,11 +333,11 @@
|
||||
и после, между прогонами `make replicas`.
|
||||
|
||||
Проверка: в каждом файле команда, итог, дата, commit; секретов нет.
|
||||
- [ ] 9.4 Желательно: `make bench_rollout ROLLOUT_LABEL=attachments`
|
||||
- [x] 9.4 Желательно: `make bench_rollout ROLLOUT_LABEL=attachments`
|
||||
после проверки хоста (`bench.md`, «Результаты»), рабочий стенд на
|
||||
время замера остановлен. Проверка: код 0, в `summary.md` ревизии до и
|
||||
после, сверка записей сошлась; метка закоммичена.
|
||||
- [ ] 9.5 `REPORT.md` по решению 15:
|
||||
- [x] 9.5 `REPORT.md` по решению 15:
|
||||
- раздел о вложениях;
|
||||
- соответствие `MAX_UPLOAD_BYTES` и `MAX_IMAGE_PIXELS`;
|
||||
- ответы на четыре вопроса защиты;
|
||||
|
||||
Reference in New Issue
Block a user