Record the track 2 demo and the cost of a pass up to 100k

The demo on a clean stand (61dcb6b) passes 13 of 13 checks: the dry-run
finds exactly the injected ids in every class, apply writes one
audit.attachment_discarded reason=reconcile per stale intent, the
original bytes leave the quarantine while random bytes of the same
length keep it, docker stop ends a pass hanging on a paused S3 in
0.11 s with exit code 143, and 74 passes during 3454 uploads lose no
ready attachment; 46 times the HEAD recheck kept an in-flight upload
out of the quarantine.

The cost matrix now reaches 100k: a shallow pass takes 0.108 / 1.032 /
3.761 / 20.211 s for 1k / 10k / 30k / 100k with ceil(N/1000) listings
and SQL pages and the same 87-88 MiB of RSS. t(100k)/t(10k) is 19.6,
outside the preregistered 10 +- 30 %, because a RustFS listing page
costs 91 ms at 1k and 197 ms at 100k objects in the prefix; the
reconciler's own CPU and SQL stay linear. The previous 30k series moves
to cost-b191623/, and the old after-checks give way to the full series
that follows the track.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
This commit is contained in:
Artyom
2026-10-30 09:54:29 +03:00
co-authored by Claude Opus 5.5
parent 6ffb90d1d1
commit d2e684f0cf
72 changed files with 6333 additions and 4241 deletions
+10 -11
View File
@@ -8,27 +8,26 @@
| Файл | Что внутри | Чем получен |
| --- | --- | --- |
| `demo.json` | итог сценария: коммит, внедрённые ключи и id по классам, 12 проверок с деталями, `ok` | `make reconcile-demo` |
| `demo.json` | итог сценария: коммит, внедрённые id по классам (у объекта без строки — uuid его ключа), 13 проверок с деталями, `ok` | `make reconcile-demo` |
| `ready-before.json`, `ready-untouched.json` | id → SHA-256 готовых вложений до внедрения; то же без повреждаемых — ожидаемый набор после `apply` | `reconcile_check.py --save-ready` |
| `check-before.json`, `check-after.json` | независимая проверка (`--deep`) после внедрения и после двух `apply` | `reconcile_check.py` |
| `dry-run.json`, `apply-1.json`, `apply-2.json` | `--dry-run --deep`, затем `--apply --deep` дважды, `--grace 60 --skew 0` | сервис `reconciler` |
| `healed.json` | `--apply` после возврата объекта вложения в карантине `missing` | сервис `reconciler` |
| `dry-run.json`, `apply-1.json`, `apply-2.json` | `--dry-run --deep`, затем `--apply --deep` дважды, `--grace 60 --skew 0`; `audit` — события `audit.*` журнала прохода | сервис `reconciler` |
| `healed.json` | `--apply` без `--deep` после того, как под ключ одного вложения в карантине `missing` вернули исходные байты, а под ключ другого — случайные той же длины | сервис `reconciler` |
| `sigterm.json` | `docker stop` прохода, висящего на листинге остановленного (`pause`) S3: состояние до остановки, `init`, секунды, код выхода, не пропущен ли следующий проход | `reconcile_track.py` |
| `lock-busy.json` | проход, пока `pg_advisory_lock(20260004)` держит другой сеанс: `skipped` | сервис `reconciler` |
| `s3-down.json` | проход при остановленном `s3`: код, время, stdout, строки журнала `reconcile.*` | сервис `reconciler` |
| `ready-load.json` | id → SHA-256 всех вложений, которые должны пережить нагрузку: готовые до неё и каждое `201` под ней | `reconcile_track.py` |
| `under-load.json`, `check-under-load.json` | 75 с загрузок через LB (3 потока, каждое 10-е вложение — в сообщение) и `--apply` подряд: ответы, время проходов, исправленное, увиденное свежим; проверка `--deep --expect-ready` после | `reconcile_track.py` |
| `cost/n<N>-<режим>-<повтор>.json` | отчёт прохода `--dry-run` на N объектах и строках; режимы `shallow` (страница 1000), `deep` (`--deep`), `page100` (страница 100); `screen` — чужая нагрузка CPU за окно прогона | `make reconcile-cost` |
| `under-load.json`, `check-under-load.json` | 75 с загрузок через LB (3 потока, каждое 10-е вложение — в сообщение) и `--apply` подряд: ответы, время проходов, исправленное, увиденное свежим, отказы после повторной проверки (`rechecked_total`); проверка `--deep --expect-ready` после | `reconcile_track.py` |
| `cost/n<N>-<режим>-<повтор>.json` | отчёт прохода `--dry-run` на N = 1k, 10k, 30k, 100k объектах и строках; режимы `shallow` (страница 1000), `deep` (`--deep`), `page100` (страница 100, на 10k); `screen` — чужая нагрузка CPU за окно прогона. Перед серией прошлые `n*-*.json` и `disturbed/` удаляются | `make reconcile-cost` |
| `cost/disturbed/*.json` | отбракованные прогоны (чужого CPU больше 2 ядер во всей VM или 0,25 на закреплённых ядрах сверх запуска контейнера); каждый повторён после 15 с тишины | `make reconcile-cost` |
| `cost/summary.json`, `cost/summary.txt` | медианы трёх прогонов на точку, запросы, RSS, CPU, наибольшая чужая нагрузка, число отбракованных; закрепление CPU и лимиты `s3`/`postgres`; время наполнения | `make reconcile-cost` |
| `cost/host-docker-stats.txt` | `docker stats --all` и `docker ps` всего хоста до и после серии, loadavg macOS | `make reconcile-cost` |
| `cost-2026-10-29/` | первая полная серия той же матрицы (`5ca2cdc`, 29.10): без отбраковки и без `cpu_seconds`, рядом шёл k6 другой лабы; для сравнения между днями, `NOTE.txt` | `make reconcile-cost` на `5ca2cdc` |
| `after-checks.txt` | после трека на стенде из чистого состояния: `make test`, `test-persistence`, `test-failover`, `restore`, `reconcile`, `lint`, `check`, `validate`, `student-test` | команды внутри файла |
| `cost-aborted/` | две прерванные попытки наполнить RustFS до 100k объектов и их точки 1k/10k; диагноз — `NOTE.txt` | `make reconcile-cost` до `5ca2cdc` |
| `cost-b191623/` | прошлая серия 1k/10k/30k (`b191623`, 30.10) с отбраковкой: до исправления keyset (SQL-страниц на одну больше) и до снятия карантина по SHA-256; для сравнения | `make reconcile-cost` на `b191623` |
| `cost-2026-10-29/` | первая полная серия 1k/10k/30k (`5ca2cdc`, 29.10): без отбраковки и без `cpu_seconds`, рядом шёл k6 другой лабы; для сравнения между днями, `NOTE.txt` | `make reconcile-cost` на `5ca2cdc` |
| `cost-aborted/` | две прерванные попытки наполнить RustFS до 100k в прежних конфигурациях: RustFS 1 GiB (упор в лимит памяти) и 16 параллельных `PutObject` (запись мешала сама себе); их точки 1k/10k; диагноз — `NOTE.txt`. Итоговая конфигурация (3 GiB, 4 записи) 100k наполняет | `make reconcile-cost` до `5ca2cdc` |
Поля `screen`: `window_seconds` — окно между двумя чтениями счётчиков VM (`/proc/uptime`), `ours_cores` — CPU
`s3`, `postgres` и процесса сверки за окно, `other_cores` и `other_on_pinned_cores` — чужой CPU во всей VM и на
ядрах `s3`/`postgres`/`reconciler` сверх запуска контейнера (1,5 и 0,5 ядро·с), в ядрах, `disturbed` — прогон
отбракован. Итоговая серия снята на `b191623`: её `summary.json` записывает пороги, но не запас на запуск
контейнера и число повторов — это значения по умолчанию сценария этого коммита (1,5 и 0,5 ядро·с, 3 повтора);
начиная с `8cb5a66` они пишутся в `summary.json`. Объём данных в матрице — синтетические вложения
отбракован. Пороги, запас на запуск контейнера и число повторов записаны в `cost/summary.json` (`screen`). Объём данных в матрице — синтетические вложения
`student-tests/tools/seed_storage.py` (PNG 8 × 8, 264 B, строки `ready` двухчасовой давности).
-150
View File
@@ -1,150 +0,0 @@
date: 2026-10-30T04:08:39Z commit: 8cb5a66 стенд из чистого состояния: make reset-data && make up
$ make test
python3 tests/smoke.py --base-url "http://127.0.0.1:18084"
PASS: lab4 public API scenario; infrastructure and grading evidence remain separate.
exit=0
$ make test-persistence ACTION_SCRIPT=scripts/restart.sh
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://127.0.0.1:18084"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
exit=0
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18084"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 128, "failed_attempts": 0, "first_recovery_seconds": 2.704}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
exit=0
$ make restore
scripts/restore.sh
Container mtusi2026lab4-api-1 Running
Container mtusi2026lab4-api-2 Starting
Container mtusi2026lab4-api-2 Started
Container mtusi2026lab4-api-1 Waiting
Container mtusi2026lab4-api-2 Waiting
Container mtusi2026lab4-api-1 Healthy
Container mtusi2026lab4-api-2 Healthy
exit=0
$ make reconcile
docker compose run --rm --no-deps reconciler
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 3,
"rows": 3
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.019,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 1
},
"waited_seconds": {
"s3_list": 0.01,
"sql": 0.003
},
"hashed_bytes": 0,
"cpu_seconds": 0.455,
"max_rss_mib": 80.3
}
}
exit=0
$ make lint
.venv/bin/ruff check src student-tests scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
All checks passed!
.venv/bin/ruff format --check src student-tests scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py
143 files already formatted
.venv/bin/ty check src student-tests
All checks passed!
exit=0
$ make check
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
exit=0
$ make validate
.venv/bin/python scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
exit=0
$ make student-test
.venv/bin/pytest
============================= test session starts ==============================
platform darwin -- Python 3.14.5, pytest-9.1.1, pluggy-1.6.0
rootdir: <repo>
configfile: pyproject.toml
testpaths: student-tests
plugins: asyncio-1.4.0, anyio-4.15.1
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
collected 359 items / 17 deselected / 342 selected
student-tests/adapters/test_s3_storage.py ... [ 0%]
student-tests/api/test_access.py ......... [ 3%]
student-tests/api/test_attachments.py .................................. [ 13%]
... [ 14%]
student-tests/api/test_bounded_contexts.py .................. [ 19%]
student-tests/api/test_cache.py ...... [ 21%]
student-tests/api/test_concurrency.py .... [ 22%]
student-tests/api/test_consistency.py ........ [ 24%]
student-tests/api/test_drain.py .. [ 25%]
student-tests/api/test_events.py ...... [ 27%]
student-tests/api/test_idempotency.py ............... [ 31%]
student-tests/api/test_instance.py .... [ 32%]
student-tests/api/test_message_attachments.py ........... [ 35%]
student-tests/api/test_migrations.py ....... [ 38%]
student-tests/api/test_negative.py ..................................... [ 48%]
[ 48%]
student-tests/api/test_operability.py .............. [ 52%]
student-tests/api/test_pagination.py .................. [ 58%]
student-tests/api/test_pool.py .. [ 58%]
student-tests/api/test_reconciler.py ...... [ 60%]
student-tests/api/test_sessions.py .................. [ 65%]
student-tests/api/test_timeout.py ...... [ 67%]
student-tests/api/test_upload_consistency.py ......... [ 70%]
student-tests/api/test_wire_format.py .. [ 70%]
student-tests/unit/test_attachments_domain.py .......................... [ 78%]
. [ 78%]
student-tests/unit/test_authentication.py ..................... [ 84%]
student-tests/unit/test_cursor.py ........... [ 88%]
student-tests/unit/test_domain.py ...................................... [ 99%]
.. [ 99%]
student-tests/unit/test_timeouts.py . [100%]
===================== 342 passed, 17 deselected in 38.62s ======================
exit=0
+78 -28
View File
@@ -32,44 +32,44 @@
"errors": 0,
"samples": {
"missing": [
"01a18ae1-398b-70fa-97ec-8bd9575df28c",
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-3a28-7787-966a-82d0e9d03c3e"
"01a18b6d-7f51-7100-9ff4-889e89caccd9",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6"
],
"size_mismatch": [
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-3a31-7218-b535-13bed0785a1b",
"01a18ae1-3a39-70f1-85a1-e44886ad2399"
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fcb-759d-913d-211ccaa962ad",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea"
],
"sha_mismatch": [
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a42-720c-9027-57607a41b3fb",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2"
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1"
],
"pending_stale": [
"08b76286-4490-413c-8a90-cda6d8c81364",
"54c99776-2646-4c17-82cc-850c127a8bcb",
"565a4877-f2b6-4b6b-92c9-7a52530d43ff",
"5b16bada-62a1-4d85-b5c8-d558b1d00360",
"8fb9339c-c7a0-49ef-8cd1-058a87a351ad",
"d0aba66b-62cd-4c5c-bfba-69f002e94493"
"2a195d07-ad0a-415c-9d36-f1a6d367c616",
"32651f33-918e-4dff-a137-f0b4196286a1",
"42fba42a-d020-4f89-aad3-e9e21840d25b",
"4fe15617-cab8-469d-a25e-34a981e7c4c5",
"6678cf2a-8292-4943-8955-75d7df8f9307",
"8ec129d3-3721-4449-af97-4355cc19dd80"
],
"orphan_fresh": [
"attachments/14ebe128-13d8-498b-ac58-34e85cdde467",
"attachments/c1fc3f2b-a23e-47e3-93ce-574df23d3771"
"attachments/4b21266b-8dfc-4c67-9ffc-11575d794795",
"attachments/fa6c3d18-8e06-4b14-8a7d-acd5e1671890"
],
"orphan_stale": [
"attachments/7afa0148-2cd2-49a8-b81f-515991cd9823",
"attachments/d004f135-010c-48ac-853e-dd3871aac719",
"attachments/fbed5e6c-0c1a-4f22-980f-cf419963ddc5"
"attachments/57198272-dd0d-46d6-97f0-e68295a5e65e",
"attachments/aeebb4d7-95f8-4ebd-ac86-1031bc42e469",
"attachments/fc854d9e-6be4-4b3a-a676-d2dcf9be5993"
],
"pending_fresh": [
"acd92195-baee-4647-ba55-e9258240c898",
"be34c7f5-2c8b-4e95-b7f1-5a949ffb633e"
"80259edc-7cb8-4181-9878-f642a2b34d71",
"a3e9efb4-3de5-43a8-8d18-296bf48cc4f8"
]
},
"cost": {
"seconds": 0.111,
"seconds": 0.107,
"requests": {
"s3_list": 1,
"s3_head": 6,
@@ -78,13 +78,63 @@
"sql": 19
},
"waited_seconds": {
"s3_list": 0.016,
"sql": 0.004
"s3_list": 0.015,
"sql": 0.003
},
"hashed_bytes": 1453621,
"cpu_seconds": 0.481,
"max_rss_mib": 80.7
"cpu_seconds": 0.479,
"max_rss_mib": 84.9
},
"exit_code": 0,
"wall_seconds": 0.925
"wall_seconds": 0.922,
"audit": [
{
"attachment_id": "2a195d07-ad0a-415c-9d36-f1a6d367c616",
"chat_id": "01a18b6d-798f-75e5-ac4e-9af3b84aaa94",
"reason": "reconcile",
"event": "audit.attachment_discarded",
"level": "info",
"ts": "2026-10-30T06:10:43.376524Z"
},
{
"attachment_id": "32651f33-918e-4dff-a137-f0b4196286a1",
"chat_id": "01a18b6d-798f-75e5-ac4e-9af3b84aaa94",
"reason": "reconcile",
"event": "audit.attachment_discarded",
"level": "info",
"ts": "2026-10-30T06:10:43.386996Z"
},
{
"attachment_id": "42fba42a-d020-4f89-aad3-e9e21840d25b",
"chat_id": "01a18b6d-798f-75e5-ac4e-9af3b84aaa94",
"reason": "reconcile",
"event": "audit.attachment_discarded",
"level": "info",
"ts": "2026-10-30T06:10:43.396935Z"
},
{
"attachment_id": "4fe15617-cab8-469d-a25e-34a981e7c4c5",
"chat_id": "01a18b6d-798f-75e5-ac4e-9af3b84aaa94",
"reason": "reconcile",
"event": "audit.attachment_discarded",
"level": "info",
"ts": "2026-10-30T06:10:43.398761Z"
},
{
"attachment_id": "6678cf2a-8292-4943-8955-75d7df8f9307",
"chat_id": "01a18b6d-798f-75e5-ac4e-9af3b84aaa94",
"reason": "reconcile",
"event": "audit.attachment_discarded",
"level": "info",
"ts": "2026-10-30T06:10:43.409239Z"
},
{
"attachment_id": "8ec129d3-3721-4449-af97-4355cc19dd80",
"chat_id": "01a18b6d-798f-75e5-ac4e-9af3b84aaa94",
"reason": "reconcile",
"event": "audit.attachment_discarded",
"level": "info",
"ts": "2026-10-30T06:10:43.418544Z"
}
]
}
+19 -18
View File
@@ -32,27 +32,27 @@
"errors": 0,
"samples": {
"quarantined": [
"01a18ae1-398b-70fa-97ec-8bd9575df28c",
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-3a28-7787-966a-82d0e9d03c3e",
"01a18ae1-3a31-7218-b535-13bed0785a1b",
"01a18ae1-3a39-70f1-85a1-e44886ad2399",
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a42-720c-9027-57607a41b3fb",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2"
"01a18b6d-7f51-7100-9ff4-889e89caccd9",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6",
"01a18b6d-7fcb-759d-913d-211ccaa962ad",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea",
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1"
],
"orphan_fresh": [
"attachments/14ebe128-13d8-498b-ac58-34e85cdde467",
"attachments/c1fc3f2b-a23e-47e3-93ce-574df23d3771"
"attachments/4b21266b-8dfc-4c67-9ffc-11575d794795",
"attachments/fa6c3d18-8e06-4b14-8a7d-acd5e1671890"
],
"pending_fresh": [
"acd92195-baee-4647-ba55-e9258240c898",
"be34c7f5-2c8b-4e95-b7f1-5a949ffb633e"
"80259edc-7cb8-4181-9878-f642a2b34d71",
"a3e9efb4-3de5-43a8-8d18-296bf48cc4f8"
]
},
"cost": {
"seconds": 0.032,
"seconds": 0.031,
"requests": {
"s3_list": 1,
"s3_head": 0,
@@ -62,12 +62,13 @@
},
"waited_seconds": {
"s3_list": 0.015,
"sql": 0.003
"sql": 0.004
},
"hashed_bytes": 1453621,
"cpu_seconds": 0.461,
"max_rss_mib": 80.4
"cpu_seconds": 0.478,
"max_rss_mib": 85.3
},
"exit_code": 0,
"wall_seconds": 0.83
"wall_seconds": 0.856,
"audit": []
}
+19 -19
View File
@@ -1,5 +1,5 @@
{
"checked_at": "2026-10-30T03:37:33Z",
"checked_at": "2026-10-30T06:10:45Z",
"grace_seconds": 60,
"deep": true,
"objects": 18,
@@ -22,31 +22,31 @@
},
"samples": {
"orphan_fresh": [
"attachments/14ebe128-13d8-498b-ac58-34e85cdde467",
"attachments/c1fc3f2b-a23e-47e3-93ce-574df23d3771"
"attachments/4b21266b-8dfc-4c67-9ffc-11575d794795",
"attachments/fa6c3d18-8e06-4b14-8a7d-acd5e1671890"
],
"pending_fresh": [
"acd92195-baee-4647-ba55-e9258240c898",
"be34c7f5-2c8b-4e95-b7f1-5a949ffb633e"
"80259edc-7cb8-4181-9878-f642a2b34d71",
"a3e9efb4-3de5-43a8-8d18-296bf48cc4f8"
],
"quarantined": [
"01a18ae1-398b-70fa-97ec-8bd9575df28c",
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a28-7787-966a-82d0e9d03c3e",
"01a18ae1-3a31-7218-b535-13bed0785a1b",
"01a18ae1-3a39-70f1-85a1-e44886ad2399",
"01a18ae1-3a42-720c-9027-57607a41b3fb",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2"
"01a18b6d-7f51-7100-9ff4-889e89caccd9",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6",
"01a18b6d-7fcb-759d-913d-211ccaa962ad",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1"
]
},
"seconds": {
"s3_list": 0.342,
"s3_incomplete": 0.333,
"pg_query": 0.135,
"deep_hash": 0.632,
"total": 1.442
"s3_list": 0.349,
"s3_incomplete": 0.341,
"pg_query": 0.129,
"deep_hash": 0.583,
"total": 1.402
},
"violations": [],
"ok": true
+28 -28
View File
@@ -1,5 +1,5 @@
{
"checked_at": "2026-10-30T03:37:27Z",
"checked_at": "2026-10-30T06:10:40Z",
"grace_seconds": 60,
"deep": true,
"objects": 24,
@@ -22,52 +22,52 @@
},
"samples": {
"orphan_stale": [
"attachments/7afa0148-2cd2-49a8-b81f-515991cd9823",
"attachments/d004f135-010c-48ac-853e-dd3871aac719",
"attachments/fbed5e6c-0c1a-4f22-980f-cf419963ddc5"
"attachments/57198272-dd0d-46d6-97f0-e68295a5e65e",
"attachments/aeebb4d7-95f8-4ebd-ac86-1031bc42e469",
"attachments/fc854d9e-6be4-4b3a-a676-d2dcf9be5993"
],
"orphan_fresh": [
"attachments/14ebe128-13d8-498b-ac58-34e85cdde467",
"attachments/c1fc3f2b-a23e-47e3-93ce-574df23d3771"
"attachments/4b21266b-8dfc-4c67-9ffc-11575d794795",
"attachments/fa6c3d18-8e06-4b14-8a7d-acd5e1671890"
],
"missing_referenced": [
"01a18ae1-398b-70fa-97ec-8bd9575df28c",
"01a18ae1-39be-7126-bee7-b8d03ea89876"
"01a18b6d-7f51-7100-9ff4-889e89caccd9",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39"
],
"missing_unreferenced": [
"01a18ae1-3a28-7787-966a-82d0e9d03c3e"
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6"
],
"size_mismatch": [
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-3a31-7218-b535-13bed0785a1b",
"01a18ae1-3a39-70f1-85a1-e44886ad2399"
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fcb-759d-913d-211ccaa962ad",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea"
],
"sha_mismatch": [
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a42-720c-9027-57607a41b3fb",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2"
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1"
],
"pending_stale_with_object": [
"08b76286-4490-413c-8a90-cda6d8c81364",
"5b16bada-62a1-4d85-b5c8-d558b1d00360",
"8fb9339c-c7a0-49ef-8cd1-058a87a351ad"
"32651f33-918e-4dff-a137-f0b4196286a1",
"42fba42a-d020-4f89-aad3-e9e21840d25b",
"8ec129d3-3721-4449-af97-4355cc19dd80"
],
"pending_stale_without_object": [
"54c99776-2646-4c17-82cc-850c127a8bcb",
"565a4877-f2b6-4b6b-92c9-7a52530d43ff",
"d0aba66b-62cd-4c5c-bfba-69f002e94493"
"2a195d07-ad0a-415c-9d36-f1a6d367c616",
"4fe15617-cab8-469d-a25e-34a981e7c4c5",
"6678cf2a-8292-4943-8955-75d7df8f9307"
],
"pending_fresh": [
"acd92195-baee-4647-ba55-e9258240c898",
"be34c7f5-2c8b-4e95-b7f1-5a949ffb633e"
"80259edc-7cb8-4181-9878-f642a2b34d71",
"a3e9efb4-3de5-43a8-8d18-296bf48cc4f8"
]
},
"seconds": {
"s3_list": 0.323,
"s3_incomplete": 0.308,
"pg_query": 0.131,
"deep_hash": 0.616,
"total": 1.378
"s3_list": 0.333,
"s3_incomplete": 0.335,
"pg_query": 0.13,
"deep_hash": 0.605,
"total": 1.404
},
"violations": [
"orphan_stale",
+17 -17
View File
@@ -1,10 +1,10 @@
{
"checked_at": "2026-10-30T03:40:26Z",
"checked_at": "2026-10-30T06:13:49Z",
"grace_seconds": 60,
"deep": true,
"objects": 3435,
"rows": 3437,
"ready_healthy": 3429,
"objects": 3472,
"rows": 3473,
"ready_healthy": 3465,
"counts": {
"orphan_stale": 0,
"orphan_fresh": 0,
@@ -22,22 +22,22 @@
},
"samples": {
"quarantined": [
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a28-7787-966a-82d0e9d03c3e",
"01a18ae1-3a31-7218-b535-13bed0785a1b",
"01a18ae1-3a39-70f1-85a1-e44886ad2399",
"01a18ae1-3a42-720c-9027-57607a41b3fb",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2"
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6",
"01a18b6d-7fcb-759d-913d-211ccaa962ad",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1"
]
},
"seconds": {
"s3_list": 0.585,
"s3_incomplete": 0.327,
"pg_query": 0.15,
"deep_hash": 68.548,
"total": 69.618
"s3_list": 0.614,
"s3_incomplete": 0.344,
"pg_query": 0.138,
"deep_hash": 69.163,
"total": 70.267
},
"violations": [],
"ok": true
+6
View File
@@ -21,3 +21,9 @@ date: 2026-10-29T01:40Z commit: 7dca40c
в один префикс только мешают друг другу. 100k объектов в один префикс на этом стенде не наполнить за разумное
время; матрица снята до N = 30 000 при 4 параллельных записях. Сырые отчёты 1k/10k второго прогона заменены
повторным прогоном с той же конфигурацией.
date: 2026-10-30T06:51Z commit: 61dcb6b
Дополнение. В итоговой конфигурации (RustFS 2 CPU / 3 GiB, 4 параллельных PutObject) наполнение 30k → 100k заняло
1580 с, ≈ 44 объекта/с против ≈ 148 на отрезке 10k → 30k: запись по-прежнему дорожает с ростом префикса, но идёт.
Точка 100k снята (cost/summary.json, cost/n100000-*.json). Вывод «100k не наполнить за разумное время» выше
относится к двум прерванным попыткам в прежних конфигурациях.
@@ -0,0 +1,77 @@
== before the series date: 2026-10-30T03:48:33Z commit: b191623 host loadavg: 2.06 2.33 3.16
$ docker stats --no-stream --all
NAME CPU % MEM USAGE / LIMIT MEM % PIDS
mtusi2026lab4-api-1 13.57% 119.8MiB / 512MiB 23.39% 12
mtusi2026lab4-api-2 0.66% 145.6MiB / 512MiB 28.43% 12
mtusi2026lab4-migrate-1 0.00% 0B / 0B 0.00% 0
mtusi2026lab4-s3-init-1 0.00% 0B / 0B 0.00% 0
mtusi2026lab4-postgres-1 0.20% 47.31MiB / 1GiB 4.62% 10
mtusi2026lab4-lb-1 0.18% 19.65MiB / 128MiB 15.35% 17
mtusi2026lab4-s3-1 0.04% 57.48MiB / 512MiB 11.23% 14
mtusi2026lab4-valkey-1 0.73% 12.86MiB / 128MiB 10.05% 6
mtusi-lab6-test-rabbitmq-44e4f19b 0.36% 61.67MiB / 7.653GiB 0.79% 53
infra-tsvision-backend-1 0.00% 0B / 0B 0.00% 0
infra-tsvision-consumer-1 0.00% 0B / 0B 0.00% 0
infra-minio-init-1 0.00% 0B / 0B 0.00% 0
infra-postgres-1 0.00% 0B / 0B 0.00% 0
infra-rabbitmq-1 0.00% 0B / 0B 0.00% 0
infra-minio-1 0.00% 0B / 0B 0.00% 0
infra-mediamtx-1 0.00% 0B / 0B 0.00% 0
infra-dragonfly-1 0.00% 0B / 0B 0.00% 0
boring_merkle 0.00% 0B / 0B 0.00% 0
infallible_solomon 0.00% 0B / 0B 0.00% 0
friendly_goldwasser 0.00% 0B / 0B 0.00% 0
magical_noether 0.00% 0B / 0B 0.00% 0
postgres 0.00% 0B / 0B 0.00% 0
main_postgres 0.00% 0B / 0B 0.00% 0
fulfilment-backend 0.00% 0B / 0B 0.00% 0
$ docker ps
mtusi2026lab4-api-1 mtusi-messenger:mtusi2026lab4 Up 7 seconds (healthy)
mtusi2026lab4-api-2 mtusi-messenger:mtusi2026lab4 Up 7 seconds (healthy)
mtusi2026lab4-postgres-1 postgres:17.11-alpine Up 13 seconds (healthy)
mtusi2026lab4-lb-1 haproxy:3.2.26-alpine Up 13 seconds (healthy)
mtusi2026lab4-s3-1 rustfs/rustfs:1.0.4 Up 13 seconds (healthy)
mtusi2026lab4-valkey-1 valkey/valkey:8.1.11-alpine Up 13 seconds (healthy)
mtusi-lab6-test-rabbitmq-44e4f19b rabbitmq:4.1.8-management-alpine Up 16 hours
== after the series date: 2026-10-30T03:54:45Z commit: b191623 host loadavg: 2.79 2.46 2.91
$ docker stats --no-stream --all
NAME CPU % MEM USAGE / LIMIT MEM % PIDS
mtusi2026lab4-postgres-1 1.35% 36.84MiB / 1GiB 3.60% 10
mtusi2026lab4-s3-1 4.72% 74.16MiB / 512MiB 14.49% 13
fleetle-r5-check-step 11.00% 287.9MiB / 7.653GiB 3.67% 12
mtusi2026lab4-api-1 0.29% 151.2MiB / 512MiB 29.53% 12
mtusi2026lab4-api-2 0.35% 161MiB / 512MiB 31.44% 12
mtusi2026lab4-migrate-1 0.00% 0B / 0B 0.00% 0
mtusi2026lab4-s3-init-1 0.00% 0B / 0B 0.00% 0
mtusi2026lab4-lb-1 0.08% 19.49MiB / 128MiB 15.23% 17
mtusi2026lab4-valkey-1 1.20% 12.85MiB / 128MiB 10.04% 6
mtusi-lab6-test-rabbitmq-44e4f19b 0.17% 61.82MiB / 7.653GiB 0.79% 53
infra-tsvision-backend-1 0.00% 0B / 0B 0.00% 0
infra-tsvision-consumer-1 0.00% 0B / 0B 0.00% 0
infra-minio-init-1 0.00% 0B / 0B 0.00% 0
infra-postgres-1 0.00% 0B / 0B 0.00% 0
infra-rabbitmq-1 0.00% 0B / 0B 0.00% 0
infra-minio-1 0.00% 0B / 0B 0.00% 0
infra-mediamtx-1 0.00% 0B / 0B 0.00% 0
infra-dragonfly-1 0.00% 0B / 0B 0.00% 0
boring_merkle 0.00% 0B / 0B 0.00% 0
infallible_solomon 0.00% 0B / 0B 0.00% 0
friendly_goldwasser 0.00% 0B / 0B 0.00% 0
magical_noether 0.00% 0B / 0B 0.00% 0
postgres 0.00% 0B / 0B 0.00% 0
main_postgres 0.00% 0B / 0B 0.00% 0
fulfilment-backend 0.00% 0B / 0B 0.00% 0
$ docker ps
mtusi2026lab4-postgres-1 postgres:17.11-alpine Up 7 seconds (healthy)
mtusi2026lab4-s3-1 rustfs/rustfs:1.0.4 Up 7 seconds (healthy)
fleetle-r5-check-step node:24.22.1-alpine Up 13 seconds
mtusi2026lab4-api-1 mtusi-messenger:mtusi2026lab4 Up 6 minutes (healthy)
mtusi2026lab4-api-2 mtusi-messenger:mtusi2026lab4 Up 6 minutes (healthy)
mtusi2026lab4-lb-1 haproxy:3.2.26-alpine Up 6 minutes (healthy)
mtusi2026lab4-valkey-1 valkey/valkey:8.1.11-alpine Up 6 minutes (healthy)
mtusi-lab6-test-rabbitmq-44e4f19b rabbitmq:4.1.8-management-alpine Up 16 hours
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 1000,
"rows": 1000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.493,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 1000,
"s3_delete": 0,
"sql": 2
},
"waited_seconds": {
"s3_list": 0.091,
"sql": 0.009
},
"hashed_bytes": 375553,
"cpu_seconds": 0.857,
"max_rss_mib": 83.3
},
"exit_code": 0,
"wall_seconds": 1.284,
"screen": {
"window_seconds": 1.55,
"ours_cores": 0.7,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 1000,
"rows": 1000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.479,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 1000,
"s3_delete": 0,
"sql": 2
},
"waited_seconds": {
"s3_list": 0.091,
"sql": 0.01
},
"hashed_bytes": 375553,
"cpu_seconds": 0.837,
"max_rss_mib": 83.0
},
"exit_code": 0,
"wall_seconds": 1.278,
"screen": {
"window_seconds": 1.53,
"ours_cores": 0.65,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 1000,
"rows": 1000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.502,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 1000,
"s3_delete": 0,
"sql": 2
},
"waited_seconds": {
"s3_list": 0.09,
"sql": 0.008
},
"hashed_bytes": 375553,
"cpu_seconds": 0.857,
"max_rss_mib": 83.0
},
"exit_code": 0,
"wall_seconds": 1.301,
"screen": {
"window_seconds": 1.57,
"ours_cores": 0.69,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 1000,
"rows": 1000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.106,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 2
},
"waited_seconds": {
"s3_list": 0.09,
"sql": 0.009
},
"hashed_bytes": 0,
"cpu_seconds": 0.467,
"max_rss_mib": 82.5
},
"exit_code": 0,
"wall_seconds": 0.908,
"screen": {
"window_seconds": 1.16,
"ours_cores": 0.48,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.07,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 1000,
"rows": 1000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.105,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 2
},
"waited_seconds": {
"s3_list": 0.092,
"sql": 0.008
},
"hashed_bytes": 0,
"cpu_seconds": 0.472,
"max_rss_mib": 82.5
},
"exit_code": 0,
"wall_seconds": 0.915,
"screen": {
"window_seconds": 1.17,
"ours_cores": 0.47,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.07,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 1000,
"rows": 1000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.103,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 2
},
"waited_seconds": {
"s3_list": 0.089,
"sql": 0.008
},
"hashed_bytes": 0,
"cpu_seconds": 0.471,
"max_rss_mib": 82.5
},
"exit_code": 0,
"wall_seconds": 0.887,
"screen": {
"window_seconds": 1.16,
"ours_cores": 0.47,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 5.041,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 10000,
"s3_delete": 0,
"sql": 11
},
"waited_seconds": {
"s3_list": 1.018,
"sql": 0.045
},
"hashed_bytes": 2751553,
"cpu_seconds": 4.727,
"max_rss_mib": 84.1
},
"exit_code": 0,
"wall_seconds": 5.845,
"screen": {
"window_seconds": 6.12,
"ours_cores": 1.16,
"other_cores": 0.15,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 5.051,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 10000,
"s3_delete": 0,
"sql": 11
},
"waited_seconds": {
"s3_list": 0.986,
"sql": 0.044
},
"hashed_bytes": 2751553,
"cpu_seconds": 4.718,
"max_rss_mib": 84.1
},
"exit_code": 0,
"wall_seconds": 5.875,
"screen": {
"window_seconds": 6.14,
"ours_cores": 1.14,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 5.034,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 10000,
"s3_delete": 0,
"sql": 11
},
"waited_seconds": {
"s3_list": 0.978,
"sql": 0.045
},
"hashed_bytes": 2751553,
"cpu_seconds": 4.709,
"max_rss_mib": 84.3
},
"exit_code": 0,
"wall_seconds": 5.882,
"screen": {
"window_seconds": 6.16,
"ours_cores": 1.14,
"other_cores": 0.02,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.07,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 100,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 2.033,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 101
},
"waited_seconds": {
"s3_list": 1.943,
"sql": 0.078
},
"hashed_bytes": 0,
"cpu_seconds": 0.802,
"max_rss_mib": 81.0
},
"exit_code": 0,
"wall_seconds": 2.866,
"screen": {
"window_seconds": 3.14,
"ours_cores": 0.84,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 100,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 2.058,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 101
},
"waited_seconds": {
"s3_list": 1.966,
"sql": 0.08
},
"hashed_bytes": 0,
"cpu_seconds": 0.804,
"max_rss_mib": 81.3
},
"exit_code": 0,
"wall_seconds": 2.883,
"screen": {
"window_seconds": 3.15,
"ours_cores": 0.84,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.91,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 100,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 2.059,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 101
},
"waited_seconds": {
"s3_list": 1.969,
"sql": 0.079
},
"hashed_bytes": 0,
"cpu_seconds": 0.806,
"max_rss_mib": 81.0
},
"exit_code": 0,
"wall_seconds": 2.885,
"screen": {
"window_seconds": 3.15,
"ours_cores": 0.85,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.91,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 1.0,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 11
},
"waited_seconds": {
"s3_list": 0.946,
"sql": 0.043
},
"hashed_bytes": 0,
"cpu_seconds": 0.734,
"max_rss_mib": 83.6
},
"exit_code": 0,
"wall_seconds": 1.804,
"screen": {
"window_seconds": 2.05,
"ours_cores": 0.73,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.08,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 1.0,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 11
},
"waited_seconds": {
"s3_list": 0.946,
"sql": 0.043
},
"hashed_bytes": 0,
"cpu_seconds": 0.733,
"max_rss_mib": 83.6
},
"exit_code": 0,
"wall_seconds": 1.808,
"screen": {
"window_seconds": 2.08,
"ours_cores": 0.74,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.08,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 1.021,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 11
},
"waited_seconds": {
"s3_list": 0.967,
"sql": 0.043
},
"hashed_bytes": 0,
"cpu_seconds": 0.75,
"max_rss_mib": 83.6
},
"exit_code": 0,
"wall_seconds": 1.85,
"screen": {
"window_seconds": 2.11,
"ours_cores": 0.73,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.08,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 30000,
"rows": 30000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 15.979,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 30000,
"s3_delete": 0,
"sql": 31
},
"waited_seconds": {
"s3_list": 3.597,
"sql": 0.133
},
"hashed_bytes": 8031553,
"cpu_seconds": 13.501,
"max_rss_mib": 84.1
},
"exit_code": 0,
"wall_seconds": 16.822,
"screen": {
"window_seconds": 17.08,
"ours_cores": 1.23,
"other_cores": 0.05,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.18,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 30000,
"rows": 30000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 16.166,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 30000,
"s3_delete": 0,
"sql": 31
},
"waited_seconds": {
"s3_list": 3.655,
"sql": 0.15
},
"hashed_bytes": 8031553,
"cpu_seconds": 13.684,
"max_rss_mib": 84.1
},
"exit_code": 0,
"wall_seconds": 16.997,
"screen": {
"window_seconds": 17.27,
"ours_cores": 1.24,
"other_cores": 0.05,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.2,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 30000,
"rows": 30000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 16.15,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 30000,
"s3_delete": 0,
"sql": 31
},
"waited_seconds": {
"s3_list": 3.62,
"sql": 0.149
},
"hashed_bytes": 8031553,
"cpu_seconds": 13.786,
"max_rss_mib": 84.1
},
"exit_code": 0,
"wall_seconds": 16.997,
"screen": {
"window_seconds": 17.26,
"ours_cores": 1.24,
"other_cores": 0.2,
"other_on_pinned_cores": 0.01,
"host_loadavg_1m": 2.16,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 30000,
"rows": 30000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 3.729,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 31
},
"waited_seconds": {
"s3_list": 3.562,
"sql": 0.145
},
"hashed_bytes": 0,
"cpu_seconds": 1.365,
"max_rss_mib": 83.8
},
"exit_code": 0,
"wall_seconds": 4.576,
"screen": {
"window_seconds": 4.85,
"ours_cores": 0.92,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.29,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 30000,
"rows": 30000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 3.719,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 31
},
"waited_seconds": {
"s3_list": 3.553,
"sql": 0.144
},
"hashed_bytes": 0,
"cpu_seconds": 1.356,
"max_rss_mib": 83.7
},
"exit_code": 0,
"wall_seconds": 4.561,
"screen": {
"window_seconds": 4.83,
"ours_cores": 0.92,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.35,
"disturbed": false
}
}
@@ -0,0 +1,61 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 30000,
"rows": 30000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 3.751,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 31
},
"waited_seconds": {
"s3_list": 3.584,
"sql": 0.145
},
"hashed_bytes": 0,
"cpu_seconds": 1.371,
"max_rss_mib": 83.8
},
"exit_code": 0,
"wall_seconds": 4.595,
"screen": {
"window_seconds": 4.86,
"ours_cores": 0.94,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.32,
"disturbed": false
}
}
@@ -0,0 +1,284 @@
{
"date": "2026-10-30T03:48:39Z",
"commit": "b191623",
"pins": {
"S3_CPUS": "2",
"S3_MEMORY": "3g",
"S3_CPUSET": "11-12",
"PG_CPUSET": "13-14",
"RECONCILE_CPUSET": "15"
},
"limits": [
"/mtusi2026lab4-postgres-1 cpus=2000000000 memory=1073741824 cpuset=13-14",
"/mtusi2026lab4-s3-1 cpus=2000000000 memory=3221225472 cpuset=11-12"
],
"seeds": [
{
"added": 999,
"rows_total": 1000,
"object_bytes": 264,
"seconds": {
"s3_put": 1.439,
"pg_copy": 0.015
},
"target": 1000
},
{
"added": 9000,
"rows_total": 10000,
"object_bytes": 264,
"seconds": {
"s3_put": 23.399,
"pg_copy": 0.106
},
"target": 10000
},
{
"added": 20000,
"rows_total": 30000,
"object_bytes": 264,
"seconds": {
"s3_put": 134.854,
"pg_copy": 0.247
},
"target": 30000
}
],
"screen": {
"max_other_cores": 2.0,
"max_other_on_pinned_cores": 0.25
},
"results": [
{
"objects": 1000,
"mode": "deep",
"runs": 3,
"seconds_median": 0.493,
"seconds_all": [
0.493,
0.479,
0.502
],
"wall_seconds_median": 1.284,
"us_per_object": 493.0,
"cpu_seconds_median": 0.857,
"s3_list_wait_median": 0.091,
"sql_wait_median": 0.009,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 1000,
"s3_delete": 0,
"sql": 2
},
"max_rss_mib_median": 83.0,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 1000,
"rows": 1000
},
"found_nonzero": {}
},
{
"objects": 1000,
"mode": "shallow",
"runs": 3,
"seconds_median": 0.105,
"seconds_all": [
0.106,
0.105,
0.103
],
"wall_seconds_median": 0.908,
"us_per_object": 105.0,
"cpu_seconds_median": 0.471,
"s3_list_wait_median": 0.09,
"sql_wait_median": 0.008,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 2
},
"max_rss_mib_median": 82.5,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 1000,
"rows": 1000
},
"found_nonzero": {}
},
{
"objects": 10000,
"mode": "deep",
"runs": 3,
"seconds_median": 5.041,
"seconds_all": [
5.041,
5.051,
5.034
],
"wall_seconds_median": 5.875,
"us_per_object": 504.1,
"cpu_seconds_median": 4.718,
"s3_list_wait_median": 0.986,
"sql_wait_median": 0.045,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 10000,
"s3_delete": 0,
"sql": 11
},
"max_rss_mib_median": 84.1,
"other_cores_max": 0.15,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found_nonzero": {}
},
{
"objects": 10000,
"mode": "page100",
"runs": 3,
"seconds_median": 2.058,
"seconds_all": [
2.033,
2.058,
2.059
],
"wall_seconds_median": 2.883,
"us_per_object": 205.8,
"cpu_seconds_median": 0.804,
"s3_list_wait_median": 1.966,
"sql_wait_median": 0.079,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 101
},
"max_rss_mib_median": 81.0,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found_nonzero": {}
},
{
"objects": 10000,
"mode": "shallow",
"runs": 3,
"seconds_median": 1.0,
"seconds_all": [
1.0,
1.0,
1.021
],
"wall_seconds_median": 1.808,
"us_per_object": 100.0,
"cpu_seconds_median": 0.734,
"s3_list_wait_median": 0.946,
"sql_wait_median": 0.043,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 11
},
"max_rss_mib_median": 83.6,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 10000,
"rows": 10000
},
"found_nonzero": {}
},
{
"objects": 30000,
"mode": "deep",
"runs": 3,
"seconds_median": 16.15,
"seconds_all": [
15.979,
16.166,
16.15
],
"wall_seconds_median": 16.997,
"us_per_object": 538.3,
"cpu_seconds_median": 13.684,
"s3_list_wait_median": 3.62,
"sql_wait_median": 0.149,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 30000,
"s3_delete": 0,
"sql": 31
},
"max_rss_mib_median": 84.1,
"other_cores_max": 0.2,
"other_on_pinned_cores_max": 0.01,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 30000,
"rows": 30000
},
"found_nonzero": {}
},
{
"objects": 30000,
"mode": "shallow",
"runs": 3,
"seconds_median": 3.729,
"seconds_all": [
3.729,
3.719,
3.751
],
"wall_seconds_median": 4.576,
"us_per_object": 124.3,
"cpu_seconds_median": 1.365,
"s3_list_wait_median": 3.562,
"sql_wait_median": 0.145,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 31
},
"max_rss_mib_median": 83.8,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 30000,
"rows": 30000
},
"found_nonzero": {}
}
]
}
@@ -0,0 +1,9 @@
date: 2026-10-30T03:48:39Z commit: b191623 pins: {'S3_CPUS': '2', 'S3_MEMORY': '3g', 'S3_CPUSET': '11-12', 'PG_CPUSET': '13-14', 'RECONCILE_CPUSET': '15'}
objects mode runs t_med,s wall,s us/obj cpu,s list,s sql,s lists sqls gets rss,MiB other pinned rejected
1000 deep 3 0.493 1.28 493.0 0.86 0.091 0.009 1 2 1000 83.0 0.00 0.00 0
1000 shallow 3 0.105 0.91 105.0 0.47 0.090 0.008 1 2 0 82.5 0.00 0.00 0
10000 deep 3 5.041 5.88 504.1 4.72 0.986 0.045 10 11 10000 84.1 0.15 0.00 0
10000 page100 3 2.058 2.88 205.8 0.80 1.966 0.079 100 101 0 81.0 0.00 0.00 0
10000 shallow 3 1.000 1.81 100.0 0.73 0.946 0.043 10 11 0 83.6 0.00 0.00 0
30000 deep 3 16.150 17.00 538.3 13.68 3.620 0.149 30 31 30000 84.1 0.20 0.01 0
30000 shallow 3 3.729 4.58 124.3 1.36 3.562 0.145 30 31 0 83.8 0.00 0.00 0
+30 -32
View File
@@ -1,15 +1,15 @@
== before the series date: 2026-10-30T03:48:33Z commit: b191623 host loadavg: 2.06 2.33 3.16
== before the series date: 2026-10-30T06:14:31Z commit: 61dcb6b host loadavg: 2.05 2.31 2.31
$ docker stats --no-stream --all
NAME CPU % MEM USAGE / LIMIT MEM % PIDS
mtusi2026lab4-api-1 13.57% 119.8MiB / 512MiB 23.39% 12
mtusi2026lab4-api-2 0.66% 145.6MiB / 512MiB 28.43% 12
mtusi2026lab4-api-1 7.46% 123.5MiB / 512MiB 24.12% 12
mtusi2026lab4-api-2 10.43% 163.2MiB / 512MiB 31.87% 13
mtusi2026lab4-migrate-1 0.00% 0B / 0B 0.00% 0
mtusi2026lab4-s3-init-1 0.00% 0B / 0B 0.00% 0
mtusi2026lab4-postgres-1 0.20% 47.31MiB / 1GiB 4.62% 10
mtusi2026lab4-lb-1 0.18% 19.65MiB / 128MiB 15.35% 17
mtusi2026lab4-s3-1 0.04% 57.48MiB / 512MiB 11.23% 14
mtusi2026lab4-valkey-1 0.73% 12.86MiB / 128MiB 10.05% 6
mtusi-lab6-test-rabbitmq-44e4f19b 0.36% 61.67MiB / 7.653GiB 0.79% 53
mtusi2026lab4-postgres-1 0.13% 50.27MiB / 1GiB 4.91% 10
mtusi2026lab4-lb-1 0.86% 17.67MiB / 128MiB 13.81% 17
mtusi2026lab4-valkey-1 0.15% 13.08MiB / 128MiB 10.22% 6
mtusi2026lab4-s3-1 0.02% 59.36MiB / 512MiB 11.59% 14
mtusi-lab6-test-rabbitmq-44e4f19b 0.18% 62.39MiB / 7.653GiB 0.80% 53
infra-tsvision-backend-1 0.00% 0B / 0B 0.00% 0
infra-tsvision-consumer-1 0.00% 0B / 0B 0.00% 0
infra-minio-init-1 0.00% 0B / 0B 0.00% 0
@@ -27,28 +27,27 @@ main_postgres 0.00% 0B / 0B 0.00% 0
fulfilment-backend 0.00% 0B / 0B 0.00% 0
$ docker ps
mtusi2026lab4-api-1 mtusi-messenger:mtusi2026lab4 Up 7 seconds (healthy)
mtusi2026lab4-api-2 mtusi-messenger:mtusi2026lab4 Up 7 seconds (healthy)
mtusi2026lab4-postgres-1 postgres:17.11-alpine Up 13 seconds (healthy)
mtusi2026lab4-lb-1 haproxy:3.2.26-alpine Up 13 seconds (healthy)
mtusi2026lab4-s3-1 rustfs/rustfs:1.0.4 Up 13 seconds (healthy)
mtusi2026lab4-valkey-1 valkey/valkey:8.1.11-alpine Up 13 seconds (healthy)
mtusi-lab6-test-rabbitmq-44e4f19b rabbitmq:4.1.8-management-alpine Up 16 hours
mtusi2026lab4-api-1 mtusi-messenger:mtusi2026lab4 Up 12 seconds (healthy)
mtusi2026lab4-api-2 mtusi-messenger:mtusi2026lab4 Up 12 seconds (healthy)
mtusi2026lab4-postgres-1 postgres:17.11-alpine Up 18 seconds (healthy)
mtusi2026lab4-lb-1 haproxy:3.2.26-alpine Up 18 seconds (healthy)
mtusi2026lab4-valkey-1 valkey/valkey:8.1.11-alpine Up 18 seconds (healthy)
mtusi2026lab4-s3-1 rustfs/rustfs:1.0.4 Up 18 seconds (healthy)
mtusi-lab6-test-rabbitmq-44e4f19b rabbitmq:4.1.8-management-alpine Up 19 hours
== after the series date: 2026-10-30T03:54:45Z commit: b191623 host loadavg: 2.79 2.46 2.91
== after the series date: 2026-10-30T06:51:44Z commit: 6ffb90d host loadavg: 4.41 4.70 4.50
$ docker stats --no-stream --all
NAME CPU % MEM USAGE / LIMIT MEM % PIDS
mtusi2026lab4-postgres-1 1.35% 36.84MiB / 1GiB 3.60% 10
mtusi2026lab4-s3-1 4.72% 74.16MiB / 512MiB 14.49% 13
fleetle-r5-check-step 11.00% 287.9MiB / 7.653GiB 3.67% 12
mtusi2026lab4-api-1 0.29% 151.2MiB / 512MiB 29.53% 12
mtusi2026lab4-api-2 0.35% 161MiB / 512MiB 31.44% 12
mtusi2026lab4-postgres-1 0.12% 38.99MiB / 1GiB 3.81% 10
mtusi2026lab4-s3-1 5.49% 101.5MiB / 512MiB 19.83% 14
mtusi2026lab4-api-1 0.42% 180.7MiB / 512MiB 35.30% 12
mtusi2026lab4-api-2 0.65% 188MiB / 512MiB 36.71% 13
mtusi2026lab4-migrate-1 0.00% 0B / 0B 0.00% 0
mtusi2026lab4-s3-init-1 0.00% 0B / 0B 0.00% 0
mtusi2026lab4-lb-1 0.08% 19.49MiB / 128MiB 15.23% 17
mtusi2026lab4-valkey-1 1.20% 12.85MiB / 128MiB 10.04% 6
mtusi-lab6-test-rabbitmq-44e4f19b 0.17% 61.82MiB / 7.653GiB 0.79% 53
mtusi2026lab4-lb-1 0.76% 17.84MiB / 128MiB 13.93% 17
mtusi2026lab4-valkey-1 0.18% 13.94MiB / 128MiB 10.89% 6
mtusi-lab6-test-rabbitmq-44e4f19b 0.28% 61.61MiB / 7.653GiB 0.79% 53
infra-tsvision-backend-1 0.00% 0B / 0B 0.00% 0
infra-tsvision-consumer-1 0.00% 0B / 0B 0.00% 0
infra-minio-init-1 0.00% 0B / 0B 0.00% 0
@@ -66,12 +65,11 @@ main_postgres 0.00% 0B / 0B 0.00% 0
fulfilment-backend 0.00% 0B / 0B 0.00% 0
$ docker ps
mtusi2026lab4-postgres-1 postgres:17.11-alpine Up 7 seconds (healthy)
mtusi2026lab4-s3-1 rustfs/rustfs:1.0.4 Up 7 seconds (healthy)
fleetle-r5-check-step node:24.22.1-alpine Up 13 seconds
mtusi2026lab4-api-1 mtusi-messenger:mtusi2026lab4 Up 6 minutes (healthy)
mtusi2026lab4-api-2 mtusi-messenger:mtusi2026lab4 Up 6 minutes (healthy)
mtusi2026lab4-lb-1 haproxy:3.2.26-alpine Up 6 minutes (healthy)
mtusi2026lab4-valkey-1 valkey/valkey:8.1.11-alpine Up 6 minutes (healthy)
mtusi-lab6-test-rabbitmq-44e4f19b rabbitmq:4.1.8-management-alpine Up 16 hours
mtusi2026lab4-postgres-1 postgres:17.11-alpine Up 8 seconds (healthy)
mtusi2026lab4-s3-1 rustfs/rustfs:1.0.4 Up 8 seconds (healthy)
mtusi2026lab4-api-1 mtusi-messenger:mtusi2026lab4 Up 37 minutes (healthy)
mtusi2026lab4-api-2 mtusi-messenger:mtusi2026lab4 Up 37 minutes (healthy)
mtusi2026lab4-lb-1 haproxy:3.2.26-alpine Up 37 minutes (healthy)
mtusi2026lab4-valkey-1 valkey/valkey:8.1.11-alpine Up 37 minutes (healthy)
mtusi-lab6-test-rabbitmq-44e4f19b rabbitmq:4.1.8-management-alpine Up 19 hours
+12 -11
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.493,
"seconds": 0.546,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 1000,
"s3_delete": 0,
"sql": 2
"sql": 1
},
"waited_seconds": {
"s3_list": 0.091,
"sql": 0.009
"s3_list": 0.094,
"sql": 0.011
},
"hashed_bytes": 375553,
"cpu_seconds": 0.857,
"max_rss_mib": 83.3
"cpu_seconds": 0.924,
"max_rss_mib": 87.4
},
"exit_code": 0,
"wall_seconds": 1.284,
"wall_seconds": 1.376,
"audit": [],
"screen": {
"window_seconds": 1.55,
"ours_cores": 0.7,
"other_cores": 0.0,
"window_seconds": 1.64,
"ours_cores": 0.73,
"other_cores": 0.35,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"host_loadavg_1m": 1.91,
"disturbed": false
}
}
+13 -12
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.479,
"seconds": 0.587,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 1000,
"s3_delete": 0,
"sql": 2
"sql": 1
},
"waited_seconds": {
"s3_list": 0.091,
"sql": 0.01
"s3_list": 0.113,
"sql": 0.015
},
"hashed_bytes": 375553,
"cpu_seconds": 0.837,
"max_rss_mib": 83.0
"cpu_seconds": 0.957,
"max_rss_mib": 87.4
},
"exit_code": 0,
"wall_seconds": 1.278,
"wall_seconds": 1.44,
"audit": [],
"screen": {
"window_seconds": 1.53,
"ours_cores": 0.65,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"window_seconds": 1.72,
"ours_cores": 0.67,
"other_cores": 0.26,
"other_on_pinned_cores": 0.01,
"host_loadavg_1m": 2.0,
"disturbed": false
}
}
+10 -9
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.502,
"seconds": 0.503,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 1000,
"s3_delete": 0,
"sql": 2
"sql": 1
},
"waited_seconds": {
"s3_list": 0.09,
"sql": 0.008
"s3_list": 0.091,
"sql": 0.011
},
"hashed_bytes": 375553,
"cpu_seconds": 0.857,
"max_rss_mib": 83.0
"cpu_seconds": 0.878,
"max_rss_mib": 87.4
},
"exit_code": 0,
"wall_seconds": 1.301,
"wall_seconds": 1.348,
"audit": [],
"screen": {
"window_seconds": 1.57,
"window_seconds": 1.61,
"ours_cores": 0.69,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"host_loadavg_1m": 2.0,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.106,
"seconds": 0.108,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 2
"sql": 1
},
"waited_seconds": {
"s3_list": 0.09,
"sql": 0.009
"s3_list": 0.091,
"sql": 0.011
},
"hashed_bytes": 0,
"cpu_seconds": 0.467,
"max_rss_mib": 82.5
"cpu_seconds": 0.487,
"max_rss_mib": 86.9
},
"exit_code": 0,
"wall_seconds": 0.908,
"wall_seconds": 0.96,
"audit": [],
"screen": {
"window_seconds": 1.16,
"ours_cores": 0.48,
"window_seconds": 1.23,
"ours_cores": 0.47,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.07,
"host_loadavg_1m": 1.82,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.105,
"seconds": 0.11,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 2
"sql": 1
},
"waited_seconds": {
"s3_list": 0.092,
"sql": 0.008
"s3_list": 0.091,
"sql": 0.013
},
"hashed_bytes": 0,
"cpu_seconds": 0.472,
"max_rss_mib": 82.5
"cpu_seconds": 0.487,
"max_rss_mib": 86.9
},
"exit_code": 0,
"wall_seconds": 0.915,
"wall_seconds": 0.946,
"audit": [],
"screen": {
"window_seconds": 1.17,
"ours_cores": 0.47,
"window_seconds": 1.23,
"ours_cores": 0.46,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.07,
"host_loadavg_1m": 1.82,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 0.103,
"seconds": 0.108,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 2
"sql": 1
},
"waited_seconds": {
"s3_list": 0.089,
"sql": 0.008
"s3_list": 0.091,
"sql": 0.011
},
"hashed_bytes": 0,
"cpu_seconds": 0.471,
"max_rss_mib": 82.5
"cpu_seconds": 0.486,
"max_rss_mib": 86.9
},
"exit_code": 0,
"wall_seconds": 0.887,
"wall_seconds": 0.929,
"audit": [],
"screen": {
"window_seconds": 1.16,
"ours_cores": 0.47,
"window_seconds": 1.2,
"ours_cores": 0.48,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"host_loadavg_1m": 1.91,
"disturbed": false
}
}
+12 -11
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 5.041,
"seconds": 5.028,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 10000,
"s3_delete": 0,
"sql": 11
"sql": 10
},
"waited_seconds": {
"s3_list": 1.018,
"sql": 0.045
"s3_list": 1.002,
"sql": 0.065
},
"hashed_bytes": 2751553,
"cpu_seconds": 4.727,
"max_rss_mib": 84.1
"cpu_seconds": 4.719,
"max_rss_mib": 88.5
},
"exit_code": 0,
"wall_seconds": 5.845,
"wall_seconds": 5.865,
"audit": [],
"screen": {
"window_seconds": 6.12,
"ours_cores": 1.16,
"other_cores": 0.15,
"window_seconds": 6.13,
"ours_cores": 1.14,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"host_loadavg_1m": 2.87,
"disturbed": false
}
}
+10 -9
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 5.051,
"seconds": 5.05,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 10000,
"s3_delete": 0,
"sql": 11
"sql": 10
},
"waited_seconds": {
"s3_list": 0.986,
"sql": 0.044
"s3_list": 0.969,
"sql": 0.06
},
"hashed_bytes": 2751553,
"cpu_seconds": 4.718,
"max_rss_mib": 84.1
"cpu_seconds": 4.747,
"max_rss_mib": 88.5
},
"exit_code": 0,
"wall_seconds": 5.875,
"wall_seconds": 5.899,
"audit": [],
"screen": {
"window_seconds": 6.14,
"window_seconds": 6.15,
"ours_cores": 1.14,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"host_loadavg_1m": 2.81,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 5.034,
"seconds": 5.041,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 10000,
"s3_delete": 0,
"sql": 11
"sql": 10
},
"waited_seconds": {
"s3_list": 0.978,
"sql": 0.045
"s3_list": 0.988,
"sql": 0.06
},
"hashed_bytes": 2751553,
"cpu_seconds": 4.709,
"max_rss_mib": 84.3
"cpu_seconds": 4.739,
"max_rss_mib": 88.5
},
"exit_code": 0,
"wall_seconds": 5.882,
"wall_seconds": 5.887,
"audit": [],
"screen": {
"window_seconds": 6.16,
"window_seconds": 6.14,
"ours_cores": 1.14,
"other_cores": 0.02,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.07,
"host_loadavg_1m": 2.74,
"disturbed": false
}
}
+10 -9
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 2.033,
"seconds": 2.032,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 101
"sql": 100
},
"waited_seconds": {
"s3_list": 1.943,
"sql": 0.078
"s3_list": 1.94,
"sql": 0.081
},
"hashed_bytes": 0,
"cpu_seconds": 0.802,
"max_rss_mib": 81.0
"cpu_seconds": 0.813,
"max_rss_mib": 85.5
},
"exit_code": 0,
"wall_seconds": 2.866,
"wall_seconds": 2.869,
"audit": [],
"screen": {
"window_seconds": 3.14,
"ours_cores": 0.84,
"ours_cores": 0.86,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.99,
"host_loadavg_1m": 2.68,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 2.058,
"seconds": 2.017,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 101
"sql": 100
},
"waited_seconds": {
"s3_list": 1.966,
"sql": 0.08
"s3_list": 1.925,
"sql": 0.081
},
"hashed_bytes": 0,
"cpu_seconds": 0.804,
"max_rss_mib": 81.3
"cpu_seconds": 0.814,
"max_rss_mib": 85.8
},
"exit_code": 0,
"wall_seconds": 2.883,
"wall_seconds": 2.87,
"audit": [],
"screen": {
"window_seconds": 3.15,
"ours_cores": 0.84,
"window_seconds": 3.14,
"ours_cores": 0.85,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.91,
"host_loadavg_1m": 2.68,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 2.059,
"seconds": 2.031,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 101
"sql": 100
},
"waited_seconds": {
"s3_list": 1.969,
"sql": 0.079
"s3_list": 1.938,
"sql": 0.082
},
"hashed_bytes": 0,
"cpu_seconds": 0.806,
"max_rss_mib": 81.0
"cpu_seconds": 0.819,
"max_rss_mib": 85.6
},
"exit_code": 0,
"wall_seconds": 2.885,
"wall_seconds": 2.891,
"audit": [],
"screen": {
"window_seconds": 3.15,
"ours_cores": 0.85,
"window_seconds": 3.16,
"ours_cores": 0.84,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 1.91,
"host_loadavg_1m": 2.63,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 1.0,
"seconds": 1.016,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 11
"sql": 10
},
"waited_seconds": {
"s3_list": 0.946,
"sql": 0.043
"s3_list": 0.958,
"sql": 0.047
},
"hashed_bytes": 0,
"cpu_seconds": 0.734,
"max_rss_mib": 83.6
"cpu_seconds": 0.753,
"max_rss_mib": 88.4
},
"exit_code": 0,
"wall_seconds": 1.804,
"wall_seconds": 1.864,
"audit": [],
"screen": {
"window_seconds": 2.05,
"ours_cores": 0.73,
"window_seconds": 2.13,
"ours_cores": 0.72,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.08,
"host_loadavg_1m": 2.49,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 1.0,
"seconds": 1.032,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 11
"sql": 10
},
"waited_seconds": {
"s3_list": 0.946,
"sql": 0.043
"s3_list": 0.974,
"sql": 0.047
},
"hashed_bytes": 0,
"cpu_seconds": 0.733,
"max_rss_mib": 83.6
"cpu_seconds": 0.75,
"max_rss_mib": 88.3
},
"exit_code": 0,
"wall_seconds": 1.808,
"wall_seconds": 1.864,
"audit": [],
"screen": {
"window_seconds": 2.08,
"ours_cores": 0.74,
"window_seconds": 2.13,
"ours_cores": 0.73,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.08,
"host_loadavg_1m": 2.77,
"disturbed": false
}
}
+12 -11
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 1.021,
"seconds": 1.053,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 11
"sql": 10
},
"waited_seconds": {
"s3_list": 0.967,
"sql": 0.043
"s3_list": 0.993,
"sql": 0.049
},
"hashed_bytes": 0,
"cpu_seconds": 0.75,
"max_rss_mib": 83.6
"cpu_seconds": 0.761,
"max_rss_mib": 88.3
},
"exit_code": 0,
"wall_seconds": 1.85,
"wall_seconds": 1.926,
"audit": [],
"screen": {
"window_seconds": 2.11,
"ours_cores": 0.73,
"other_cores": 0.0,
"window_seconds": 2.19,
"ours_cores": 0.74,
"other_cores": 0.16,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.08,
"host_loadavg_1m": 2.77,
"disturbed": false
}
}
@@ -0,0 +1,62 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 100000,
"rows": 100000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 60.912,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 100000,
"s3_delete": 0,
"sql": 100
},
"waited_seconds": {
"s3_list": 20.21,
"sql": 0.467
},
"hashed_bytes": 26511553,
"cpu_seconds": 43.838,
"max_rss_mib": 88.5
},
"exit_code": 0,
"wall_seconds": 61.798,
"audit": [],
"screen": {
"window_seconds": 62.08,
"ours_cores": 1.25,
"other_cores": 0.07,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 4.98,
"disturbed": false
}
}
@@ -0,0 +1,62 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 100000,
"rows": 100000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 60.561,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 100000,
"s3_delete": 0,
"sql": 100
},
"waited_seconds": {
"s3_list": 19.909,
"sql": 0.476
},
"hashed_bytes": 26511553,
"cpu_seconds": 43.786,
"max_rss_mib": 88.6
},
"exit_code": 0,
"wall_seconds": 61.529,
"audit": [],
"screen": {
"window_seconds": 61.79,
"ours_cores": 1.25,
"other_cores": 0.08,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 5.01,
"disturbed": false
}
}
@@ -0,0 +1,62 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": true,
"page_size": 1000,
"scanned": {
"objects": 100000,
"rows": 100000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 60.637,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 100000,
"s3_delete": 0,
"sql": 100
},
"waited_seconds": {
"s3_list": 19.869,
"sql": 0.479
},
"hashed_bytes": 26511553,
"cpu_seconds": 43.901,
"max_rss_mib": 88.5
},
"exit_code": 0,
"wall_seconds": 61.545,
"audit": [],
"screen": {
"window_seconds": 61.82,
"ours_cores": 1.25,
"other_cores": 0.07,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 4.62,
"disturbed": false
}
}
@@ -0,0 +1,62 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 100000,
"rows": 100000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 23.196,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 100
},
"waited_seconds": {
"s3_list": 22.669,
"sql": 0.469
},
"hashed_bytes": 0,
"cpu_seconds": 3.488,
"max_rss_mib": 88.4
},
"exit_code": 0,
"wall_seconds": 24.175,
"audit": [],
"screen": {
"window_seconds": 24.45,
"ours_cores": 0.92,
"other_cores": 0.02,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 3.72,
"disturbed": false
}
}
@@ -0,0 +1,62 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 100000,
"rows": 100000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 19.891,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 100
},
"waited_seconds": {
"s3_list": 19.36,
"sql": 0.473
},
"hashed_bytes": 0,
"cpu_seconds": 3.502,
"max_rss_mib": 88.4
},
"exit_code": 0,
"wall_seconds": 20.826,
"audit": [],
"screen": {
"window_seconds": 21.12,
"ours_cores": 1.08,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 4.23,
"disturbed": false
}
}
@@ -0,0 +1,62 @@
{
"mode": "dry-run",
"skipped": false,
"grace_seconds": 3600,
"skew_seconds": 60,
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 100000,
"rows": 100000
},
"found": {
"orphan_stale": 0,
"orphan_fresh": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"pending_fresh": 0,
"quarantined": 0,
"healed": 0
},
"fixed": {
"orphan_stale": 0,
"missing": 0,
"size_mismatch": 0,
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 0
},
"rechecked": {},
"errors": 0,
"samples": {},
"cost": {
"seconds": 20.211,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 100
},
"waited_seconds": {
"s3_list": 19.675,
"sql": 0.478
},
"hashed_bytes": 0,
"cpu_seconds": 3.515,
"max_rss_mib": 88.4
},
"exit_code": 0,
"wall_seconds": 21.189,
"audit": [],
"screen": {
"window_seconds": 21.5,
"ours_cores": 1.04,
"other_cores": 0.05,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 5.46,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 15.979,
"seconds": 15.993,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 30000,
"s3_delete": 0,
"sql": 31
"sql": 30
},
"waited_seconds": {
"s3_list": 3.597,
"sql": 0.133
"s3_list": 3.665,
"sql": 0.162
},
"hashed_bytes": 8031553,
"cpu_seconds": 13.501,
"max_rss_mib": 84.1
"cpu_seconds": 13.451,
"max_rss_mib": 88.6
},
"exit_code": 0,
"wall_seconds": 16.822,
"wall_seconds": 16.853,
"audit": [],
"screen": {
"window_seconds": 17.08,
"window_seconds": 17.12,
"ours_cores": 1.23,
"other_cores": 0.05,
"other_cores": 0.03,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.18,
"host_loadavg_1m": 3.51,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 16.166,
"seconds": 15.785,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 30000,
"s3_delete": 0,
"sql": 31
"sql": 30
},
"waited_seconds": {
"s3_list": 3.655,
"s3_list": 3.668,
"sql": 0.15
},
"hashed_bytes": 8031553,
"cpu_seconds": 13.684,
"max_rss_mib": 84.1
"cpu_seconds": 13.373,
"max_rss_mib": 88.8
},
"exit_code": 0,
"wall_seconds": 16.997,
"wall_seconds": 16.654,
"audit": [],
"screen": {
"window_seconds": 17.27,
"ours_cores": 1.24,
"other_cores": 0.05,
"window_seconds": 16.92,
"ours_cores": 1.25,
"other_cores": 0.09,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.2,
"host_loadavg_1m": 3.22,
"disturbed": false
}
}
+13 -12
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 16.15,
"seconds": 15.903,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 30000,
"s3_delete": 0,
"sql": 31
"sql": 30
},
"waited_seconds": {
"s3_list": 3.62,
"sql": 0.149
"s3_list": 3.633,
"sql": 0.162
},
"hashed_bytes": 8031553,
"cpu_seconds": 13.786,
"max_rss_mib": 84.1
"cpu_seconds": 13.421,
"max_rss_mib": 88.5
},
"exit_code": 0,
"wall_seconds": 16.997,
"wall_seconds": 16.761,
"audit": [],
"screen": {
"window_seconds": 17.26,
"ours_cores": 1.24,
"other_cores": 0.2,
"other_on_pinned_cores": 0.01,
"host_loadavg_1m": 2.16,
"window_seconds": 17.03,
"ours_cores": 1.23,
"other_cores": 0.04,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 3.17,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 3.729,
"seconds": 3.737,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 31
"sql": 30
},
"waited_seconds": {
"s3_list": 3.562,
"sql": 0.145
"s3_list": 3.573,
"sql": 0.143
},
"hashed_bytes": 0,
"cpu_seconds": 1.365,
"max_rss_mib": 83.8
"cpu_seconds": 1.35,
"max_rss_mib": 88.4
},
"exit_code": 0,
"wall_seconds": 4.576,
"wall_seconds": 4.595,
"audit": [],
"screen": {
"window_seconds": 4.85,
"ours_cores": 0.92,
"window_seconds": 4.87,
"ours_cores": 0.9,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.29,
"host_loadavg_1m": 3.72,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 3.719,
"seconds": 3.791,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 31
"sql": 30
},
"waited_seconds": {
"s3_list": 3.553,
"sql": 0.144
"s3_list": 3.624,
"sql": 0.146
},
"hashed_bytes": 0,
"cpu_seconds": 1.356,
"max_rss_mib": 83.7
"cpu_seconds": 1.369,
"max_rss_mib": 88.3
},
"exit_code": 0,
"wall_seconds": 4.561,
"wall_seconds": 4.651,
"audit": [],
"screen": {
"window_seconds": 4.83,
"ours_cores": 0.92,
"window_seconds": 4.92,
"ours_cores": 0.94,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.35,
"host_loadavg_1m": 3.74,
"disturbed": false
}
}
+11 -10
View File
@@ -32,30 +32,31 @@
"errors": 0,
"samples": {},
"cost": {
"seconds": 3.751,
"seconds": 3.761,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 31
"sql": 30
},
"waited_seconds": {
"s3_list": 3.584,
"sql": 0.145
"s3_list": 3.593,
"sql": 0.148
},
"hashed_bytes": 0,
"cpu_seconds": 1.371,
"max_rss_mib": 83.8
"cpu_seconds": 1.364,
"max_rss_mib": 88.4
},
"exit_code": 0,
"wall_seconds": 4.595,
"wall_seconds": 4.62,
"audit": [],
"screen": {
"window_seconds": 4.86,
"ours_cores": 0.94,
"window_seconds": 4.88,
"ours_cores": 0.92,
"other_cores": 0.0,
"other_on_pinned_cores": 0.0,
"host_loadavg_1m": 2.32,
"host_loadavg_1m": 3.84,
"disturbed": false
}
}
+171 -90
View File
@@ -1,6 +1,6 @@
{
"date": "2026-10-30T03:48:39Z",
"commit": "b191623",
"date": "2026-10-30T06:14:37Z",
"commit": "61dcb6b",
"pins": {
"S3_CPUS": "2",
"S3_MEMORY": "3g",
@@ -18,8 +18,8 @@
"rows_total": 1000,
"object_bytes": 264,
"seconds": {
"s3_put": 1.439,
"pg_copy": 0.015
"s3_put": 1.017,
"pg_copy": 0.016
},
"target": 1000
},
@@ -28,8 +28,8 @@
"rows_total": 10000,
"object_bytes": 264,
"seconds": {
"s3_put": 23.399,
"pg_copy": 0.106
"s3_put": 20.901,
"pg_copy": 0.112
},
"target": 10000
},
@@ -38,42 +38,57 @@
"rows_total": 30000,
"object_bytes": 264,
"seconds": {
"s3_put": 134.854,
"pg_copy": 0.247
"s3_put": 135.164,
"pg_copy": 0.251
},
"target": 30000
},
{
"added": 70000,
"rows_total": 100000,
"object_bytes": 264,
"seconds": {
"s3_put": 1579.901,
"pg_copy": 0.921
},
"target": 100000
}
],
"screen": {
"max_other_cores": 2.0,
"max_other_on_pinned_cores": 0.25
"max_other_on_pinned_cores": 0.25,
"start_allowance_core_seconds": {
"vm": 1.5,
"pinned": 0.5
},
"retries": 3
},
"results": [
{
"objects": 1000,
"mode": "deep",
"runs": 3,
"seconds_median": 0.493,
"seconds_median": 0.546,
"seconds_all": [
0.493,
0.479,
0.502
0.546,
0.587,
0.503
],
"wall_seconds_median": 1.284,
"us_per_object": 493.0,
"cpu_seconds_median": 0.857,
"s3_list_wait_median": 0.091,
"sql_wait_median": 0.009,
"wall_seconds_median": 1.376,
"us_per_object": 546.0,
"cpu_seconds_median": 0.924,
"s3_list_wait_median": 0.094,
"sql_wait_median": 0.011,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 1000,
"s3_delete": 0,
"sql": 2
"sql": 1
},
"max_rss_mib_median": 83.0,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"max_rss_mib_median": 87.4,
"other_cores_max": 0.35,
"other_on_pinned_cores_max": 0.01,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
@@ -86,25 +101,25 @@
"objects": 1000,
"mode": "shallow",
"runs": 3,
"seconds_median": 0.105,
"seconds_median": 0.108,
"seconds_all": [
0.106,
0.105,
0.103
0.108,
0.11,
0.108
],
"wall_seconds_median": 0.908,
"us_per_object": 105.0,
"cpu_seconds_median": 0.471,
"s3_list_wait_median": 0.09,
"sql_wait_median": 0.008,
"wall_seconds_median": 0.946,
"us_per_object": 108.0,
"cpu_seconds_median": 0.487,
"s3_list_wait_median": 0.091,
"sql_wait_median": 0.011,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 2
"sql": 1
},
"max_rss_mib_median": 82.5,
"max_rss_mib_median": 86.9,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
@@ -121,24 +136,24 @@
"runs": 3,
"seconds_median": 5.041,
"seconds_all": [
5.041,
5.051,
5.034
5.028,
5.05,
5.041
],
"wall_seconds_median": 5.875,
"wall_seconds_median": 5.887,
"us_per_object": 504.1,
"cpu_seconds_median": 4.718,
"s3_list_wait_median": 0.986,
"sql_wait_median": 0.045,
"cpu_seconds_median": 4.739,
"s3_list_wait_median": 0.988,
"sql_wait_median": 0.06,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 10000,
"s3_delete": 0,
"sql": 11
"sql": 10
},
"max_rss_mib_median": 84.1,
"other_cores_max": 0.15,
"max_rss_mib_median": 88.5,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
@@ -152,25 +167,25 @@
"objects": 10000,
"mode": "page100",
"runs": 3,
"seconds_median": 2.058,
"seconds_median": 2.031,
"seconds_all": [
2.033,
2.058,
2.059
2.032,
2.017,
2.031
],
"wall_seconds_median": 2.883,
"us_per_object": 205.8,
"cpu_seconds_median": 0.804,
"s3_list_wait_median": 1.966,
"sql_wait_median": 0.079,
"wall_seconds_median": 2.87,
"us_per_object": 203.1,
"cpu_seconds_median": 0.814,
"s3_list_wait_median": 1.938,
"sql_wait_median": 0.081,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 101
"sql": 100
},
"max_rss_mib_median": 81.0,
"max_rss_mib_median": 85.6,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
@@ -185,26 +200,26 @@
"objects": 10000,
"mode": "shallow",
"runs": 3,
"seconds_median": 1.0,
"seconds_median": 1.032,
"seconds_all": [
1.0,
1.0,
1.021
1.016,
1.032,
1.053
],
"wall_seconds_median": 1.808,
"us_per_object": 100.0,
"cpu_seconds_median": 0.734,
"s3_list_wait_median": 0.946,
"sql_wait_median": 0.043,
"wall_seconds_median": 1.864,
"us_per_object": 103.2,
"cpu_seconds_median": 0.753,
"s3_list_wait_median": 0.974,
"sql_wait_median": 0.047,
"requests": {
"s3_list": 10,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 11
"sql": 10
},
"max_rss_mib_median": 83.6,
"other_cores_max": 0.0,
"max_rss_mib_median": 88.3,
"other_cores_max": 0.16,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
@@ -218,27 +233,27 @@
"objects": 30000,
"mode": "deep",
"runs": 3,
"seconds_median": 16.15,
"seconds_median": 15.903,
"seconds_all": [
15.979,
16.166,
16.15
15.993,
15.785,
15.903
],
"wall_seconds_median": 16.997,
"us_per_object": 538.3,
"cpu_seconds_median": 13.684,
"s3_list_wait_median": 3.62,
"sql_wait_median": 0.149,
"wall_seconds_median": 16.761,
"us_per_object": 530.1,
"cpu_seconds_median": 13.421,
"s3_list_wait_median": 3.665,
"sql_wait_median": 0.162,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 30000,
"s3_delete": 0,
"sql": 31
"sql": 30
},
"max_rss_mib_median": 84.1,
"other_cores_max": 0.2,
"other_on_pinned_cores_max": 0.01,
"max_rss_mib_median": 88.6,
"other_cores_max": 0.09,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
@@ -251,25 +266,25 @@
"objects": 30000,
"mode": "shallow",
"runs": 3,
"seconds_median": 3.729,
"seconds_median": 3.761,
"seconds_all": [
3.729,
3.719,
3.751
3.737,
3.791,
3.761
],
"wall_seconds_median": 4.576,
"us_per_object": 124.3,
"cpu_seconds_median": 1.365,
"s3_list_wait_median": 3.562,
"sql_wait_median": 0.145,
"wall_seconds_median": 4.62,
"us_per_object": 125.4,
"cpu_seconds_median": 1.364,
"s3_list_wait_median": 3.593,
"sql_wait_median": 0.146,
"requests": {
"s3_list": 30,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 31
"sql": 30
},
"max_rss_mib_median": 83.8,
"max_rss_mib_median": 88.4,
"other_cores_max": 0.0,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
@@ -279,6 +294,72 @@
"rows": 30000
},
"found_nonzero": {}
},
{
"objects": 100000,
"mode": "deep",
"runs": 3,
"seconds_median": 60.637,
"seconds_all": [
60.912,
60.561,
60.637
],
"wall_seconds_median": 61.545,
"us_per_object": 606.4,
"cpu_seconds_median": 43.838,
"s3_list_wait_median": 19.909,
"sql_wait_median": 0.476,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 100000,
"s3_delete": 0,
"sql": 100
},
"max_rss_mib_median": 88.5,
"other_cores_max": 0.08,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 100000,
"rows": 100000
},
"found_nonzero": {}
},
{
"objects": 100000,
"mode": "shallow",
"runs": 3,
"seconds_median": 20.211,
"seconds_all": [
23.196,
19.891,
20.211
],
"wall_seconds_median": 21.189,
"us_per_object": 202.1,
"cpu_seconds_median": 3.502,
"s3_list_wait_median": 19.675,
"sql_wait_median": 0.473,
"requests": {
"s3_list": 100,
"s3_head": 0,
"s3_get": 0,
"s3_delete": 0,
"sql": 100
},
"max_rss_mib_median": 88.4,
"other_cores_max": 0.05,
"other_on_pinned_cores_max": 0.0,
"disturbed_kept": 0,
"rejected": 0,
"scanned": {
"objects": 100000,
"rows": 100000
},
"found_nonzero": {}
}
]
}
+10 -8
View File
@@ -1,9 +1,11 @@
date: 2026-10-30T03:48:39Z commit: b191623 pins: {'S3_CPUS': '2', 'S3_MEMORY': '3g', 'S3_CPUSET': '11-12', 'PG_CPUSET': '13-14', 'RECONCILE_CPUSET': '15'}
date: 2026-10-30T06:14:37Z commit: 61dcb6b pins: {'S3_CPUS': '2', 'S3_MEMORY': '3g', 'S3_CPUSET': '11-12', 'PG_CPUSET': '13-14', 'RECONCILE_CPUSET': '15'}
objects mode runs t_med,s wall,s us/obj cpu,s list,s sql,s lists sqls gets rss,MiB other pinned rejected
1000 deep 3 0.493 1.28 493.0 0.86 0.091 0.009 1 2 1000 83.0 0.00 0.00 0
1000 shallow 3 0.105 0.91 105.0 0.47 0.090 0.008 1 2 0 82.5 0.00 0.00 0
10000 deep 3 5.041 5.88 504.1 4.72 0.986 0.045 10 11 10000 84.1 0.15 0.00 0
10000 page100 3 2.058 2.88 205.8 0.80 1.966 0.079 100 101 0 81.0 0.00 0.00 0
10000 shallow 3 1.000 1.81 100.0 0.73 0.946 0.043 10 11 0 83.6 0.00 0.00 0
30000 deep 3 16.150 17.00 538.3 13.68 3.620 0.149 30 31 30000 84.1 0.20 0.01 0
30000 shallow 3 3.729 4.58 124.3 1.36 3.562 0.145 30 31 0 83.8 0.00 0.00 0
1000 deep 3 0.546 1.38 546.0 0.92 0.094 0.011 1 1 1000 87.4 0.35 0.01 0
1000 shallow 3 0.108 0.95 108.0 0.49 0.091 0.011 1 1 0 86.9 0.00 0.00 0
10000 deep 3 5.041 5.89 504.1 4.74 0.988 0.060 10 10 10000 88.5 0.00 0.00 0
10000 page100 3 2.031 2.87 203.1 0.81 1.938 0.081 100 100 0 85.6 0.00 0.00 0
10000 shallow 3 1.032 1.86 103.2 0.75 0.974 0.047 10 10 0 88.3 0.16 0.00 0
30000 deep 3 15.903 16.76 530.1 13.42 3.665 0.162 30 30 30000 88.6 0.09 0.00 0
30000 shallow 3 3.761 4.62 125.4 1.36 3.593 0.146 30 30 0 88.4 0.00 0.00 0
100000 deep 3 60.637 61.55 606.4 43.84 19.909 0.476 100 100 100000 88.5 0.08 0.00 0
100000 shallow 3 20.211 21.19 202.1 3.50 19.675 0.473 100 100 0 88.4 0.05 0.00 0
+69 -57
View File
@@ -1,75 +1,75 @@
{
"started_at": "2026-10-30T03:36:18Z",
"commit": "5c3c238",
"started_at": "2026-10-30T06:09:31Z",
"commit": "61dcb6b",
"faults": {
"injected": {
"orphan_stale": [
"87628221-3345-4132-aecd-e73599775aee",
"a8114789-f176-44d5-8b05-8a680f0fe748",
"48a0d380-9f91-4c0a-8f8e-28245cf232d6"
"aeebb4d7-95f8-4ebd-ac86-1031bc42e469",
"57198272-dd0d-46d6-97f0-e68295a5e65e",
"fc854d9e-6be4-4b3a-a676-d2dcf9be5993"
],
"missing": [
"01a18ae1-3a28-7787-966a-82d0e9d03c3e",
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-398b-70fa-97ec-8bd9575df28c"
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7f51-7100-9ff4-889e89caccd9"
],
"size_mismatch": [
"01a18ae1-3a39-70f1-85a1-e44886ad2399",
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-3a31-7218-b535-13bed0785a1b"
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea",
"01a18b6d-7fcb-759d-913d-211ccaa962ad"
],
"sha_mismatch": [
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2",
"01a18ae1-3a42-720c-9027-57607a41b3fb"
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1",
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5"
],
"pending_stale": [
"54c99776-2646-4c17-82cc-850c127a8bcb",
"565a4877-f2b6-4b6b-92c9-7a52530d43ff",
"d0aba66b-62cd-4c5c-bfba-69f002e94493",
"5b16bada-62a1-4d85-b5c8-d558b1d00360",
"08b76286-4490-413c-8a90-cda6d8c81364",
"8fb9339c-c7a0-49ef-8cd1-058a87a351ad"
"4fe15617-cab8-469d-a25e-34a981e7c4c5",
"2a195d07-ad0a-415c-9d36-f1a6d367c616",
"6678cf2a-8292-4943-8955-75d7df8f9307",
"32651f33-918e-4dff-a137-f0b4196286a1",
"8ec129d3-3721-4449-af97-4355cc19dd80",
"42fba42a-d020-4f89-aad3-e9e21840d25b"
],
"orphan_fresh": [
"e26f690b-11ec-4540-b34d-a85b00ab8398",
"be85cf70-7931-4917-b839-9a9d7496b041"
"4b21266b-8dfc-4c67-9ffc-11575d794795",
"fa6c3d18-8e06-4b14-8a7d-acd5e1671890"
],
"pending_fresh": [
"be34c7f5-2c8b-4e95-b7f1-5a949ffb633e",
"acd92195-baee-4647-ba55-e9258240c898"
"80259edc-7cb8-4181-9878-f642a2b34d71",
"a3e9efb4-3de5-43a8-8d18-296bf48cc4f8"
]
},
"targets": {
"missing": [
"01a18ae1-398b-70fa-97ec-8bd9575df28c",
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-3a28-7787-966a-82d0e9d03c3e"
"01a18b6d-7f51-7100-9ff4-889e89caccd9",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6"
],
"corrupt": [
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-3a31-7218-b535-13bed0785a1b",
"01a18ae1-3a39-70f1-85a1-e44886ad2399"
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fcb-759d-913d-211ccaa962ad",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea"
],
"bitflip": [
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a42-720c-9027-57607a41b3fb",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2"
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1"
]
},
"linked": [
"01a18ae1-398b-70fa-97ec-8bd9575df28c",
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a0d-72ee-898c-647c03e98798",
"01a18ae1-3a1f-7044-82a6-1fac3ffdeebc"
"01a18b6d-7f51-7100-9ff4-889e89caccd9",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7faf-755c-8611-45ce0959e2d8",
"01a18b6d-7fb8-736c-82d1-6b8705f0d8d5"
]
},
"seconds": 248.3,
"seconds": 257.9,
"checks": [
{
"name": "dry-run находит каждый внедрённый класс",
"name": "dry-run находит каждый внедрённый класс, и найдено ровно внедрённое",
"ok": true,
"expected": {
"orphan_stale": 3,
@@ -106,7 +106,7 @@
]
},
{
"name": "apply исправляет ровно найденное dry-run",
"name": "apply исправляет ровно найденное dry-run, удаление намерений — в журнале аудита",
"ok": true,
"dry_run": {
"orphan_stale": 3,
@@ -121,7 +121,8 @@
"size_mismatch": 3,
"sha_mismatch": 3,
"pending_stale": 6
}
},
"audit_discarded": 6
},
{
"name": "повторный apply — 0 действий",
@@ -169,15 +170,15 @@
"name": "строки с расхождениями и ссылки сообщений сохранены",
"ok": true,
"quarantine": {
"01a18ae1-398b-70fa-97ec-8bd9575df28c": "missing",
"01a18ae1-39be-7126-bee7-b8d03ea89876": "missing",
"01a18ae1-39cf-70c4-87ec-8a0e01854664": "size_mismatch",
"01a18ae1-39da-71b9-ba5d-08d485329231": "sha_mismatch",
"01a18ae1-3a28-7787-966a-82d0e9d03c3e": "missing",
"01a18ae1-3a31-7218-b535-13bed0785a1b": "size_mismatch",
"01a18ae1-3a39-70f1-85a1-e44886ad2399": "size_mismatch",
"01a18ae1-3a42-720c-9027-57607a41b3fb": "sha_mismatch",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2": "sha_mismatch"
"01a18b6d-7f51-7100-9ff4-889e89caccd9": "missing",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39": "missing",
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55": "size_mismatch",
"01a18b6d-7fa5-75f1-accf-49b1e660d865": "sha_mismatch",
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6": "missing",
"01a18b6d-7fcb-759d-913d-211ccaa962ad": "size_mismatch",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea": "size_mismatch",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5": "sha_mismatch",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1": "sha_mismatch"
},
"message_links": 6
},
@@ -186,7 +187,7 @@
"ok": true
},
{
"name": "восстановленный объект выходит из карантина",
"name": "вернули те же байты — карантин снят; другие байты той же длины — остался",
"ok": true,
"fixed": {
"orphan_stale": 0,
@@ -195,17 +196,28 @@
"sha_mismatch": 0,
"pending_stale": 0,
"healed": 1
}
},
"s3_get": 5
},
{
"name": "второй одновременный проход пропущен",
"ok": true
},
{
"name": "SIGTERM: идущий проход останавливается за секунды, блокировка освобождена",
"ok": true,
"s3": "paused",
"state_before_stop": "running",
"init": "true",
"stop_seconds": 0.11,
"exit_code": 143,
"next_pass_skipped": false
},
{
"name": "S3 остановлен: код 1 за ограниченное время, состояние не изменилось",
"ok": true,
"exit_code": 1,
"wall_seconds": 1.633
"wall_seconds": 2.802
},
{
"name": "под нагрузкой: исправлены только внедрённые, ошибок нет",
@@ -217,13 +229,13 @@
"sha_mismatch": 0,
"pending_stale": 5
},
"runs": 75
"runs": 74
},
{
"name": "под нагрузкой: все 201 готовы и совпадают по SHA-256, потерь 0",
"ok": true,
"uploads": {
"201": 3418
"201": 3454
},
"lost_ready": 0
}
+28 -27
View File
@@ -32,44 +32,44 @@
"errors": 0,
"samples": {
"missing": [
"01a18ae1-398b-70fa-97ec-8bd9575df28c",
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-3a28-7787-966a-82d0e9d03c3e"
"01a18b6d-7f51-7100-9ff4-889e89caccd9",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6"
],
"size_mismatch": [
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-3a31-7218-b535-13bed0785a1b",
"01a18ae1-3a39-70f1-85a1-e44886ad2399"
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fcb-759d-913d-211ccaa962ad",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea"
],
"sha_mismatch": [
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a42-720c-9027-57607a41b3fb",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2"
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1"
],
"pending_stale": [
"08b76286-4490-413c-8a90-cda6d8c81364",
"54c99776-2646-4c17-82cc-850c127a8bcb",
"565a4877-f2b6-4b6b-92c9-7a52530d43ff",
"5b16bada-62a1-4d85-b5c8-d558b1d00360",
"8fb9339c-c7a0-49ef-8cd1-058a87a351ad",
"d0aba66b-62cd-4c5c-bfba-69f002e94493"
"2a195d07-ad0a-415c-9d36-f1a6d367c616",
"32651f33-918e-4dff-a137-f0b4196286a1",
"42fba42a-d020-4f89-aad3-e9e21840d25b",
"4fe15617-cab8-469d-a25e-34a981e7c4c5",
"6678cf2a-8292-4943-8955-75d7df8f9307",
"8ec129d3-3721-4449-af97-4355cc19dd80"
],
"orphan_fresh": [
"attachments/14ebe128-13d8-498b-ac58-34e85cdde467",
"attachments/c1fc3f2b-a23e-47e3-93ce-574df23d3771"
"attachments/4b21266b-8dfc-4c67-9ffc-11575d794795",
"attachments/fa6c3d18-8e06-4b14-8a7d-acd5e1671890"
],
"orphan_stale": [
"attachments/7afa0148-2cd2-49a8-b81f-515991cd9823",
"attachments/d004f135-010c-48ac-853e-dd3871aac719",
"attachments/fbed5e6c-0c1a-4f22-980f-cf419963ddc5"
"attachments/57198272-dd0d-46d6-97f0-e68295a5e65e",
"attachments/aeebb4d7-95f8-4ebd-ac86-1031bc42e469",
"attachments/fc854d9e-6be4-4b3a-a676-d2dcf9be5993"
],
"pending_fresh": [
"acd92195-baee-4647-ba55-e9258240c898",
"be34c7f5-2c8b-4e95-b7f1-5a949ffb633e"
"80259edc-7cb8-4181-9878-f642a2b34d71",
"a3e9efb4-3de5-43a8-8d18-296bf48cc4f8"
]
},
"cost": {
"seconds": 0.032,
"seconds": 0.031,
"requests": {
"s3_list": 1,
"s3_head": 0,
@@ -79,12 +79,13 @@
},
"waited_seconds": {
"s3_list": 0.015,
"sql": 0.004
"sql": 0.003
},
"hashed_bytes": 1453621,
"cpu_seconds": 0.453,
"max_rss_mib": 80.4
"cpu_seconds": 0.459,
"max_rss_mib": 84.9
},
"exit_code": 0,
"wall_seconds": 0.828
"wall_seconds": 0.854,
"audit": []
}
+24 -23
View File
@@ -6,7 +6,7 @@
"deep": false,
"page_size": 1000,
"scanned": {
"objects": 19,
"objects": 20,
"rows": 21
},
"found": {
@@ -31,45 +31,46 @@
"rechecked": {},
"errors": 0,
"samples": {
"healed": [
"01a18ae1-398b-70fa-97ec-8bd9575df28c"
],
"quarantined": [
"01a18ae1-39be-7126-bee7-b8d03ea89876",
"01a18ae1-39cf-70c4-87ec-8a0e01854664",
"01a18ae1-39da-71b9-ba5d-08d485329231",
"01a18ae1-3a28-7787-966a-82d0e9d03c3e",
"01a18ae1-3a31-7218-b535-13bed0785a1b",
"01a18ae1-3a39-70f1-85a1-e44886ad2399",
"01a18ae1-3a42-720c-9027-57607a41b3fb",
"01a18ae1-3a4b-7278-8c5f-ee9734008ac2"
"01a18b6d-7f95-7491-92e4-8ecd4a5edd55",
"01a18b6d-7fc2-73ac-839f-0107ae03f9a6",
"01a18b6d-7fcb-759d-913d-211ccaa962ad",
"01a18b6d-7fd4-77f4-98c3-ebe387a28aea",
"01a18b6d-7f6a-7296-81fd-fb883beb9b39",
"01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1",
"01a18b6d-7fa5-75f1-accf-49b1e660d865",
"01a18b6d-7fdc-74b5-85bf-4a96931cd5d5"
],
"orphan_fresh": [
"attachments/14ebe128-13d8-498b-ac58-34e85cdde467",
"attachments/c1fc3f2b-a23e-47e3-93ce-574df23d3771"
"attachments/4b21266b-8dfc-4c67-9ffc-11575d794795",
"attachments/fa6c3d18-8e06-4b14-8a7d-acd5e1671890"
],
"pending_fresh": [
"acd92195-baee-4647-ba55-e9258240c898",
"be34c7f5-2c8b-4e95-b7f1-5a949ffb633e"
"80259edc-7cb8-4181-9878-f642a2b34d71",
"a3e9efb4-3de5-43a8-8d18-296bf48cc4f8"
],
"healed": [
"01a18b6d-7f51-7100-9ff4-889e89caccd9"
]
},
"cost": {
"seconds": 0.026,
"seconds": 0.029,
"requests": {
"s3_list": 1,
"s3_head": 0,
"s3_get": 0,
"s3_get": 5,
"s3_delete": 0,
"sql": 2
},
"waited_seconds": {
"s3_list": 0.015,
"sql": 0.004
"sql": 0.003
},
"hashed_bytes": 0,
"cpu_seconds": 0.453,
"max_rss_mib": 80.2
"hashed_bytes": 559085,
"cpu_seconds": 0.457,
"max_rss_mib": 84.7
},
"exit_code": 0,
"wall_seconds": 0.825
"wall_seconds": 0.861,
"audit": []
}
+3 -2
View File
@@ -43,8 +43,9 @@
"waited_seconds": {},
"hashed_bytes": 0,
"cpu_seconds": 0.441,
"max_rss_mib": 79.7
"max_rss_mib": 84.3
},
"exit_code": 0,
"wall_seconds": 0.791
"wall_seconds": 0.828,
"audit": []
}
+1 -1
View File
@@ -1 +1 @@
{"01a18add-79c3-7617-aeb9-47ac191fde74": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-398b-70fa-97ec-8bd9575df28c": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-39be-7126-bee7-b8d03ea89876": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-39cf-70c4-87ec-8a0e01854664": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-39da-71b9-ba5d-08d485329231": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a0d-72ee-898c-647c03e98798": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a1f-7044-82a6-1fac3ffdeebc": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a28-7787-966a-82d0e9d03c3e": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a31-7218-b535-13bed0785a1b": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a39-70f1-85a1-e44886ad2399": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a42-720c-9027-57607a41b3fb": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a4b-7278-8c5f-ee9734008ac2": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a7f-755c-9469-4759be280984": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a90-73db-85d8-62c6cbcc243e": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3a9b-71e0-a38a-24e66b67b0d6": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3aa3-74ca-9944-8cf410640874": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3aad-760c-82ec-55dd5666efcc": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3ab6-7148-8cbb-1b9430fcc792": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18ae1-3abd-74ac-a6d8-32e9623f009a": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75"}
{"01a18b6d-7a2d-74b8-8054-34d1c7ffafb2": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7f51-7100-9ff4-889e89caccd9": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7f6a-7296-81fd-fb883beb9b39": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7f95-7491-92e4-8ecd4a5edd55": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7fa5-75f1-accf-49b1e660d865": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7faf-755c-8611-45ce0959e2d8": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7fb8-736c-82d1-6b8705f0d8d5": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7fc2-73ac-839f-0107ae03f9a6": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7fcb-759d-913d-211ccaa962ad": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7fd4-77f4-98c3-ebe387a28aea": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7fdc-74b5-85bf-4a96931cd5d5": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7fe5-7056-a2f5-7cdf0aa614f1": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7fed-7311-a61b-856f592f4c15": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7ff5-7198-8cfe-1a3a519a068d": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-7ffd-753e-b7be-6d2e414ebabf": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-8004-770d-ac29-2f71c77fb811": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-800c-731c-a6ea-b344156734ac": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-8015-7041-990b-5a1740e06fb8": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75", "01a18b6d-801e-7651-85ba-1ae91663d2cd": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75"}
File diff suppressed because it is too large Load Diff
+10 -10
View File
@@ -1,12 +1,12 @@
{
"01a18add-79c3-7617-aeb9-47ac191fde74": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3a0d-72ee-898c-647c03e98798": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3a1f-7044-82a6-1fac3ffdeebc": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3a7f-755c-9469-4759be280984": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3a90-73db-85d8-62c6cbcc243e": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3a9b-71e0-a38a-24e66b67b0d6": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3aa3-74ca-9944-8cf410640874": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3aad-760c-82ec-55dd5666efcc": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3ab6-7148-8cbb-1b9430fcc792": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18ae1-3abd-74ac-a6d8-32e9623f009a": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75"
"01a18b6d-7a2d-74b8-8054-34d1c7ffafb2": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-7faf-755c-8611-45ce0959e2d8": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-7fb8-736c-82d1-6b8705f0d8d5": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-7fed-7311-a61b-856f592f4c15": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-7ff5-7198-8cfe-1a3a519a068d": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-7ffd-753e-b7be-6d2e414ebabf": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-8004-770d-ac29-2f71c77fb811": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-800c-731c-a6ea-b344156734ac": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-8015-7041-990b-5a1740e06fb8": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75",
"01a18b6d-801e-7651-85ba-1ae91663d2cd": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75"
}
+2 -2
View File
@@ -1,8 +1,8 @@
{
"exit_code": 1,
"wall_seconds": 1.633,
"wall_seconds": 2.802,
"stdout": "",
"log": [
"{\"error\":\"EndpointConnectionError\",\"event\":\"reconcile.failed\",\"level\":\"error\",\"ts\":\"2026-10-30T03:37:52.208419Z\"}"
"{\"error\":\"EndpointConnectionError\",\"event\":\"reconcile.failed\",\"level\":\"error\",\"ts\":\"2026-10-30T06:11:14.108318Z\"}"
]
}
+8
View File
@@ -0,0 +1,8 @@
{
"s3": "paused",
"state_before_stop": "running",
"init": "true",
"stop_seconds": 0.11,
"exit_code": 143,
"next_pass_skipped": false
}
+77 -75
View File
@@ -2,87 +2,86 @@
"seconds": 75,
"upload_threads": 3,
"uploads": {
"201": 3418
"201": 3454
},
"created": 3418,
"created": 3454,
"linked_every": 10,
"reconciler_runs": 75,
"reconciler_runs": 74,
"run_seconds": [
0.038,
0.036,
0.035,
0.039,
0.04,
0.043,
0.045,
0.062,
0.057,
0.059,
0.062,
0.071,
0.068,
0.08,
0.076,
0.085,
0.085,
0.086,
0.088,
0.104,
0.104,
0.104,
0.038,
0.042,
0.05,
0.05,
0.054,
0.06,
0.06,
0.069,
0.074,
0.07,
0.078,
0.09,
0.097,
0.095,
0.105,
0.102,
0.117,
0.113,
0.106,
0.111,
0.118,
0.124,
0.13,
0.129,
0.131,
0.164,
0.145,
0.139,
0.145,
0.148,
0.152,
0.122,
0.127,
0.137,
0.144,
0.146,
0.149,
0.146,
0.163,
0.154,
0.158,
0.157,
0.173,
0.168,
0.168,
0.161,
0.171,
0.177,
0.184,
0.185,
0.192,
0.188,
0.192,
0.207,
0.203,
0.217,
0.21,
0.22,
0.22,
0.213,
0.229,
0.247,
0.245,
0.247,
0.247,
0.275,
0.262,
0.27,
0.267,
0.179,
0.197,
0.191,
0.196,
0.195,
0.205,
0.215,
0.224,
0.227,
0.23,
0.23,
0.228,
0.25,
0.25,
0.251,
0.264,
0.28,
0.273,
0.285,
0.273,
0.273,
0.279,
0.283,
0.318,
0.289,
0.278,
0.308,
0.304,
0.3,
0.312,
0.309,
0.325,
0.317,
0.343,
0.335,
0.323,
0.326,
0.357,
0.359
0.316,
0.33,
0.328,
0.331,
0.332,
0.369,
0.345,
0.33,
0.342,
0.345,
0.378
],
"fixed_total": {
"orphan_stale": 5,
@@ -92,8 +91,11 @@
"pending_stale": 5
},
"fresh_seen_total": {
"orphan_fresh": 247,
"pending_fresh": 82
"orphan_fresh": 217,
"pending_fresh": 62
},
"rechecked_total": {
"missing": 46
},
"errors": 0,
"skipped": 0,
@@ -106,9 +108,9 @@
"pending_stale": 2
},
"check_after": {
"objects": 3435,
"rows": 3437,
"ready_healthy": 3429,
"objects": 3472,
"rows": 3473,
"ready_healthy": 3465,
"counts": {
"orphan_stale": 0,
"orphan_fresh": 0,