fix reproducibility

This commit is contained in:
2026-10-23 18:21:10 +03:00
parent a7f3ab11a7
commit 0ae9e07b09
21 changed files with 667 additions and 300 deletions
+6
View File
@@ -12,6 +12,12 @@ CACHE_URL_FILE=/run/secrets/cache_url
API_REPLICAS=2
# Per-replica CPU limit. Series A keeps replicas * API_CPU_LIMIT = 1.0.
API_CPU_LIMIT=0.50
# Per-replica memory. Track 1 keeps replicas * API_MEM_LIMIT = 512m.
API_MEM_LIMIT=256m
# Base profile for docker compose up. scripts/run_track1.py passes the same defaults.
HTTP_WORKERS=8
DB_POOL_SIZE=8
NGINX_WORKERS=1
# PostgreSQL cgroup CPU limit.
DB_CPU_LIMIT=1.0
# Nginx cgroup CPU limit. 0.25 saturated in the track-1 plateau runs.
+24 -20
View File
@@ -80,42 +80,46 @@ Nginx 1.28: `resolver 127.0.0.11 valid=2s`, `server api:8080 resolve`, `max_fail
## Трек 1. Цена масштабирования
Гипотеза. При суммарном CPU API = 1.0 дополнительные реплики не поднимают RPS: каждая реплика упирается в свой кусок квоты. Пока квота PostgreSQL заполнена, лишний CPU базы поднимает RPS; когда квота API и Nginx уже полные, дальнейший CPU базы даёт меньше. Пул соединений и число процессов Nginx сами по себе не добавляют пропускную способность, пока у реплики 8 потоков обработки и те же CPU-квоты.
Гипотеза. При суммарном CPU API = 1.0 и суммарной RAM API = 512 MiB дополнительные реплики не поднимают RPS: каждая реплика упирается в свой кусок квоты. Пока квота PostgreSQL заполнена, лишний CPU базы поднимает RPS; когда квота API и Nginx уже полные, дальнейший CPU базы даёт меньше.
Методика. Публичный `tests/load.py`, closed-loop `POST /messages`, `--duration 30 --concurrency 50`. Перед каждым зачётным окном отброшенный прогрев 10 с с теми же 50 воркерами. Каждый запуск создаёт свой чат. Ошибок записи во всех зачётных окнах ниже не было. `docker stats` снимался примерно раз в 3 с только во время 30 с; в выводе Docker 100% — это одно ядро. Лимиты после `up` проверены по `HostConfig.NanoCpus`.
Методика. Публичный `tests/load.py`, closed-loop `POST /messages`, `--duration 30 --concurrency 50`. Перед каждым зачётным окном отброшенный прогрев 10 с с теми же 50 воркерами. Каждый запуск создаёт свой чат. `docker stats` снимался примерно раз в 3 с только во время 30 с; в выводе Docker 100% — это одно ядро. Скрипт `scripts/run_track1.py` на каждую точку делает `docker compose down` и `docker compose up -d --build`, передаёт переменные окружения и до нагрузки сверяет `docker inspect`.
Фиксировано во всех точках серий A и B: `shared_buffers=256MB`, `synchronous_commit=off`, `max_connections=200`, память db 512 MiB, cache 0.5 CPU, Nginx `worker_processes 1` и квота lb 0.25 CPU, пул PostgreSQL 8, 8 потоков HTTP на реплику. Хост: Intel Core i5-12450H, 12 потоков, 14 GiB RAM. Скрипт прогона: `scripts/run_track1.py`. Сырые JSON лежат в `evidence/track1-*.json`.
Методика воспроизводима: суммарные ресурсы фиксировались через переменные окружения и проверялись через docker inspect. Базовые значения (8 потоков, пул 8, один процесс Nginx) восстановлены в коде по умолчанию. `HTTP_WORKERS` и `DB_POOL_SIZE` читаются процессом API, `NGINX_WORKERS` подставляется в `lb/nginx.conf.template`. Без этих переменных `docker compose up` поднимает тот же профиль 8/8/1 и 256 MiB на реплику. В сериях A и B суммарная RAM API всегда 512 MiB: 1×512m, 2×256m, 4×128m. Скрипт записывает прогон только если `HostConfig.NanoCpus` и `HostConfig.Memory` совпали с этими значениями, сумма Memory API равна 536870912, в логе каждой реплики есть `pg pool ready (8 connections)` и `http workers: 8`, а `nginx -T` показывает `worker_processes 1;`.
### Серия A. Реплики API, суммарный CPU = 1.0, CPU базы = 2.0
Фиксировано во всех точках серий A и B: `shared_buffers=256MB`, `synchronous_commit=off`, `max_connections=200`, память db 512 MiB, cache 0.5 CPU, квота lb 0.25 CPU, пул PostgreSQL 8, 8 потоков HTTP на реплику, Nginx `worker_processes 1` и `worker_connections 1024`. Хост: Intel Core i5-12450H, 12 потоков, 14 GiB RAM. Сырые JSON: `evidence/track1-A-*.json` и `evidence/track1-B-*.json`.
| Реплики | CPU на реплику | Успешный RPS | p50, мс | p95, мс | p99, мс | Ошибки |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | 1.0 | 349.994 | 121.454 | 202.572 | 1115.09 | 0 |
| 2 | 0.5 | 327.611 | 131.433 | 289.233 | 404.102 | 0 |
| 4 | 0.25 | 320.002 | 151.294 | 295.941 | 407.03 | 0 |
### Серия A. Реплики API, суммарный CPU = 1.0, суммарная RAM = 512 MiB, CPU базы = 2.0
От 1 к 2 репликам RPS −6.4%, от 1 к 4 — −8.6%. Медиана стала хуже: 121 → 131 → 151 мс. На одной реплике API держал около 90–101% (квота 1 ядро), база — до 207% (квота 2 ядра), Nginx — до 26% (квота 0.25 ядра). На четырёх репликах максимум API по контейнеру был 27.5% при квоте 0.25 ядра.
| Реплики | CPU на реплику | RAM на реплику | Успешный RPS | p50, мс | p95, мс | p99, мс | Ошибки |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | 1.0 | 512 MiB | 378.703 | 90.288 | 204.67 | 1124.75 | 21 |
| 2 | 0.5 | 256 MiB | 346.098 | 128.325 | 269.126 | 380.157 | 0 |
| 4 | 0.25 | 128 MiB | 327.486 | 102.151 | 487.734 | 626.914 | 0 |
Сумма RAM по inspect на каждой точке — 536870912 байт. NanoCpus реплик: 1000000000, 500000000 и 250000000. От 1 к 2 репликам RPS −8.6%, от 1 к 4 — −13.5%. На одной реплике API держал максимум 105% при квоте 1 ядро, база — 212% при квоте 2 ядра, Nginx — 28% при квоте 0.25 ядра. В этом окне 21 неуспешный запрос из 11649 (error ratio 0.0018); успешный RPS считается только по 11628 ответам. На четырёх репликах максимум API по контейнеру был 27.2% при квоте 0.25 ядра, ошибок не было.
![Серия A: RPS и p99 от числа реплик](evidence/api_scaling.png)
Рост числа реплик при том же суммарном CPU API не помогает. Запрос по-прежнему ограничен долей ядра на процесс, а дробление добавляет хвост задержки. p99 одной реплики (1115 мс) — один длинный хвост в том же окне, не отдельная серия.
Рост числа реплик при том же суммарном CPU и той же суммарной RAM API не помогает. Запрос по-прежнему ограничен долей ядра на процесс. p99 одной реплики (1125 мс) — длинный хвост в том же окне.
### Серия B. CPU PostgreSQL, API зафиксирован: 4 × 0.25
### Серия B. CPU PostgreSQL, API зафиксирован: 4 × 0.25 CPU и 4 × 128 MiB
| CPU базы | Успешный RPS | p50, мс | p95, мс | p99, мс | Ошибки | Кто упёрся в квоту |
| --- | --- | --- | --- | --- | --- | --- |
| 1.0 | 165.641 | 297.372 | 499.711 | 598.3 | 0 | база, max 107% |
| 2.0 | 331.742 | 112.739 | 377.522 | 459.163 | 0 | база max 207%; API max 27.6%; lb max 26.0% |
| 4.0 | 417.528 | 99.961 | 288.263 | 371.702 | 0 | API max 27.4%; lb max 26.2%; база max 242% из 400% |
| 1.0 | 178.949 | 286.653 | 480.072 | 538.062 | 0 | база, max 103% |
| 2.0 | 334.227 | 117.124 | 366.356 | 455.007 | 0 | база max 191%; API max 28.2%; lb max 25.8% |
| 4.0 | 348.996 | 110.449 | 394.75 | 479.704 | 0 | API max 27.9%; lb max 26.0%; база max 204% из 400% |
Второе ядро базы относительно первого: 331.742 / 165.641 = **+100.3% RPS**. Это подтверждённое улучшение: при 1.0 CPU база была на квоте, API (max 14%) и Nginx (max 12%) — нет. Четвёртое ядро относительно двух: 417.528 / 331.742 − 1 = **+25.9%**. База при лимите 4.0 использовала максимум около 2.4 ядра, а API и Nginx уже стояли на своих квотах. Поэтому дальнейший CPU базы помогает меньше.
На всех трёх точках inspect показал 4 контейнера API по NanoCpus 250000000 и Memory 134217728, сумма 536870912. NanoCpus базы: 1000000000, 2000000000, 4000000000.
Второе ядро базы относительно первого: 334.227 / 178.949 − 1 = **+86.8% RPS**. Это подтверждённое улучшение: при 1.0 CPU база была на квоте, API (max 19%) и Nginx (max 12%) — нет. Четвёртое ядро относительно двух: 348.996 / 334.227 − 1 = **+4.4%**. База при лимите 4.0 использовала максимум около 2.0 ядра, а API и Nginx уже стояли на своих квотах. Поэтому дальнейший CPU базы почти не поднимает RPS.
![Серия B: RPS и p99 от CPU базы](evidence/db_scaling.png)
Крестики и звёзды на графике — не точки этой серии. Это поздний парный прогон, он ниже исходных 417.528. См. следующий блок.
### Тюнинг Nginx и пула
Строки ниже сняты до параметризации: память была 256 MiB на реплику, а пул и потоки были зашиты в бинарник. Текущий `scripts/run_track1.py` эти точки не повторяет. Зачётные серии — таблицы A и B выше.
Проверялось ровно то, что предлагалось как снятие узкого места: `worker_processes auto`, `worker_connections 4096`, пул `PgPool(..., 32)`. В первом таком прогоне потоков HTTP было 8, поэтому из 32 соединений реплика одновременно занимала не больше восьми.
Сразу после серии B тот же тюнинг вместе с квотой lb 2.0 CPU дал 336.611 RPS. Повтор исходной точки серии B (пул 8, один процесс Nginx, lb 0.25, база 4.0 CPU), сделанный позже, дал уже не 417.528, а **344.846** RPS (p50 129.553, p95 307.781, p99 397.828). Абсолютные 417.528 в конце сессии не воспроизвелись: долгий прогон на одном ноутбуке смешал сравнение. Зачётный вывод по тюнингу — только парная пара, снятая подряд:
@@ -130,7 +134,7 @@ Nginx 1.28: `resolver 127.0.0.11 valid=2s`, `server api:8080 resolve`, `max_fail
Снятие потолка потоков (`workers_n` 8 → 32) при том же пуле 32, Nginx `auto`, 4 × 0.25 CPU API и 4.0 CPU базы: (273.854 − 350.038) / 350.038 = **−21.8% RPS**. p50 120.943 → 176.149 мс, p99 396.982 → 519.185 мс, ошибок 0. База при этом занята меньше (среднее 110% и максимум 179% против 130% и 204% при 8 потоках), то есть лишние соединения пула не начали работать. Квота API по-прежнему выбрана: максимум 27.6% при лимите 0.25 ядра на реплику. 32 потока делят четверть ядра и добавляют переключения и ожидание на мьютексах, а CPU, которым можно было бы держать больше запросов к базе, не появляется. Ограничение в 8 потоков не было причиной простоя пула.
Плато серии B снимается вторым ядром PostgreSQL (+100.3% от 165.641 до 331.742), а не пулом, не числом процессов Nginx и не числом потоков HTTP. Квота API в 1.0 CPU на все реплики при этом не менялась.
Плато воспроизводимой серии B снимается вторым ядром PostgreSQL (+86.8%, от 178.949 до 334.227). Квота API в 1.0 CPU на все реплики при этом не менялась. Более ранний тюнинг пула и числа процессов Nginx на незафиксированной RAM дал +1.5% и −21.8% и зачётным результатом не является.
## Ограничения и следующие шаги
@@ -138,7 +142,7 @@ Nginx 1.28: `resolver 127.0.0.11 valid=2s`, `server api:8080 resolve`, `max_fail
Nginx OSS не опрашивает `/health/ready` сам. Готовность для LB — это 502/503/504 и обрыв соединения плюс `max_fails`. Окно drain (2 с) реплика ещё может ответить 201 со своим `X-Instance-Id`.
`tests/load.py` — closed-loop, без прозрачных retry, процентили только успешных запросов. Серия трека 1 — отдельные окна по 30 с при concurrency 50; один ранний прогон на 36 с в таблице отказов к ней не относится. Поздний повтор точки «база 4.0 CPU» дал 344.846 RPS вместо 417.528, поэтому абсолютный уровень после долгой сессии плавает.
`tests/load.py` — closed-loop, без прозрачных retry, процентили только успешных запросов. Серия трека 1 — отдельные окна по 30 с при concurrency 50; один ранний прогон на 36 с в таблице отказов к ней не относится. В воспроизводимой серии A точка с одной репликой дала 21 неуспешный запрос; остальные пять окон серии A и B — без ошибок.
Сессии живут в Valkey с `appendfsync everysec`: теоретически последняя секунда записи может не попасть на диск при жёстком kill cache. `restart.sh` шлёт обычный SIGTERM, persistence-тест с этим прошёл.
+9 -2
View File
@@ -50,6 +50,8 @@ services:
CACHE_URL_FILE: /run/secrets/cache_url
IDEMPOTENCY_TTL_SECONDS: ${IDEMPOTENCY_TTL_SECONDS:-86400}
DRAIN_SECONDS: ${DRAIN_SECONDS:-2}
HTTP_WORKERS: ${HTTP_WORKERS:-8}
DB_POOL_SIZE: ${DB_POOL_SIZE:-8}
secrets:
- database_url
- cache_url
@@ -64,7 +66,7 @@ services:
resources:
limits:
cpus: ${API_CPU_LIMIT:-0.50}
memory: ${API_MEM_LIMIT:-256M}
memory: ${API_MEM_LIMIT:-256m}
cpus: ${API_CPU_LIMIT:-0.50} # для совместимости с docker compose up
mem_limit: ${API_MEM_LIMIT:-256m}
stop_grace_period: 15s
@@ -76,8 +78,13 @@ services:
condition: service_healthy
ports:
- "8080:8080"
environment:
NGINX_WORKERS: ${NGINX_WORKERS:-1}
NGINX_ENVSUBST_TEMPLATE_DIR: /etc/nginx/templates
NGINX_ENVSUBST_OUTPUT_DIR: /etc/nginx
NGINX_ENVSUBST_FILTER: "^NGINX_"
volumes:
- ./lb/nginx.conf:/etc/nginx/nginx.conf:ro
- ./lb/nginx.conf.template:/etc/nginx/templates/nginx.conf.template:ro
healthcheck:
test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1:8080/health/ready"]
interval: 5s
Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 70 KiB

+46 -25
View File
@@ -2,52 +2,73 @@
"series": "A",
"api_replicas": 1,
"api_cpu_limit": 1.0,
"api_mem_limit": "512m",
"api_mem_limit_bytes": 536870912,
"api_mem_total_bytes": 536870912,
"db_cpu_limit": 2.0,
"http_workers": 8,
"pg_pool": 8,
"nginx_worker_processes": "1",
"lb_cpu_limit": 0.25,
"warmup_seconds": 10,
"inspect": [
"/mtusi2026lab3-api-1 1000000000 map[com.docker.compose.config-hash:a0d0bfc74956001c4529e42788007cb2de76144eda4a10a4ef7be0ff88f426f0 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-1 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-cache-1 500000000 map[com.docker.compose.config-hash:a7d737a55154622d18c1f35f33d3767c8335e00fb23f230f1afca917e900252d com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:32627109abf6f741121096b45c732f758876803efd7b2e1018ebc0350d117119 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:cache com.docker.compose.version:5.5.1 org.opencontainers.image.source:https://github.com/valkey-io/valkey]",
"/mtusi2026lab3-db-1 2000000000 map[com.docker.compose.config-hash:e1447a13d70e0172bc8c2b2b96bd30e491dd7eddbc4f79f3412bd6e987477e84 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:cbf42f0edc946c0bc1b8eeed62403734e1f25dc67a3491455d07a92fdb558b28 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:db-1 com.docker.compose.service:db com.docker.compose.version:5.5.1]",
"/mtusi2026lab3-lb-1 250000000 map[com.docker.compose.config-hash:8afd1a6fe04af86c6e4c4bc0c6be3e01db2e2c692a325086cc477df1a110aebb com.docker.compose.container-number:1 com.docker.compose.depends_on:api:service_healthy:false com.docker.compose.image:sha256:09ab424a8c788f8d0fe3a64429f6d19dfa526885c8609b748d0943a75dcb9f8c com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:lb com.docker.compose.version:5.5.1 maintainer:NGINX Docker Maintainers <docker-maint@nginx.com>]"
],
"inspect": {
"verified": true,
"api": [
{
"name": "mtusi2026lab3-api-1",
"nano_cpus": 1000000000,
"memory": 536870912,
"http_workers": "8",
"db_pool_size": "8"
}
],
"api_memory_sum": 536870912,
"db": {
"name": "mtusi2026lab3-db-1",
"nano_cpus": 2000000000
},
"lb": {
"name": "mtusi2026lab3-lb-1",
"nano_cpus": 250000000,
"nginx_worker_processes": "worker_processes 1;"
}
},
"cpu": {
"lb": {
"samples": 6,
"mean_cpu_percent": 26.55,
"max_cpu_percent": 28.44
},
"api": {
"samples": 6,
"mean_cpu_percent": 90.41,
"max_cpu_percent": 100.84
"mean_cpu_percent": 97.93,
"max_cpu_percent": 105.29
},
"db": {
"samples": 6,
"mean_cpu_percent": 186.35,
"max_cpu_percent": 207.04
},
"lb": {
"samples": 6,
"mean_cpu_percent": 25.69,
"max_cpu_percent": 26.41
"mean_cpu_percent": 198.84,
"max_cpu_percent": 212.46
},
"cache": {
"samples": 6,
"mean_cpu_percent": 7.52,
"max_cpu_percent": 8.86
"mean_cpu_percent": 9.22,
"max_cpu_percent": 17.24
}
},
"load": {
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.758,
"elapsed_seconds": 30.705,
"concurrency": 50,
"successful_requests": 10765,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 349.994,
"successful_requests": 11628,
"failed_requests": 21,
"error_ratio": 0.001803,
"successful_rps": 378.703,
"successful_latency_ms": {
"p50": 121.454,
"p95": 202.572,
"p99": 1115.09
"p50": 90.288,
"p95": 204.67,
"p99": 1124.75
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
@@ -2,16 +2,16 @@
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.758,
"elapsed_seconds": 30.705,
"concurrency": 50,
"successful_requests": 10765,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 349.994,
"successful_requests": 11628,
"failed_requests": 21,
"error_ratio": 0.001803,
"successful_rps": 378.703,
"successful_latency_ms": {
"p50": 121.454,
"p95": 202.572,
"p99": 1115.09
"p50": 90.288,
"p95": 204.67,
"p99": 1124.75
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+51 -24
View File
@@ -2,53 +2,80 @@
"series": "A",
"api_replicas": 2,
"api_cpu_limit": 0.5,
"api_mem_limit": "256m",
"api_mem_limit_bytes": 268435456,
"api_mem_total_bytes": 536870912,
"db_cpu_limit": 2.0,
"http_workers": 8,
"pg_pool": 8,
"nginx_worker_processes": "1",
"lb_cpu_limit": 0.25,
"warmup_seconds": 10,
"inspect": [
"/mtusi2026lab3-api-1 500000000 map[com.docker.compose.config-hash:9ba43bae4b038b5ebe586704e952ebccfab2ccc1d385ed1b26af3b79f702ddbd com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-1 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-2 500000000 map[com.docker.compose.config-hash:9ba43bae4b038b5ebe586704e952ebccfab2ccc1d385ed1b26af3b79f702ddbd com.docker.compose.container-number:2 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-cache-1 500000000 map[com.docker.compose.config-hash:a7d737a55154622d18c1f35f33d3767c8335e00fb23f230f1afca917e900252d com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:32627109abf6f741121096b45c732f758876803efd7b2e1018ebc0350d117119 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:cache com.docker.compose.version:5.5.1 org.opencontainers.image.source:https://github.com/valkey-io/valkey]",
"/mtusi2026lab3-db-1 2000000000 map[com.docker.compose.config-hash:e1447a13d70e0172bc8c2b2b96bd30e491dd7eddbc4f79f3412bd6e987477e84 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:cbf42f0edc946c0bc1b8eeed62403734e1f25dc67a3491455d07a92fdb558b28 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:db-1 com.docker.compose.service:db com.docker.compose.version:5.5.1]",
"/mtusi2026lab3-lb-1 250000000 map[com.docker.compose.config-hash:8afd1a6fe04af86c6e4c4bc0c6be3e01db2e2c692a325086cc477df1a110aebb com.docker.compose.container-number:1 com.docker.compose.depends_on:api:service_healthy:false com.docker.compose.image:sha256:09ab424a8c788f8d0fe3a64429f6d19dfa526885c8609b748d0943a75dcb9f8c com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:lb com.docker.compose.version:5.5.1 maintainer:NGINX Docker Maintainers <docker-maint@nginx.com>]"
],
"inspect": {
"verified": true,
"api": [
{
"name": "mtusi2026lab3-api-1",
"nano_cpus": 500000000,
"memory": 268435456,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-2",
"nano_cpus": 500000000,
"memory": 268435456,
"http_workers": "8",
"db_pool_size": "8"
}
],
"api_memory_sum": 536870912,
"db": {
"name": "mtusi2026lab3-db-1",
"nano_cpus": 2000000000
},
"lb": {
"name": "mtusi2026lab3-lb-1",
"nano_cpus": 250000000,
"nginx_worker_processes": "worker_processes 1;"
}
},
"cpu": {
"lb": {
"samples": 6,
"mean_cpu_percent": 24.03,
"max_cpu_percent": 26.81
},
"api": {
"samples": 12,
"mean_cpu_percent": 47.38,
"max_cpu_percent": 55.02
"mean_cpu_percent": 43.88,
"max_cpu_percent": 53.96
},
"db": {
"samples": 6,
"mean_cpu_percent": 166.57,
"max_cpu_percent": 204.93
},
"lb": {
"samples": 6,
"mean_cpu_percent": 19.7,
"max_cpu_percent": 24.94
"mean_cpu_percent": 166.58,
"max_cpu_percent": 192.08
},
"cache": {
"samples": 6,
"mean_cpu_percent": 8.12,
"max_cpu_percent": 15.09
"mean_cpu_percent": 7.98,
"max_cpu_percent": 16.26
}
},
"load": {
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.161,
"elapsed_seconds": 30.231,
"concurrency": 50,
"successful_requests": 9881,
"successful_requests": 10463,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 327.611,
"successful_rps": 346.098,
"successful_latency_ms": {
"p50": 131.433,
"p95": 289.233,
"p99": 404.102
"p50": 128.325,
"p95": 269.126,
"p99": 380.157
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
@@ -2,16 +2,16 @@
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.161,
"elapsed_seconds": 30.231,
"concurrency": 50,
"successful_requests": 9881,
"successful_requests": 10463,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 327.611,
"successful_rps": 346.098,
"successful_latency_ms": {
"p50": 131.433,
"p95": 289.233,
"p99": 404.102
"p50": 128.325,
"p95": 269.126,
"p99": 380.157
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+65 -26
View File
@@ -2,55 +2,94 @@
"series": "A",
"api_replicas": 4,
"api_cpu_limit": 0.25,
"api_mem_limit": "128m",
"api_mem_limit_bytes": 134217728,
"api_mem_total_bytes": 536870912,
"db_cpu_limit": 2.0,
"http_workers": 8,
"pg_pool": 8,
"nginx_worker_processes": "1",
"lb_cpu_limit": 0.25,
"warmup_seconds": 10,
"inspect": [
"/mtusi2026lab3-api-1 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-1 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-2 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:2 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-2 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-3 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:3 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-4 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:4 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-cache-1 500000000 map[com.docker.compose.config-hash:a7d737a55154622d18c1f35f33d3767c8335e00fb23f230f1afca917e900252d com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:32627109abf6f741121096b45c732f758876803efd7b2e1018ebc0350d117119 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:cache com.docker.compose.version:5.5.1 org.opencontainers.image.source:https://github.com/valkey-io/valkey]",
"/mtusi2026lab3-db-1 2000000000 map[com.docker.compose.config-hash:e1447a13d70e0172bc8c2b2b96bd30e491dd7eddbc4f79f3412bd6e987477e84 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:cbf42f0edc946c0bc1b8eeed62403734e1f25dc67a3491455d07a92fdb558b28 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:db-1 com.docker.compose.service:db com.docker.compose.version:5.5.1]",
"/mtusi2026lab3-lb-1 250000000 map[com.docker.compose.config-hash:8afd1a6fe04af86c6e4c4bc0c6be3e01db2e2c692a325086cc477df1a110aebb com.docker.compose.container-number:1 com.docker.compose.depends_on:api:service_healthy:false com.docker.compose.image:sha256:09ab424a8c788f8d0fe3a64429f6d19dfa526885c8609b748d0943a75dcb9f8c com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:lb com.docker.compose.version:5.5.1 maintainer:NGINX Docker Maintainers <docker-maint@nginx.com>]"
],
"inspect": {
"verified": true,
"api": [
{
"name": "mtusi2026lab3-api-1",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-2",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-3",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-4",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
}
],
"api_memory_sum": 536870912,
"db": {
"name": "mtusi2026lab3-db-1",
"nano_cpus": 2000000000
},
"lb": {
"name": "mtusi2026lab3-lb-1",
"nano_cpus": 250000000,
"nginx_worker_processes": "worker_processes 1;"
}
},
"cpu": {
"lb": {
"samples": 7,
"mean_cpu_percent": 17.02,
"max_cpu_percent": 25.95
},
"api": {
"samples": 28,
"mean_cpu_percent": 18.64,
"max_cpu_percent": 27.51
"mean_cpu_percent": 20.06,
"max_cpu_percent": 27.19
},
"db": {
"samples": 7,
"mean_cpu_percent": 124.05,
"max_cpu_percent": 189.16
},
"lb": {
"samples": 7,
"mean_cpu_percent": 19.08,
"max_cpu_percent": 26.16
"mean_cpu_percent": 139.19,
"max_cpu_percent": 210.12
},
"cache": {
"samples": 7,
"mean_cpu_percent": 9.49,
"max_cpu_percent": 14.99
"mean_cpu_percent": 10.01,
"max_cpu_percent": 17.01
}
},
"load": {
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.347,
"elapsed_seconds": 30.276,
"concurrency": 50,
"successful_requests": 9711,
"successful_requests": 9915,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 320.002,
"successful_rps": 327.486,
"successful_latency_ms": {
"p50": 151.294,
"p95": 295.941,
"p99": 407.03
"p50": 102.151,
"p95": 487.734,
"p99": 626.914
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
@@ -2,16 +2,16 @@
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.347,
"elapsed_seconds": 30.276,
"concurrency": 50,
"successful_requests": 9711,
"successful_requests": 9915,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 320.002,
"successful_rps": 327.486,
"successful_latency_ms": {
"p50": 151.294,
"p95": 295.941,
"p99": 407.03
"p50": 102.151,
"p95": 487.734,
"p99": 626.914
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+65 -26
View File
@@ -2,55 +2,94 @@
"series": "B",
"api_replicas": 4,
"api_cpu_limit": 0.25,
"api_mem_limit": "128m",
"api_mem_limit_bytes": 134217728,
"api_mem_total_bytes": 536870912,
"db_cpu_limit": 1.0,
"http_workers": 8,
"pg_pool": 8,
"nginx_worker_processes": "1",
"lb_cpu_limit": 0.25,
"warmup_seconds": 10,
"inspect": [
"/mtusi2026lab3-api-1 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-1 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-2 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:2 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-2 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-3 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:3 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-3 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-4 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:4 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-4 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-cache-1 500000000 map[com.docker.compose.config-hash:a7d737a55154622d18c1f35f33d3767c8335e00fb23f230f1afca917e900252d com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:32627109abf6f741121096b45c732f758876803efd7b2e1018ebc0350d117119 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:cache com.docker.compose.version:5.5.1 org.opencontainers.image.source:https://github.com/valkey-io/valkey]",
"/mtusi2026lab3-db-1 1000000000 map[com.docker.compose.config-hash:5a9910bcb7033eb32e9c8f69ff91ca585ca4927069203321b91d33cdf049f2d2 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:cbf42f0edc946c0bc1b8eeed62403734e1f25dc67a3491455d07a92fdb558b28 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:db-1 com.docker.compose.service:db com.docker.compose.version:5.5.1]",
"/mtusi2026lab3-lb-1 250000000 map[com.docker.compose.config-hash:8afd1a6fe04af86c6e4c4bc0c6be3e01db2e2c692a325086cc477df1a110aebb com.docker.compose.container-number:1 com.docker.compose.depends_on:api:service_healthy:false com.docker.compose.image:sha256:09ab424a8c788f8d0fe3a64429f6d19dfa526885c8609b748d0943a75dcb9f8c com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:lb com.docker.compose.version:5.5.1 maintainer:NGINX Docker Maintainers <docker-maint@nginx.com>]"
],
"inspect": {
"verified": true,
"api": [
{
"name": "mtusi2026lab3-api-1",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-2",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-3",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-4",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
}
],
"api_memory_sum": 536870912,
"db": {
"name": "mtusi2026lab3-db-1",
"nano_cpus": 1000000000
},
"lb": {
"name": "mtusi2026lab3-lb-1",
"nano_cpus": 250000000,
"nginx_worker_processes": "worker_processes 1;"
}
},
"cpu": {
"lb": {
"samples": 7,
"mean_cpu_percent": 8.62,
"max_cpu_percent": 11.8
},
"api": {
"samples": 28,
"mean_cpu_percent": 9.62,
"max_cpu_percent": 14.31
"mean_cpu_percent": 11.86,
"max_cpu_percent": 19.43
},
"db": {
"samples": 7,
"mean_cpu_percent": 73.45,
"max_cpu_percent": 107.26
},
"lb": {
"samples": 7,
"mean_cpu_percent": 7.77,
"max_cpu_percent": 11.61
"mean_cpu_percent": 75.17,
"max_cpu_percent": 102.55
},
"cache": {
"samples": 7,
"mean_cpu_percent": 6.37,
"max_cpu_percent": 10.33
"mean_cpu_percent": 6.74,
"max_cpu_percent": 11.3
}
},
"load": {
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.313,
"elapsed_seconds": 30.333,
"concurrency": 50,
"successful_requests": 5021,
"successful_requests": 5428,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 165.641,
"successful_rps": 178.949,
"successful_latency_ms": {
"p50": 297.372,
"p95": 499.711,
"p99": 598.3
"p50": 286.653,
"p95": 480.072,
"p99": 538.062
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
@@ -2,16 +2,16 @@
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.313,
"elapsed_seconds": 30.333,
"concurrency": 50,
"successful_requests": 5021,
"successful_requests": 5428,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 165.641,
"successful_rps": 178.949,
"successful_latency_ms": {
"p50": 297.372,
"p95": 499.711,
"p99": 598.3
"p50": 286.653,
"p95": 480.072,
"p99": 538.062
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+65 -26
View File
@@ -2,55 +2,94 @@
"series": "B",
"api_replicas": 4,
"api_cpu_limit": 0.25,
"api_mem_limit": "128m",
"api_mem_limit_bytes": 134217728,
"api_mem_total_bytes": 536870912,
"db_cpu_limit": 2.0,
"http_workers": 8,
"pg_pool": 8,
"nginx_worker_processes": "1",
"lb_cpu_limit": 0.25,
"warmup_seconds": 10,
"inspect": [
"/mtusi2026lab3-api-1 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-1 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-2 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:2 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-2 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-3 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:3 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-3 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-4 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:4 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-4 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-cache-1 500000000 map[com.docker.compose.config-hash:a7d737a55154622d18c1f35f33d3767c8335e00fb23f230f1afca917e900252d com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:32627109abf6f741121096b45c732f758876803efd7b2e1018ebc0350d117119 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:cache com.docker.compose.version:5.5.1 org.opencontainers.image.source:https://github.com/valkey-io/valkey]",
"/mtusi2026lab3-db-1 2000000000 map[com.docker.compose.config-hash:e1447a13d70e0172bc8c2b2b96bd30e491dd7eddbc4f79f3412bd6e987477e84 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:cbf42f0edc946c0bc1b8eeed62403734e1f25dc67a3491455d07a92fdb558b28 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:db-1 com.docker.compose.service:db com.docker.compose.version:5.5.1]",
"/mtusi2026lab3-lb-1 250000000 map[com.docker.compose.config-hash:8afd1a6fe04af86c6e4c4bc0c6be3e01db2e2c692a325086cc477df1a110aebb com.docker.compose.container-number:1 com.docker.compose.depends_on:api:service_healthy:false com.docker.compose.image:sha256:09ab424a8c788f8d0fe3a64429f6d19dfa526885c8609b748d0943a75dcb9f8c com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:lb com.docker.compose.version:5.5.1 maintainer:NGINX Docker Maintainers <docker-maint@nginx.com>]"
],
"inspect": {
"verified": true,
"api": [
{
"name": "mtusi2026lab3-api-1",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-2",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-3",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-4",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
}
],
"api_memory_sum": 536870912,
"db": {
"name": "mtusi2026lab3-db-1",
"nano_cpus": 2000000000
},
"lb": {
"name": "mtusi2026lab3-lb-1",
"nano_cpus": 250000000,
"nginx_worker_processes": "worker_processes 1;"
}
},
"cpu": {
"lb": {
"samples": 7,
"mean_cpu_percent": 16.89,
"max_cpu_percent": 25.75
},
"api": {
"samples": 28,
"mean_cpu_percent": 19.77,
"max_cpu_percent": 27.63
"mean_cpu_percent": 20.09,
"max_cpu_percent": 28.18
},
"db": {
"samples": 7,
"mean_cpu_percent": 136.68,
"max_cpu_percent": 206.58
},
"lb": {
"samples": 7,
"mean_cpu_percent": 16.56,
"max_cpu_percent": 25.95
"mean_cpu_percent": 131.47,
"max_cpu_percent": 190.53
},
"cache": {
"samples": 7,
"mean_cpu_percent": 5.61,
"max_cpu_percent": 7.95
"mean_cpu_percent": 9.64,
"max_cpu_percent": 19.24
}
},
"load": {
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.391,
"elapsed_seconds": 30.491,
"concurrency": 50,
"successful_requests": 10082,
"successful_requests": 10191,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 331.742,
"successful_rps": 334.227,
"successful_latency_ms": {
"p50": 112.739,
"p95": 377.522,
"p99": 459.163
"p50": 117.124,
"p95": 366.356,
"p99": 455.007
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
@@ -2,16 +2,16 @@
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.391,
"elapsed_seconds": 30.491,
"concurrency": 50,
"successful_requests": 10082,
"successful_requests": 10191,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 331.742,
"successful_rps": 334.227,
"successful_latency_ms": {
"p50": 112.739,
"p95": 377.522,
"p99": 459.163
"p50": 117.124,
"p95": 366.356,
"p99": 455.007
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+65 -26
View File
@@ -2,55 +2,94 @@
"series": "B",
"api_replicas": 4,
"api_cpu_limit": 0.25,
"api_mem_limit": "128m",
"api_mem_limit_bytes": 134217728,
"api_mem_total_bytes": 536870912,
"db_cpu_limit": 4.0,
"http_workers": 8,
"pg_pool": 8,
"nginx_worker_processes": "1",
"lb_cpu_limit": 0.25,
"warmup_seconds": 10,
"inspect": [
"/mtusi2026lab3-api-1 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-1 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-2 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:2 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-2 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-3 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:3 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-3 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-api-4 250000000 map[com.docker.compose.config-hash:baa6c518a5d0d2ce1a2d38ddcfa206234dfa8023729d82b99b3f7264efaf99b4 com.docker.compose.container-number:4 com.docker.compose.depends_on: com.docker.compose.image:sha256:a1ec572a54fa12c0ab467a444062c9a59e7bc804b20247d743295cee651b6fdc com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:api-4 com.docker.compose.service:api com.docker.compose.version:5.5.1 org.opencontainers.image.version:25.10]",
"/mtusi2026lab3-cache-1 500000000 map[com.docker.compose.config-hash:a7d737a55154622d18c1f35f33d3767c8335e00fb23f230f1afca917e900252d com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:32627109abf6f741121096b45c732f758876803efd7b2e1018ebc0350d117119 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:cache com.docker.compose.version:5.5.1 org.opencontainers.image.source:https://github.com/valkey-io/valkey]",
"/mtusi2026lab3-db-1 4000000000 map[com.docker.compose.config-hash:00262326412a2d2388e383afd1f1c5d3453baeebf2e459788f9a579b8340f994 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:cbf42f0edc946c0bc1b8eeed62403734e1f25dc67a3491455d07a92fdb558b28 com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.replace:db-1 com.docker.compose.service:db com.docker.compose.version:5.5.1]",
"/mtusi2026lab3-lb-1 250000000 map[com.docker.compose.config-hash:8afd1a6fe04af86c6e4c4bc0c6be3e01db2e2c692a325086cc477df1a110aebb com.docker.compose.container-number:1 com.docker.compose.depends_on:api:service_healthy:false com.docker.compose.image:sha256:09ab424a8c788f8d0fe3a64429f6d19dfa526885c8609b748d0943a75dcb9f8c com.docker.compose.oneoff:False com.docker.compose.project:mtusi2026lab3 com.docker.compose.project.config_files:/home/andrey/mtusi2026lab3/compose.yaml com.docker.compose.project.environment_file:/home/andrey/mtusi2026lab3/.env com.docker.compose.project.working_dir:/home/andrey/mtusi2026lab3 com.docker.compose.service:lb com.docker.compose.version:5.5.1 maintainer:NGINX Docker Maintainers <docker-maint@nginx.com>]"
],
"inspect": {
"verified": true,
"api": [
{
"name": "mtusi2026lab3-api-1",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-2",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-3",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
},
{
"name": "mtusi2026lab3-api-4",
"nano_cpus": 250000000,
"memory": 134217728,
"http_workers": "8",
"db_pool_size": "8"
}
],
"api_memory_sum": 536870912,
"db": {
"name": "mtusi2026lab3-db-1",
"nano_cpus": 4000000000
},
"lb": {
"name": "mtusi2026lab3-lb-1",
"nano_cpus": 250000000,
"nginx_worker_processes": "worker_processes 1;"
}
},
"cpu": {
"lb": {
"samples": 7,
"mean_cpu_percent": 17.63,
"max_cpu_percent": 25.98
},
"api": {
"samples": 28,
"mean_cpu_percent": 19.49,
"max_cpu_percent": 27.41
"mean_cpu_percent": 19.88,
"max_cpu_percent": 27.94
},
"db": {
"samples": 7,
"mean_cpu_percent": 141.25,
"max_cpu_percent": 241.9
},
"lb": {
"samples": 7,
"mean_cpu_percent": 16.49,
"max_cpu_percent": 26.16
"mean_cpu_percent": 138.77,
"max_cpu_percent": 203.66
},
"cache": {
"samples": 7,
"mean_cpu_percent": 6.55,
"max_cpu_percent": 8.51
"mean_cpu_percent": 10.56,
"max_cpu_percent": 17.54
}
},
"load": {
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.223,
"elapsed_seconds": 30.31,
"concurrency": 50,
"successful_requests": 12619,
"successful_requests": 10578,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 417.528,
"successful_rps": 348.996,
"successful_latency_ms": {
"p50": 99.961,
"p95": 288.263,
"p99": 371.702
"p50": 110.449,
"p95": 394.75,
"p99": 479.704
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
@@ -2,16 +2,16 @@
"lab": 3,
"workload": "closed-loop POST messages; no automatic retries",
"requested_duration_seconds": 30.0,
"elapsed_seconds": 30.223,
"elapsed_seconds": 30.31,
"concurrency": 50,
"successful_requests": 12619,
"successful_requests": 10578,
"failed_requests": 0,
"error_ratio": 0.0,
"successful_rps": 417.528,
"successful_rps": 348.996,
"successful_latency_ms": {
"p50": 99.961,
"p95": 288.263,
"p99": 371.702
"p50": 110.449,
"p95": 394.75,
"p99": 479.704
},
"note": "Latency percentiles cover successes only. Timeouts/errors are counted separately; closed-loop has coordinated omission and client overhead."
}
+2 -2
View File
@@ -1,9 +1,9 @@
worker_processes auto;
worker_processes ${NGINX_WORKERS};
error_log /dev/stderr warn;
pid /tmp/nginx.pid;
events {
worker_connections 4096;
worker_connections 1024;
}
http {
+5 -15
View File
@@ -55,8 +55,6 @@ def dual_axis(ax, xs, rps, p99, xlabel, title):
def main():
series_a = sorted(load_series("A"), key=lambda row: row["replicas"])
series_b = sorted(load_series("B"), key=lambda row: row["db_cpu"])
repeat = load_series("R")
tuned = load_series("P")
if not series_a or not series_b:
raise SystemExit("need series A and B JSON in evidence/")
@@ -66,30 +64,22 @@ def main():
[row["replicas"] for row in series_a],
[row["rps"] for row in series_a],
[row["p99"] for row in series_a],
"API replicas (total CPU = 1.0, DB CPU = 2.0)",
"Серия A: реплики API при фиксированном суммарном CPU",
"API replicas (total CPU = 1.0, total RAM = 512 MiB, DB CPU = 2.0)",
"Серия A: реплики API при фиксированных суммарных CPU и RAM",
)
fig.tight_layout()
fig.savefig(EVIDENCE / "api_scaling.png", dpi=140)
plt.close(fig)
fig, ax = plt.subplots(figsize=(7.2, 4.4))
twin = dual_axis(
dual_axis(
ax,
[row["db_cpu"] for row in series_b],
[row["rps"] for row in series_b],
[row["p99"] for row in series_b],
"PostgreSQL CPU limit (4 API × 0.25 CPU)",
"Серия B: CPU базы и парный прогон тюнинга",
"PostgreSQL CPU limit (4 API × 0.25 CPU, total API RAM = 512 MiB)",
"Серия B: RPS и p99 от CPU базы",
)
if repeat and tuned:
base, point = repeat[-1], tuned[-1]
ax.scatter([base["db_cpu"]], [base["rps"]], marker="x", s=80, color="#1f4e79", zorder=5, label="repeat baseline RPS")
ax.scatter([point["db_cpu"] + 0.08], [point["rps"]], marker="*", s=160, color="#1f4e79", zorder=5, label="tuned RPS")
twin.scatter([base["db_cpu"]], [base["p99"]], marker="x", s=80, color="#b85c38", zorder=5, label="repeat baseline p99")
twin.scatter([point["db_cpu"] + 0.08], [point["p99"]], marker="*", s=160, color="#b85c38", zorder=5, label="tuned p99")
handles = ax.get_lines() + twin.get_lines() + list(ax.collections) + list(twin.collections)
ax.legend(handles, [h.get_label() for h in handles], loc="best", fontsize=8)
fig.tight_layout()
fig.savefig(EVIDENCE / "db_scaling.png", dpi=140)
plt.close(fig)
+207 -67
View File
@@ -1,5 +1,9 @@
#!/usr/bin/env python3
"""One measured load.py run under explicit Compose CPU limits. Prints nothing secret."""
"""Series A and B with a fixed total API RAM budget and the base 8/8/1 profile.
The profile is applied through Compose environment, then checked with docker inspect
before the load window. Nothing secret is printed.
"""
import json
import os
import subprocess
@@ -8,30 +12,66 @@ import threading
import time
import urllib.error
import urllib.request
from decimal import Decimal
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
HTTP_WORKERS = "8"
DB_POOL_SIZE = "8"
NGINX_WORKERS = "1"
LB_CPU_LIMIT = "0.25"
TOTAL_API_RAM = 512 * 1024 * 1024
def env_for(api_replicas, api_cpu, db_cpu):
def mem_bytes(spec):
text = str(spec).strip()
units = {"b": 1, "k": 1024, "m": 1024**2, "g": 1024**3}
unit = text[-1].lower()
if unit not in units:
raise SystemExit(f"bad memory spec: {spec}")
return int(text[:-1]) * units[unit]
def cpu_nanos(spec):
return int(Decimal(str(spec)) * Decimal(1_000_000_000))
def series_points():
points = []
for replicas, api_cpu in ((1, "1.0"), (2, "0.5"), (4, "0.25")):
per = TOTAL_API_RAM // replicas // (1024 * 1024)
points.append(("A", replicas, api_cpu, f"{per}m", "2.0"))
for db_cpu in ("1.0", "2.0", "4.0"):
per = TOTAL_API_RAM // 4 // (1024 * 1024)
points.append(("B", 4, "0.25", f"{per}m", db_cpu))
return points
def env_for(replicas, api_cpu, api_mem, db_cpu):
env = os.environ.copy()
env["API_REPLICAS"] = str(api_replicas)
env["API_REPLICAS"] = str(replicas)
env["API_CPU_LIMIT"] = str(api_cpu)
env["API_MEM_LIMIT"] = api_mem
env["DB_CPU_LIMIT"] = str(db_cpu)
env["HTTP_WORKERS"] = HTTP_WORKERS
env["DB_POOL_SIZE"] = DB_POOL_SIZE
env["NGINX_WORKERS"] = NGINX_WORKERS
env["LB_CPU_LIMIT"] = LB_CPU_LIMIT
return env
def compose(args, env):
def compose(args, env, capture=False):
result = subprocess.run(
["docker", "compose", "--env-file", str(ROOT / ".env"), *args],
cwd=ROOT,
env=env,
check=False,
text=True,
capture_output=True,
capture_output=capture,
)
if result.returncode != 0:
sys.stderr.write(result.stderr or result.stdout)
if result.returncode != 0 and capture:
sys.stderr.write(result.stderr or "")
sys.stderr.write(result.stdout or "")
return result
@@ -39,7 +79,7 @@ def wait_stack(env, replicas, timeout=180):
deadline = time.monotonic() + timeout
last = ""
while time.monotonic() < deadline:
ps = compose(["ps", "api", "--format", "{{.Health}}"], env)
ps = compose(["ps", "api", "--format", "{{.Health}}"], env, capture=True)
health = [line.strip() for line in ps.stdout.splitlines() if line.strip()]
ready = False
try:
@@ -54,18 +94,118 @@ def wait_stack(env, replicas, timeout=180):
raise SystemExit("stack did not become ready: " + last)
def observed_nanos(env):
ids = compose(["ps", "-q"], env).stdout.split()
found = []
for cid in ids:
info = subprocess.check_output(
["docker", "inspect", "-f", "{{.Name}} {{.HostConfig.NanoCpus}} {{.Config.Labels}}", cid],
text=True,
).strip()
found.append(info)
def container_ids(env, service):
result = compose(["ps", "-q", service], env, capture=True)
if result.returncode != 0:
raise SystemExit(f"compose ps {service} failed")
return [line for line in result.stdout.split() if line]
def inspect_container(cid):
raw = subprocess.check_output(["docker", "inspect", cid], text=True)
return json.loads(raw)[0]
def env_map(info):
found = {}
for item in info["Config"].get("Env") or []:
key, _, value = item.partition("=")
found[key] = value
return found
def verify_limits(env, replicas, api_cpu, api_mem, db_cpu):
expect_api_nanos = cpu_nanos(api_cpu)
expect_api_mem = mem_bytes(api_mem)
expect_db_nanos = cpu_nanos(db_cpu)
expect_lb_nanos = cpu_nanos(LB_CPU_LIMIT)
api_ids = container_ids(env, "api")
if len(api_ids) != replicas:
raise SystemExit(f"expected {replicas} api containers, found {len(api_ids)}")
api_rows = []
for cid in api_ids:
info = inspect_container(cid)
nanos = info["HostConfig"]["NanoCpus"]
memory = info["HostConfig"]["Memory"]
present = env_map(info)
name = info["Name"].lstrip("/")
if nanos != expect_api_nanos or memory != expect_api_mem:
raise SystemExit(
f"{name}: NanoCpus={nanos} Memory={memory}, "
f"expected {expect_api_nanos} and {expect_api_mem}"
)
if present.get("HTTP_WORKERS") != HTTP_WORKERS or present.get("DB_POOL_SIZE") != DB_POOL_SIZE:
raise SystemExit(
f"{name}: HTTP_WORKERS={present.get('HTTP_WORKERS')} "
f"DB_POOL_SIZE={present.get('DB_POOL_SIZE')}"
)
api_rows.append(
{
"name": name,
"nano_cpus": nanos,
"memory": memory,
"http_workers": present["HTTP_WORKERS"],
"db_pool_size": present["DB_POOL_SIZE"],
}
)
total_mem = sum(row["memory"] for row in api_rows)
if total_mem != TOTAL_API_RAM:
raise SystemExit(f"api RAM sum {total_mem} != {TOTAL_API_RAM}")
db_ids = container_ids(env, "db")
if len(db_ids) != 1:
raise SystemExit(f"expected 1 db container, found {len(db_ids)}")
db_info = inspect_container(db_ids[0])
db_nanos = db_info["HostConfig"]["NanoCpus"]
if db_nanos != expect_db_nanos:
raise SystemExit(f"db NanoCpus={db_nanos}, expected {expect_db_nanos}")
lb_ids = container_ids(env, "lb")
if len(lb_ids) != 1:
raise SystemExit(f"expected 1 lb container, found {len(lb_ids)}")
lb_info = inspect_container(lb_ids[0])
lb_nanos = lb_info["HostConfig"]["NanoCpus"]
lb_env = env_map(lb_info)
if lb_nanos != expect_lb_nanos or lb_env.get("NGINX_WORKERS") != NGINX_WORKERS:
raise SystemExit(
f"lb NanoCpus={lb_nanos} NGINX_WORKERS={lb_env.get('NGINX_WORKERS')}"
)
rendered = compose(["exec", "-T", "lb", "nginx", "-T"], env, capture=True)
if rendered.returncode != 0:
raise SystemExit("nginx -T failed")
directive = ""
rendered_text = (rendered.stdout or "") + "\n" + (rendered.stderr or "")
for line in rendered_text.splitlines():
stripped = line.strip()
if stripped.startswith("worker_processes"):
directive = stripped
break
expected_directive = f"worker_processes {NGINX_WORKERS};"
if directive != expected_directive:
raise SystemExit(f"nginx config has {directive!r}, expected {expected_directive!r}")
logs = compose(["logs", "api"], env, capture=True)
if logs.returncode != 0:
raise SystemExit("api logs failed")
pool_line = f"pg pool ready ({DB_POOL_SIZE} connections)"
worker_line = f"http workers: {HTTP_WORKERS}"
if logs.stdout.count(pool_line) < replicas or logs.stdout.count(worker_line) < replicas:
raise SystemExit("api logs do not show the base pool and worker profile on every replica")
return {
"verified": True,
"api": api_rows,
"api_memory_sum": total_mem,
"db": {"name": db_info["Name"].lstrip("/"), "nano_cpus": db_nanos},
"lb": {
"name": lb_info["Name"].lstrip("/"),
"nano_cpus": lb_nanos,
"nginx_worker_processes": directive,
},
}
def sample_stats(stop, bucket):
while not stop.is_set():
proc = subprocess.run(
@@ -81,9 +221,8 @@ def sample_stats(stop, bucket):
if len(parts) != 3:
continue
name, cpu, mem = parts
cpu_val = cpu.strip().rstrip("%")
try:
cpu_num = float(cpu_val)
cpu_num = float(cpu.strip().rstrip("%"))
except ValueError:
continue
rows.append({"name": name, "cpu_percent": cpu_num, "mem": mem.strip()})
@@ -113,43 +252,34 @@ def summarize_cpu(samples):
return out
def main():
if len(sys.argv) != 7:
raise SystemExit(
"usage: run_track1.py SERIES REPLICAS API_CPU DB_CPU POOL NGINX_WORKERS"
)
series, replicas, api_cpu, db_cpu, pool, nginx_workers = sys.argv[1:]
replicas = int(replicas)
env = env_for(replicas, api_cpu, db_cpu)
def run_point(series, replicas, api_cpu, api_mem, db_cpu):
env = env_for(replicas, api_cpu, api_mem, db_cpu)
label = f"track1-{series}-r{replicas}-apicpu{api_cpu}-dbcpu{db_cpu}"
raw = ROOT / "evidence" / f"{label}.load.json"
dest = ROOT / "evidence" / f"{label}.json"
up_db = None
for _ in range(3):
up_db = compose(
["up", "-d", "--no-deps", "--force-recreate", "db"],
env,
)
if up_db.returncode == 0:
break
time.sleep(2)
if up_db.returncode != 0:
sys.stderr.write(up_db.stderr)
raise SystemExit(up_db.returncode)
up_api = compose(
["up", "-d", "--no-deps", "--force-recreate", "--scale", f"api={replicas}", "api"],
print(
f"point series={series} replicas={replicas} api_cpu={api_cpu} "
f"api_mem={api_mem} db_cpu={db_cpu} http_workers={HTTP_WORKERS} "
f"db_pool={DB_POOL_SIZE} nginx_workers={NGINX_WORKERS}",
file=sys.stderr,
)
down = compose(["down"], env)
if down.returncode != 0:
raise SystemExit(down.returncode)
up = compose(
["up", "-d", "--build", "--scale", f"api={replicas}"],
env,
)
if up_api.returncode != 0:
sys.stderr.write(up_api.stderr)
raise SystemExit(up_api.returncode)
up_lb = compose(["up", "-d", "--no-deps", "lb"], env)
if up_lb.returncode != 0:
sys.stderr.write(up_lb.stderr)
raise SystemExit(up_lb.returncode)
if up.returncode != 0:
raise SystemExit(up.returncode)
wait_stack(env, replicas)
limits = observed_nanos(env)
limits = verify_limits(env, replicas, api_cpu, api_mem, db_cpu)
print(
f"inspect ok api_sum_memory={limits['api_memory_sum']} "
f"api_nano={limits['api'][0]['nano_cpus']} db_nano={limits['db']['nano_cpus']} "
f"nginx={limits['lb']['nginx_worker_processes']}",
file=sys.stderr,
)
warmup = subprocess.run(
[
@@ -203,9 +333,14 @@ def main():
"series": series,
"api_replicas": replicas,
"api_cpu_limit": float(api_cpu),
"api_mem_limit": api_mem,
"api_mem_limit_bytes": mem_bytes(api_mem),
"api_mem_total_bytes": TOTAL_API_RAM,
"db_cpu_limit": float(db_cpu),
"pg_pool": int(pool),
"nginx_worker_processes": nginx_workers,
"http_workers": int(HTTP_WORKERS),
"pg_pool": int(DB_POOL_SIZE),
"nginx_worker_processes": NGINX_WORKERS,
"lb_cpu_limit": float(LB_CPU_LIMIT),
"warmup_seconds": 10,
"inspect": limits,
"cpu": summarize_cpu(samples),
@@ -213,21 +348,26 @@ def main():
}
dest.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n")
lat = load["successful_latency_ms"]
print(
json.dumps(
{
"file": str(dest.relative_to(ROOT)),
"rps": load["successful_rps"],
"p50": lat["p50"],
"p95": lat["p95"],
"p99": lat["p99"],
"errors": load["failed_requests"],
"error_ratio": load["error_ratio"],
"cpu": report["cpu"],
},
ensure_ascii=False,
)
)
summary = {
"file": str(dest.relative_to(ROOT)),
"rps": load["successful_rps"],
"p50": lat["p50"],
"p95": lat["p95"],
"p99": lat["p99"],
"errors": load["failed_requests"],
"error_ratio": load["error_ratio"],
"api_mem_total_bytes": TOTAL_API_RAM,
"cpu": report["cpu"],
}
print(json.dumps(summary, ensure_ascii=False))
return summary
def main():
if len(sys.argv) != 1:
raise SystemExit("usage: python3 scripts/run_track1.py")
for series, replicas, api_cpu, api_mem, db_cpu in series_points():
run_point(series, replicas, api_cpu, api_mem, db_cpu)
if __name__ == "__main__":
+19 -3
View File
@@ -1192,6 +1192,20 @@ static void handle_client(int client,
::close(client);
}
int env_or_default(const char* name, int fallback) {
const char* raw = std::getenv(name);
if (raw == nullptr || raw[0] == '\0') {
return fallback;
}
char* end = nullptr;
errno = 0;
const long value = std::strtol(raw, &end, 10);
if (end == raw || *end != '\0' || errno != 0 || value < 1 || value > 4096) {
return fallback;
}
return static_cast<int>(value);
}
int main() {
const char* host_env = std::getenv("APP_HOST");
const char* port_env = std::getenv("APP_PORT");
@@ -1218,8 +1232,9 @@ int main() {
std::clog << "migrations ok\n";
}
PgPool pool(db_url, 32);
std::clog << "pg pool ready (32 connections)\n";
const int pool_n = env_or_default("DB_POOL_SIZE", 8);
PgPool pool(db_url, pool_n);
std::clog << "pg pool ready (" << pool_n << " connections)\n";
UserRepo users(pool);
ChatRepo chats(pool);
@@ -1284,7 +1299,8 @@ int main() {
std::mutex q_mu;
std::condition_variable q_cv;
std::queue<int> q;
const int workers_n = 32;
const int workers_n = env_or_default("HTTP_WORKERS", 8);
std::clog << "http workers: " << workers_n << "\n";
std::vector<std::thread> workers;
for (int i = 0; i < workers_n; ++i) {
workers.emplace_back([&]() {