Files
mtusi2026lab6/evidence/uploads/baseline/summary.md
T
ricoromaandClaude Opus 5.5 db3340d811 bench: capture uploads and queue labels with telemetry on
make bench_uploads TELEMETRY=on (label baseline) and make bench_queue
TELEMETRY=on QUEUE_LABEL=baseline4 on b3a5ab8 for the coverage
exclusions; record last-step decisions and draft report sections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 09:33:17 +03:00

60 lines
14 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Профиль загрузок: метка `baseline`
- Статус: **complete**
- Commit: `b3a5ab8`
- Профиль `uploads`: варианты «до», «без лимита», «с лимитом» (образ «до» — commit `9cc6c0c`); клиенты 1, 10, 20, 40; повторов 3; окно 20 с после 3 с разгона
- Тела: 1MiB — 1048276 байт (1024 × 341), SHA-256 `d65ca4573a4064ee…`, 10MiB — 10479107 байт (1024 × 3409), SHA-256 `a8f0ef17afaf6046…`
- Реплика 1 CPU / 512m без swap, буфер приёма `4096 131072 524288`; S3 1 CPU / 512m; LB 1 CPU / 256m. Обработка не запущена: задания ждут в PostgreSQL
- Код успеха: «до» — 202, «без лимита» — 202, «с лимитом» — 202 (столбец `s201` `runs.csv` считает код варианта, его называет `success_code`)
- Пик — `memory.peak` cgroup, сброшенный через свой дескриптор перед нагрузкой (`peak_source` = reset); без сброса — наибольшая выборка раз в 0.2 с (пометка `peak_sampled`). Память до окна — первая выборка. Ответы по кодам — за весь прогон, загрузок/с и перцентили — по загрузкам окна. Значения — медиана (min–max) повторов
- Посторонние контейнеры в VM во время замера: нет
- Телеметрия стенда включена (TELEMETRY=on): SDK реплики с настройками рабочего стенда, Collector, Prometheus, Loki и Tempo рабочего стенда; вариант «до» — образ кода без SDK, его сравнение с остальными вариантами смешанное (с телеметрией против без неё)
## Ячейки
«Пик старта» — `memory.peak` реплики до сброса (с её запуска: фиктивный хэш пароля, разогревающая загрузка); пик окна его не включает.
| Вариант | Тело | C | Пик старта, MiB | До окна, MiB | Пик, MiB | anon, MiB | sock, MiB | OOM | Успех | 429 | 5xx | ошибка соединения | Загрузок/с | p50, мс | p95, мс | S3 пик, MiB | S3 anon, MiB | LB пик, MiB | Пометки |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| до | 10MiB | 1 | 154.6 (154.3–154.8) | 106.3 (106.1–106.8) | 122.3 (122.1–122.4) | 112.6 (111.9–112.6) | 4.0 (4.0–4.1) | 0/3 | 262 (252–264) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 11.30 (10.85–11.50) | 81 (81–83) | 110 (108–138) | 517.6 (516.8–518.5) | 238.1 (236.5–244.7) | 28.3 (27.8–30.4) | — |
| до | 10MiB | 10 | 154.4 (153.9–154.5) | 106.7 (106.7–107.5) | 171.6 (171.0–171.8) | 145.7 (145.4–145.9) | 17.8 (17.3–18.8) | 0/3 | 259 (233–266) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 10.85 (9.80–11.20) | 885 (872–925) | 1232 (1160–1332) | 517.3 (515.5–517.4) | 269.0 (259.6–272.2) | 33.8 (33.6–34.7) | — |
| до | 10MiB | 20 | 154.6 (154.5–154.7) | 106.5 (106.2–106.8) | 179.8 (179.2–180.0) | 148.8 (148.0–149.2) | 21.9 (20.5–23.9) | 0/3 | 267 (258–273) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 10.80 (10.40–11.25) | 1852 (1798–1982) | 2096 (2016–2225) | 517.4 (516.3–519.4) | 269.2 (261.1–283.2) | 39.8 (39.7–40.2) | — |
| до | 10MiB | 40 | 154.7 (154.7–154.9) | 106.4 (106.3–107.4) | 184.1 (183.2–184.4) | 153.1 (153.0–153.8) | 19.8 (18.1–21.8) | 0/3 | 259 (258–259) | 460 (455–460) | 0 (0–0) | 0 (0–0) | 10.50 (10.45–10.70) | 1894 (1864–1952) | 2239 (2196–2246) | 518.0 (517.5–518.3) | 260.6 (253.3–269.4) | 50.0 (49.7–50.2) | — |
| до | 1MiB | 1 | 154.4 (154.4–154.6) | 106.4 (106.3–107.3) | 110.8 (110.7–111.0) | 102.5 (102.5–102.8) | 0.9 (0.9–0.9) | 0/3 | 1549 (1521–1570) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 67.25 (66.15–68.40) | 14 (14–14) | 16 (16–16) | 513.0 (512.9–513.0) | 228.0 (186.3–230.3) | 28.5 (28.4–28.7) | — |
| до | 1MiB | 10 | 154.4 (154.3–154.5) | 106.8 (106.3–107.2) | 121.3 (121.2–121.4) | 108.5 (108.5–108.7) | 4.5 (4.3–4.9) | 0/3 | 2735 (2730–2780) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 119.55 (118.40–120.50) | 82 (80–83) | 114 (110–116) | 514.9 (514.8–515.0) | 224.2 (217.9–236.8) | 32.5 (31.4–34.5) | — |
| до | 1MiB | 20 | 154.2 (154.0–154.8) | 106.9 (106.3–107.0) | 129.6 (129.0–131.1) | 111.6 (111.0–112.3) | 9.7 (9.7–9.8) | 0/3 | 2767 (2713–2857) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 119.80 (116.90–124.70) | 162 (158–164) | 197 (192–228) | 515.2 (514.7–515.3) | 228.5 (225.7–229.2) | 35.5 (35.1–35.5) | — |
| до | 1MiB | 40 | 154.2 (154.2–154.7) | 106.6 (106.1–107.1) | 134.9 (134.4–135.1) | 116.3 (116.2–117.3) | 10.0 (9.7–10.7) | 0/3 | 2819 (2743–2834) | 460 (460–460) | 0 (0–0) | 0 (0–0) | 122.90 (118.65–123.85) | 161 (160–161) | 193 (191–220) | 514.9 (514.8–515.1) | 223.5 (221.6–232.8) | 50.4 (50.3–50.5) | — |
| без лимита | 10MiB | 1 | 162.6 (162.6–162.6) | 114.8 (114.6–115.0) | 131.8 (131.7–131.9) | 120.9 (120.8–121.1) | 4.1 (4.0–4.1) | 0/3 | 255 (250–262) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 11.15 (10.85–11.45) | 84 (83–85) | 110 (109–136) | 516.2 (515.5–518.3) | 239.1 (235.1–240.5) | 29.8 (28.1–30.0) | — |
| без лимита | 10MiB | 10 | 162.4 (162.1–164.1) | 114.9 (114.8–117.3) | 238.0 (234.3–239.0) | 213.5 (213.3–215.8) | 14.5 (12.6–15.7) | 0/3 | 209 (209–210) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 8.90 (8.85–9.00) | 1093 (1059–1104) | 1523 (1520–1595) | 518.3 (516.8–519.0) | 332.8 (329.8–364.9) | 29.8 (28.0–32.2) | — |
| без лимита | 10MiB | 20 | 162.4 (162.1–164.4) | 115.0 (114.9–116.9) | 343.9 (341.9–344.1) | 316.5 (316.3–318.3) | 14.7 (13.5–19.2) | 0/3 | 224 (215–225) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 9.35 (8.85–9.45) | 2142 (2080–2210) | 2593 (2580–2700) | 520.0 (515.9–521.3) | 342.9 (340.3–360.0) | 35.5 (34.5–38.6) | — |
| без лимита | 10MiB | 40 | 162.9 (162.4–164.3) | 115.5 (115.4–116.6) | 509.1 (506.9–509.2) | 495.2 (490.1–496.3) | 13.4 (12.9–17.4) | 3/3 | 0 (0–0) | 0 (0–0) | 844 (840–880) | 0 (0–0) | 0.00 (0.00–0.00) | — | — | 184.4 (166.1–196.6) | 163.1 (150.5–171.1) | 50.7 (50.3–50.9) | oom peak_sampled |
| без лимита | 1MiB | 1 | 162.7 (162.6–162.8) | 114.8 (114.6–115.6) | 122.2 (122.2–122.7) | 113.4 (112.9–113.6) | 0.9 (0.9–0.9) | 0/3 | 1446 (1438–1478) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 62.80 (62.50–64.25) | 15 (15–15) | 19 (18–19) | 513.0 (513.0–513.0) | 217.4 (187.9–220.9) | 28.4 (27.5–30.4) | — |
| без лимита | 1MiB | 10 | 162.7 (162.4–162.8) | 115.1 (114.8–115.1) | 139.2 (139.1–139.7) | 125.2 (125.0–125.6) | 4.2 (3.7–4.4) | 0/3 | 2778 (2736–2798) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 120.80 (119.35–121.90) | 82 (80–82) | 119 (114–127) | 517.5 (517.1–518.2) | 226.9 (224.0–233.1) | 33.5 (28.7–34.0) | — |
| без лимита | 1MiB | 20 | 162.3 (162.2–162.5) | 115.3 (114.7–115.3) | 148.3 (148.2–148.7) | 136.0 (135.8–137.6) | 6.5 (4.8–6.6) | 0/3 | 2815 (2804–2843) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 122.35 (122.20–123.85) | 161 (160–161) | 204 (202–204) | 516.5 (515.6–516.8) | 224.2 (219.9–224.7) | 35.1 (34.8–35.4) | — |
| без лимита | 1MiB | 40 | 162.7 (162.2–162.8) | 114.8 (114.7–115.2) | 167.6 (166.8–168.5) | 154.4 (154.4–155.2) | 10.4 (9.1–12.9) | 0/3 | 2776 (2758–2835) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 120.30 (119.35–123.05) | 318 (312–321) | 490 (472–495) | 516.4 (515.1–520.5) | 227.8 (224.6–231.2) | 50.1 (49.9–50.4) | — |
| с лимитом | 10MiB | 1 | 164.0 (162.4–164.4) | 116.7 (115.3–116.8) | 133.4 (131.5–133.6) | 122.9 (121.0–123.0) | 4.1 (4.0–4.1) | 0/3 | 255 (252–255) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 11.00 (10.90–11.10) | 85 (83–85) | 115 (109–131) | 518.1 (516.3–518.2) | 238.6 (238.0–243.4) | 28.0 (27.9–28.2) | — |
| с лимитом | 10MiB | 10 | 164.3 (162.8–164.6) | 116.1 (114.3–116.8) | 182.6 (180.5–183.2) | 156.1 (154.7–156.3) | 15.8 (14.8–18.0) | 0/3 | 249 (246–264) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 10.40 (10.40–11.15) | 906 (880–921) | 1286 (1116–1313) | 521.5 (521.3–521.9) | 271.9 (267.5–275.1) | 33.8 (32.8–35.1) | — |
| с лимитом | 10MiB | 20 | 162.8 (162.3–164.5) | 115.4 (114.5–116.6) | 189.4 (189.2–190.6) | 158.0 (158.0–159.4) | 21.7 (20.9–22.5) | 0/3 | 258 (245–269) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 10.50 (10.00–10.95) | 1886 (1818–2013) | 2271 (2173–2292) | 517.5 (515.4–520.1) | 265.6 (265.1–273.4) | 41.6 (41.6–43.3) | — |
| с лимитом | 10MiB | 40 | 162.5 (162.3–162.7) | 114.8 (114.7–114.8) | 193.6 (193.4–194.1) | 162.0 (161.9–163.2) | 23.1 (20.1–23.8) | 0/3 | 256 (249–267) | 450 (450–454) | 0 (0–0) | 0 (0–0) | 10.40 (10.25–11.00) | 1908 (1794–1973) | 2216 (2097–2320) | 520.3 (516.6–520.3) | 262.9 (251.0–294.8) | 50.1 (49.7–50.3) | — |
| с лимитом | 1MiB | 1 | 162.6 (162.5–162.8) | 115.0 (114.9–115.3) | 122.2 (121.7–122.3) | 113.1 (112.6–113.2) | 0.9 (0.5–0.9) | 0/3 | 1447 (1395–1448) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 62.95 (60.40–63.10) | 15 (15–15) | 18 (18–20) | 513.0 (513.0–513.0) | 228.6 (225.4–229.3) | 28.2 (28.2–28.2) | — |
| с лимитом | 1MiB | 10 | 162.7 (162.5–164.2) | 115.6 (114.6–117.2) | 133.5 (133.1–135.0) | 119.6 (119.5–121.7) | 4.5 (4.2–5.2) | 0/3 | 2510 (2501–2519) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 108.65 (108.40–109.10) | 89 (86–89) | 124 (123–143) | 515.1 (514.8–515.1) | 220.5 (219.4–228.6) | 32.0 (30.8–32.8) | — |
| с лимитом | 1MiB | 20 | 162.7 (162.4–164.2) | 115.3 (115.3–117.0) | 142.1 (141.7–143.4) | 123.2 (122.9–124.9) | 9.5 (9.0–10.1) | 0/3 | 2565 (2527–2595) | 0 (0–0) | 0 (0–0) | 0 (0–0) | 110.65 (109.25–112.50) | 176 (175–179) | 218 (212–234) | 515.2 (514.9–515.3) | 224.2 (222.0–224.8) | 35.9 (35.0–36.0) | — |
| с лимитом | 1MiB | 40 | 162.5 (162.4–162.6) | 115.1 (114.6–115.3) | 147.1 (146.9–147.2) | 127.8 (127.3–128.5) | 10.0 (9.2–10.1) | 0/3 | 2487 (2401–2576) | 460 (460–460) | 0 (0–0) | 0 (0–0) | 107.30 (104.05–112.20) | 183 (175–185) | 229 (222–249) | 514.9 (514.9–515.0) | 225.5 (222.1–230.7) | 52.3 (50.0–54.7) | — |
## Сценарии спеков
- `attachments`, «Одна копия тела при записи в S3» (C = 10, 10MiB): рост пика над памятью до окна — «без лимита» — 1.21 тела на загрузку; «до» — 0.64 тела на загрузку (порог «без лимита» — 1,5 тела; «до» — справочно: что у кода до `upload-memory` больше, показано в отчёте лабы 4).
- `attachments`, «Память не растёт с числом загрузок» («с лимитом», 10MiB): пик реплики C=10 182.6, C=20 189.4, C=40 193.6 MiB (сверх C=10, MiB на клиента: C=20 0.68, C=40 0.37), sock C=10 15.8, C=20 21.7, C=40 23.1 MiB, LB C=10 33.8, C=20 41.6, C=40 50.1 MiB; OOM 0, 5xx 0, 429 1354.
- `load-balancing`, «Буферы соединений ограничены» («с лимитом», 10MiB, C = 1 → 40): пик LB (50.1 − 28.0) / 39 = 0.57 MiB на клиента; sock (23.1 − 4.1) / 39 = 0.49 MiB на клиента; сумма 1.05 (порог 1 MiB на клиента), OOM при C = 40 0/3; выполнен — нет.
## Гипотезы
Гипотезы сформулированы в лабе 4 (design change `upload-memory`, `openspec/changes/archive/2026-10-22-upload-memory/design.md`); здесь — только их проверка числами ячеек тела 10MiB, лимит реплики 512 MiB. Часть H2 про «до» ждёт код без мест загрузки: если они есть в commit «до» из шапки, её итог о коде до `upload-memory` ничего не говорит.
| Гипотеза | Утверждение | Числа | Выполнена |
|---|---|---|---|
| H1 | «с лимитом»: пик при 10 MiB и C = 10, 20, 40 ≤ половины лимита; сверх C = 10 — ≤ 1 MiB на клиента; OOM и 5xx нет, лишние — 429 | пик, MiB: C=10 182.6, C=20 189.4, C=40 193.6 (половина лимита 256); сверх C=10, MiB на клиента: C=20 0.68, C=40 0.37; OOM 0, 5xx 0, 429 1354 | да |
| H2 | «без лимита»: пик растёт на ~1,2–1,3 тела на клиента, 40 × 10 MiB — OOM; «до» — ~2,1 тела и OOM при 20 × 10 MiB | без лимита C=10: 1.21 тела/клиента, OOM 0/3; без лимита C=20: 1.14 тела/клиента, OOM 0/3; без лимита C=40: — тела/клиента, OOM 3/3; до C=10: 0.64 тела/клиента, OOM 0/3; до C=20: 0.37 тела/клиента, OOM 0/3; до C=40: 0.19 тела/клиента, OOM 0/3 | нет |
| H3 | успешных загрузок в секунду при C ≥ 10: «с лимитом» ≥ 90 % «без лимита» на тех же C, пока тот жив | 10MiB C=10: 10.40 / 8.90 = 117%; 10MiB C=20: 10.50 / 9.35 = 112%; 1MiB C=10: 108.65 / 120.80 = 90%; 1MiB C=20: 110.65 / 122.35 = 90%; 1MiB C=40: 107.30 / 120.30 = 89% | нет |
| H4 | 503 из-за срока записи в S3 нет ни в одном варианте, пока реплика жива | 503 в 69 прогонах без OOM: 0 | да |