101 Commits
Author SHA1 Message Date
Artyom cc7f58c4b0 Merge completed lab 3 for Gitea submission
Preserve the exact lab 3 tree and original history from ba78cf6da4.
Connect it to the instructor master for a cross-fork pull request.
2026-11-08 00:57:04 +03:00
Artyom ba78cf6da4 Validate lab 3 release with accepted load evidence 2026-11-06 11:52:28 +03:00
Artyom b063c8780e Finalize lab 3 report after review
Separate pre-review measurements from offline validation of the session, proxy and input fixes. Record remaining minor limitations and the final validation handoff without changing code, configuration or raw evidence.
2026-11-06 09:03:35 +03:00
Artyom 8c9ec653ed Archive the non-ASCII Basic rejection change 2026-11-06 02:20:30 +03:00
Artyom 7e19d4e33e Reject non-ASCII Basic payloads before decoding 2026-11-06 02:18:38 +03:00
Artyom f7bfd1d473 Propose non-ASCII Basic rejection as an openspec change 2026-11-06 02:17:03 +03:00
Artyom d60e29221a Archive the reject NUL text inputs change 2026-11-06 02:07:29 +03:00
Artyom e66e2d7d68 Reject NUL input before authentication and text persistence 2026-11-06 02:05:09 +03:00
Artyom 5c561eac26 Propose rejecting NUL text inputs as an openspec change 2026-11-06 02:02:16 +03:00
Artyom 82641cbd3d Archive the lab 3 session cache and proxy review fixes 2026-11-06 01:51:42 +03:00
Artyom 2af24d7762 Prevent revoked session refills and escape proxy error identifiers 2026-11-06 01:49:41 +03:00
Artyom da2f392442 Propose session cache and load balancer review fixes 2026-11-06 01:46:17 +03:00
Artyom 4513d90e48 Finalize lab 3 report and evidence provenance
Tie the grade 5 claim to the final SQLAlchemy revalidation and measured scaling results. Clarify clean-clone setup, failure timings, historical evidence, and operational constraints; record report-stage checks and evidence audits.
2026-11-06 01:29:44 +03:00
Artyom f591fa64e5 Document SQLAlchemy scaling evidence and archive completed changes 2026-11-06 01:21:26 +03:00
Artyom 55b99a24a4 Preserve failed scaling setup logs with revision metadata 2026-11-06 01:15:19 +03:00
Artyom 3fb741e6a8 Clarify scaling plots and show CPU repeat ranges 2026-11-06 00:58:21 +03:00
Artyom 07965a2fb6 Alternate fixed-rate scaling configurations across repeats 2026-11-06 00:43:04 +03:00
Artyom 57744c532c Cache the immutable message statement and validate scaling evidence 2026-11-05 23:59:54 +03:00
Artyom 05d69ba6ac Preserve scaling data and reject contaminated benchmark attempts 2026-11-05 22:42:51 +03:00
Artyom 74a39fa93e Propose remeasuring SQLAlchemy scaling as an OpenSpec change 2026-11-05 22:40:04 +03:00
Artyom 5c8f605cd3 Archive grade 4 verification and record SQLAlchemy failure evidence 2026-11-05 22:33:29 +03:00
Artyom 252e3e3ffe Measure successful recovery and verify history under load 2026-11-05 22:15:18 +03:00
Artyom 8a7e027925 Propose verifying grade 4 recovery as an OpenSpec change 2026-11-05 22:13:16 +03:00
Artyom ef0b9a385e Archive pool recovery and record current grade 3 evidence 2026-11-05 22:08:14 +03:00
Artyom 0a1f48bd73 Recover stale PostgreSQL connections before starting requests 2026-11-05 21:55:14 +03:00
Artyom 18de595740 Propose restoring PostgreSQL pool recovery as an OpenSpec change 2026-11-05 21:51:20 +03:00
Artyom 7fc911c332 Document SQLAlchemy migration checks and remaining evidence work 2026-11-05 21:41:47 +03:00
Artyom 66c614426c Reject legacy collation and persistence drift before adoption 2026-11-05 21:41:01 +03:00
Artyom 71c643c6d1 Adopt async SQLAlchemy storage and verified Alembic migrations
Keep the message and idempotency receipt in one Core CTE statement with chat-first locking. Preserve rollback, bounded dependency failures, and domain ports.

Replace the SQL runner with Alembic. Adopt legacy databases only after validating their journal and PostgreSQL schema, retain all data, and verify migrations on fresh and populated databases.
2026-11-05 21:38:12 +03:00
ArtyomandClaude Code c376882ce8 Match the tampered migration scenario name in the Alembic delta
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-11-05 00:47:10 +03:00
ArtyomandClaude Code c5f39dea25 Propose adopting SQLAlchemy and Alembic for the lab 3 storage
User requirement (2026-11-05, labs 3+): all PostgreSQL access goes through
SQLAlchemy 2.x async and migrations run through Alembic. The proposal keeps
every port, zone boundary and observable API behavior in place, moves the
checksum-based migration journal scenario to Alembic revision semantics and
pins the evidence that depends on SQL behavior for regeneration.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-11-05 00:46:59 +03:00
ArtyomandClaude Opus 5.5 72a1e57467 Archive the second lab 3 review gaps change
The deltas are synced to openspec/specs: a request that outlives the
graceful shutdown gets 503 unavailable in the common error format with
X-Request-Id and X-Instance-Id instead of the bare 500; a lost answer is
retried with the same key both on a live replica and after the replica
holding the request was killed, with one copy and the same id; make up
returns once every running replica is UP in the LB.

Before archiving, on e049c26 from a reset stand: make test, both
failover runs (first write after 2.858 and 2.682 s, 0 failed attempts),
persistence, retry-check, faults and make level4 pass, the clean clone
run follows the launch block verbatim; make check, make lint, 243 own
tests and openspec validate --all --strict pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 06:00:40 +03:00
ArtyomandClaude Opus 5.5 472ce5b9ea Report the second review fixes and the new records
The header names the code and harness commits of the second review
cycle and states that every level 3 and 4 record and the clean clone
run are on e049c26. Row 4.1 now carries the detection ranges and the
SIGTERM window of the current records, the 503 answer to a request
cancelled at shutdown replaces the 500 everywhere, and the launch block
loses the manual LB wait now done by make up and gains retry-check and
storage-evidence, so the clean clone run follows it verbatim. Rows 3.4
rest on the stand reproduction of a lost answer instead of failover runs
that never failed, a new row proves the contract, public tests and task
untouched against upstream and names the CI job, and the level 4
experiments state their repeats, data volume and resource snapshots.

Track 1: errors are attributed to the replicas that returned them, with
a failure column in the pool control table; the pool control states its
fixed variant order and growing base, pool 4 is a lower bound and no
longer compared with 4x1; the write command times are call-weighted over
all statement rows; the 3 600 rps comparison names the uneven foreign
load and rests on lock waits and CPU per request; the limits name the
database sizes behind the series A peaks and what host-others.csv
covers. AGENTS.md lists retry-check and the LB wait of make up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 05:59:52 +03:00
ArtyomandClaude Opus 5.5 7f2deec511 Record levels 3 and 4 again after the second review fixes
Every level 3 and 4 record is taken anew on e049c26 from a reset stand:
checks with the upstream diff of the contract, the public tests and the
task (empty) and of .github (only the own CI job), student tests with
the 503 answer to a request cancelled at shutdown, smoke, stack,
storage on the filled database, both failover runs with the
service.draining event, persistence, hooks, faults, limits, lb-check,
the new retry-check, three detection runs, failover-each and both
rolling updates with the database size, a docker stats snapshot and the
tag and id of every replaced image. The clean clone run executes the
launch block of the report verbatim and marks the observations made
between its steps.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 05:59:38 +03:00
ArtyomandClaude Opus 5.5 e049c265b8 Reproduce a lost answer on the stand and enrich the run records
No failover run ever lost an answer, so the retry of an unconfirmed
write was only covered in process. make retry-check holds the chat row
with SELECT ... FOR UPDATE for 1.5 s (below lock_timeout), sends a
message with a 0.5 s client timeout, finds the replica whose request
waits on the lock in pg_stat_activity and then retries with the same
key and text through the LB, once with that replica alive and once
after docker kill. It checks the 201, a single copy with the id of the
retry answer and, for the live replica, the 201 it logged without a
client; created_at tells which attempt wrote the message. CI runs it.

failover-each and rolling-update now record the database size and a
docker stats snapshot in the background of the action, rolling-update
names the tag and id of every image it replaces, and scaling.py writes
started and the compose-up log marks in UTC with Z; the track README
states that the recorded rows are host time (MSK) and what
host-others.csv covers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 05:43:44 +03:00
ArtyomandClaude Opus 5.5 fb6ff87bb3 Wait for every replica in make up and check numeric uids in CI
compose up --wait returns once lb is healthy, and lb is healthy with one
replica in rotation; the second one joins after rise 2 checks, about
2.5 s later, so make test-failover right after make up could see a
single X-Instance-Id. make up now runs scripts/lb-wait.sh, which waits
for as many UP servers as there are running api containers.

The CI non-root step compared the third column of docker compose top
with 0, but the daemon prints the user name there (root), so it could
never fail. It now reads the numeric uid of every process of every
container with docker top -eo pid,uid,comm (Docker needs the PID
column); a throwaway root container makes the check fail.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 05:41:13 +03:00
ArtyomandClaude Opus 5.5 d99a48a5db Put readiness behind a port and log the drain window through Logger
The /health/ready route imported the PostgreSQL adapter and kept the
drain flag in the HTTP layer, which gunicorn_conf imported to raise it.
Readiness is now a port of the shared application layer; its adapter in
shared/infrastructure checks the per-process drain flag and pings
PostgreSQL, and PlatformProvider assembles it. The route only knows the
port, and gunicorn_conf raises the flag through the infrastructure
module. Another critical dependency now changes the adapter and the
provider, not the HTTP layer.

The drain window was logged by a uvicorn logger with a printf string;
it is now the service.draining event of a StructlogLogger with the same
fields as the application, and failover-evidence looks for it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 05:40:16 +03:00
ArtyomandClaude Opus 5.5 aa1485bd86 Answer requests cancelled at shutdown with 503 in the contract format
After the drain window uvicorn waits for accepted requests for three
seconds and then cancels their tasks. CancelledError is a BaseException,
so neither the pipeline nor ServerErrorMiddleware saw it and uvicorn
answered with its own text/plain 500 without X-Request-Id and
X-Instance-Id. The pipeline now handles the cancellation next to the
request timeout: it logs request.cancelled, sends 503 unavailable in the
common error format when the response has not started and re-raises,
so uvicorn only closes the transport after the buffered answer.

The declared Content-Length check joins the guard as a rejection to keep
the call within the statement limit; behaviour is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 05:39:09 +03:00
ArtyomandClaude Opus 5.5 13e856ebd4 Propose closing the second lab 3 review gaps as an openspec change
The second review found that a request cancelled at the end of the
graceful shutdown gets uvicorn's plain 500 without the contract headers,
that readiness reaches the PostgreSQL adapter and a global flag of the
HTTP layer past the ports, that the drain window is logged past the
Logger port, that make up returns with one replica in rotation and that
the CI non-root check can never fail. The retry of an unconfirmed write
was never exercised on the stand either.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 05:37:31 +03:00
ArtyomandClaude Opus 5.5 5a3e372b08 Name the load of each column in the placement control table
The CPU per request and the lock waiters of the placement control are
taken at 4560 rps; the header now says so instead of leaving the reader
to guess between the two offered rates.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 04:47:39 +03:00
ArtyomandClaude Opus 5.5 d3ee257df1 Archive the lab 3 review gaps change
The deltas are synced to openspec/specs: the order of checks on posting
a message (401, then 400 for the request shape without touching the
chat, then 404 for access, then 400 for the text); three detection runs
with a 0.25 s measurement slack and the replica states after each stop
in failover-each; the upper bound of the track 1 grid with a ceiling on
the first grid point stated as a lower bound; control runs of the pool,
the LB queue and the placement.

Before archiving, on the running stand: make test passes, failover
recovers the first write in 2.78 s with 0 failed attempts, persistence
passes; make check, make lint and 243 own tests pass. All level 3 and 4
records were taken on b1c80ee, after the lab 2 carry over.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 04:47:14 +03:00
ArtyomandClaude Opus 5.5 fa09339388 Report the control runs of track 1
The LB queue does not move the peak of 8x1 (4526, 4677 and 4525 rps for
maxconn 16, 32 and 64, overlapping ranges), and the pool moves it the
other way: 4 connections per worker give 5358 rps, 16 give 3789, while
the lock waiters grow from 14.7 to 102. So the pool is not short, extra
connections only add writers to the hot chat row, which confirms the
coordination cost of the USL fit. On the same CPUs 8x0.5 peaks 7 % below
4x1 and costs 15 % more CPU per request, so the retrograde 8x1 comes from
the number of processes and connections, not from slower vCPUs.

The report also names the cause of the flaky compose up, the WalSync
stall of the dropped attempt, the disturbed control runs, and the
evidence index gets a commit column for track 1. The scenario of the
open change no longer presumes its own outcome.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 04:45:55 +03:00
ArtyomandClaude Opus 5.5 f4833c40c0 Record the control runs of track 1
make track1-control on b1c80ee: 8x1 at 5400 rps with a pool of 4, 8 and
16 connections per worker and an LB maxconn of 16, 32 and 64 per
replica, then 4x1 against 8x0.5 on the same CPUs 7-10, three repeats
with the variants interleaved.

The first placement attempt is kept in aborted/ only as a summary: four
4x1 runs had the write command at 7-34 ms instead of 0.2-0.6 ms, waiting
on WalSync, which the CPU based rejection does not see. The rerun right
after it is the one in control-cpus. The full output of the two failed
compose up attempts names the cause of the flaky switch: all replicas
are recreated at once and lb turns unhealthy without a ready server.

Plots and capacity.md now include both control series.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 04:45:55 +03:00
ArtyomandClaude Opus 5.5 a67bc6526a Report levels 3 and 4 on the code after the lab 2 carry over
The report now names a59e3a3 as the last change of the application code
and b1c80ee as the commit of every level 3 and 4 record, with the new
replica IDs and numbers: three detection runs within the formula without
the measurement slack, failover-each and both rolling updates without a
failed request, 12 stack tests and 243 own tests. The shutdown section
and the timeout chain follow the carried over budget: drain 2 s, then
3 s for accepted requests and a 500 for a stuck one, graceful timeout
6 s, stop hooks 7 s. make validate runs from .venv, faults evidence has
its own target, AGENTS.md lists the control runs of track 1.

The clean clone is taken again on f5fea5c without an explicit BASE_URL.
The track 1 part keeps the review fixes of the H2 verdict, the grid
bounds and the rounded numbers; the control runs follow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 03:17:56 +03:00
ArtyomandClaude Opus 5.5 f5fea5c59c Record levels 3 and 4 again after the lab 2 carry over
The carry over of the lab 2 fixes changed the request pipeline, logout,
the worker shutdown budget and the stop hooks, so every level 3 and 4
record is taken again on b1c80ee: checks, own tests, smoke, stack,
storage, failover of both replicas, persistence, hooks, the lab 2 stack
tests, limits and the injected violations, lb-check, three detection
runs, failover-each and both rolling updates under k6.

All of them pass: 243 own tests, 12 stack tests, no failed request under
k6 in failover-each and in both rolling updates.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 02:53:12 +03:00
ArtyomandClaude Opus 5.5 b1c80ee544 Anonymize the repository path in the faults evidence
make faults-evidence wrote the pytest rootdir with the local home
directory, so the committed faults.txt was cleaned by hand and could not
be reproduced by the command from the report. The target now replaces
the repository path with <repo> and still fails when a stack test fails,
since the pipe through sed hides the pytest exit code.

The open review change also records that the lab 2 carry over moved the
default image tag to the compose project.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 02:42:45 +03:00
ArtyomandClaude Opus 5.5 ac15b2ff62 Archive the carry over of the lab 2 fixes change
Lab 3 now stands on the finished lab 2. The deltas are synced to
openspec/specs: the zones meet only in the schema and Valkey is needed
only to confirm a logout; an outsider does not wait for a chat lock;
an empty cursor is 400 on history and on the chat list; a stuck request
at SIGTERM gets 500 from the server itself within the 6 s budget, which
still leaves 3 s after the drain window for the database timeouts; a
chunked body is limited by the bytes received; a logout rejected by
PostgreSQL withdraws its revocation mark. The event publication purpose
follows lab 2.

On a stand rebuilt from empty volumes: make test passes, failover
recovers the first write in 3.1 s with 0 failed attempts, persistence
passes, test-faults 12 passed including the full restart with logout
and the shortened session after an API restart, detection passes all
four scenarios with SIGTERM detected in 0.82 s and the stop done in
2.44 s, and the cache evidence finds a tombstone and a key that does
not outlive its session.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 02:38:45 +03:00
ArtyomandClaude Opus 5.5 a59e3a39a8 Carry over the lab 2 fixes made after 8cddb09
Brings the system changes of lab 2 from 8cddb09..102579e onto the lab 3
code, resolving each conflict by meaning:

- the request pipeline counts the body bytes actually received, so a
  chunked body over the limit gets 413 instead of reaching the app;
- logout revokes through a SessionRevocations port and withdraws the
  mark when PostgreSQL rejects the delete, so 503 means the session
  still works; PostgresSessionDirectory loses its no-op invalidate;
- an empty cursor is 400 on history and on the chat list;
- EventBus.project and the test that allowed importing an event across
  zones are gone, the config rules speak of the schema foreign key;
- the worker waits one second less than the rest of the shutdown
  budget, so it cancels a stuck request with 500 and exits 0 instead
  of being killed by the gunicorn master. Lab 3 keeps 3 s after the
  drain window for the database timeouts, so SHUTDOWN_TIMEOUT_SECONDS
  becomes 6 s and the stop hooks wait 7 s, still under the 8 s grace;
- the lab 3 single statement post already locks the chat only for a
  member, so lab 2's LOCK_CHAT edit is not needed; its test is kept;
- lab 2 tests come along with Idempotency-Key where lab 3 needs it;
  the short TTL after restart runs without api-shortttl by shortening
  the session in the database; test_drain adds a stuck request;
- make validate uses .venv with openapi-spec-validator, PYTEST_ARGS,
  faults-evidence, down with the bench profile, the image is tagged by
  the compose project, start probes every 200 ms, cache evidence fails
  without a tombstone and checks the key expiry, kdf.txt shows the
  seed stub apart.

The track 2 tooling of lab 2 (track.py, postgres-restore and the
database argument of migrate) and its evidence scripts stay out.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 02:32:49 +03:00
ArtyomandClaude Opus 5.5 79a52bab58 Propose carrying over the lab 2 fixes as an openspec change
Lab 3 was carried over from lab 2 at 8cddb09, and lab 2 changed up to
102579e after two reviews. The change brings those system fixes here:
the chunked body limit, a worker that cancels a stuck request before
the master kills it, a logout that withdraws its revocation mark when
PostgreSQL rejects it, the empty cursor, the zone rule kept to the
schema and the toolchain and stand tests. To keep the lab 3 rule that
draining leaves the database timeouts their full 3 s, the shutdown
budget grows from 5 to 6 s and the stop hooks from 6 to 7 s.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-30 02:32:12 +03:00
ArtyomandClaude Opus 5.5 6fafc8a784 Record a clean clone run without an explicit BASE_URL
The launch block of the report now works verbatim with a changed
LB_HTTP_PORT: in a fresh clone with port 18083 make test, both
failovers, persistence and storage-evidence pass without BASE_URL, the
second failover stops the other replica after one shifting request, and
storage-evidence lists only the volumes of the clone project.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:26:05 +03:00
ArtyomandClaude Opus 5.5 e722cb357d Record levels 3 and 4 again on the final code
Every level 3 and level 4 file in evidence was taken on e00eb6c, before
track 1 rewrote the message write into one statement under the chat
lock. The runs are repeated on a02d442: smoke, stack, storage, both
failovers (the second after one shifting request), persistence, hooks,
the lab 2 fault tests, limits and injected violations, 20 same-key races,
three detection runs, failover-each under k6 and both rolling updates.
All pass. checks.txt adds the real output of make check, make validate
and make lint, and student-tests.txt includes the new access order test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:24:11 +03:00
ArtyomandClaude Opus 5.5 a02d442085 Trim the scaling cost proposal to the word limit
The openspec rules keep a proposal within 400 words; the archived track 1
proposal had 405. Three phrasings outside the hypotheses are shortened,
the hypotheses stay as they were written before the runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:13:01 +03:00
ArtyomandClaude Opus 5.5 956d24a701 Repeat detection, add track 1 control runs and isolate image tags
BASE_URL defaults to the published port of lb from docker compose, so
the targets work with a changed LB_HTTP_PORT. rolling-update FROM_REV
and track1-compare tag the old revision by its commit instead of the
shared mtusi-messenger tag, which a parallel clone could overwrite.
detection runs three times to show the spread, failover-each records
docker ps -a of the replicas right after the stop, and failover-evidence
takes SHIFT to move round robin to the other replica. track1-control
varies the pool and the balancer queue at 8x1 and compares 8x0.5 with
4x1 on the same CPUs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:12:55 +03:00
ArtyomandClaude Opus 5.5 16b6b28ea9 Keep the full output of a failed compose up in track 1 runs
The orchestrator printed only the last 500 characters of stderr before
retrying, which hid the cause of a flaky up on eight replicas. The whole
output of each failed attempt now goes to compose-up-failed.log next to
summary.csv.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:12:55 +03:00
ArtyomandClaude Opus 5.5 0ce7d6ecf0 Make storage and limits evidence reproducible from any clone
storage-evidence listed volumes by the name mtusi2026lab3, so a clone in
another directory recorded the volumes of the main stand; it now takes
the project name from docker compose config and filters by its label.
make limits wrote only the date, while every evidence file promises the
date and the commit in its first line; the commit is added.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:12:55 +03:00
ArtyomandClaude Opus 5.5 d4a3124a32 List gunicorn_conf in the module map of the openspec context
Since level 4 the module holds platform behaviour, the drain window on
SIGTERM, so the architecture map names it next to main and migrate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:12:45 +03:00
ArtyomandClaude Opus 5.5 e9242816a5 Drop the type ignore comment from error classification
ty accepts the lookup without it, and the project keeps code free of
comments.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:12:45 +03:00
ArtyomandClaude Opus 5.5 a761b718b8 Check the request shape before chat access on posting a message
FastAPI rejects a missing Idempotency-Key, extra fields and wrong types
with 400 before the handler runs, so an outsider gets the same 400 for
an existing and a missing chat; only a bad text, checked by the domain,
makes the handler look up membership and answer 404. The new test pins
that order so it is a recorded rule rather than an accident.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:12:45 +03:00
ArtyomandClaude Opus 5.5 be17939ddb Propose closing the lab 3 review gaps as an openspec change
An independent review of the grade 5 submission found that the level 4
measurements under load and the fault tests were taken on the code
before track 1 rewrote the message write, that the claim that the pool
and the balancer are not the bottleneck rests on reasoning alone, and
that the retrograde 8x1 result is not separated from running on other
vCPUs. Some evidence tooling also writes data of another compose
project or cannot reproduce a committed file.

The change records the order of checks on posting a message (401,
request shape 400, access 404, text 400), requires three detection runs
with the 0.25 s measurement tolerance, a docker ps of replicas after
each stop in failover-each, an upper bound of the rate grid, and control
runs that vary the pool, the balancer queue and the CPU placement one at
a time.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 23:12:16 +03:00
ArtyomandClaude Opus 5.5 e425a9d236 Finalize the lab 3 report against recorded evidence
Rerun the clean clone pass on the final commit in its own compose project:
secrets, config, up with migrations on empty volumes, readiness, smoke,
replica spread and same key races, failover, persistence, repeated
migrations, down keeping volumes and reset-data removing them. Record the
student tests by name and the limits policy on the final code.

Check every figure of the report against evidence files and fix the ones
that drifted: data set sizes, the share of the chat lock statement, the
WAL waits, the aborted series, the disturbed steps, the restart timing
and the image ID of the rolling update. State which commit each evidence
file was taken on, answer the defense questions, index evidence/ and
bring AGENTS.md and the openspec context up to date for lab 3.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 03:13:24 +03:00
ArtyomandClaude Opus 5.5 e88673dfec Archive the scaling cost change
Track 1 of grade 5 is closed. At a fixed 4 CPU / 2 GiB the 1x4, 2x2
and 4x1 layouts peak at 5 375, 4 964 and 4 724 rps: within the
hypothesised 15 percent, yet smaller replicas cost a third more CPU
per request and reach a long tail earlier. Growing one-CPU replicas
gives 1 720, 2 936, 4 724 and only 4 260 rps at eight, a retrograde
curve with USL sigma 0.014 and kappa 0.038. The saturation point is
the chat row lock in PostgreSQL: at eight replicas 47.5 of 64
connections wait for it while the API, the balancer and the database
CPU still have room. Writing a message with one statement removed the
application round trips from under the lock: p95 at 3 600 rps fell
from 64.5 to 3.7 ms and the peak rose by 15 % with the same resources.

The new lab3-scaling-cost spec is synced to openspec/specs, with the
compare scenario stated as measured: below the knee fewer lock waits
and a lower p95, past it a higher peak. AGENTS.md lists make track1
and make plots. The neighbour CPU sampler is stopped and its file
closed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 01:42:38 +03:00
ArtyomandClaude Opus 5.5 3e63b16ca6 Report track 1 with plots and the checks after the improvement
make plots draws throughput and latency against offered load for both
series and the comparison, CPU per component, peak RPS against API
CPU with the USL fit and the improved 8x1 point, and the utilisation of
every link at each peak; capacity.md tabulates the ceilings.

after-checks.txt holds the level 3 and 4 checks on the new image:
student tests 240 passed, smoke PASS, 20 same-key races of 8 through
the balancer with 160 x 201 and one copy each, failover with 131
writes and no failure in 2.73 s, persistence PASS.

REPORT.md gains the track section: hypotheses as stated before the
runs, the method and its equal-resource rules, both series in tables,
the saturation point with database, balancer and pool figures, the
USL reading, the before and after table, conclusions per hypothesis
and the limits of a shared single host. The idempotent send section
now describes the one-statement write, the evidence table has the
track rows and the commands are in the clean clone section.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 01:42:08 +03:00
ArtyomandClaude Opus 5.5 dde113b4ed Record the before and after comparison of track 1
8x1 with the image of edc091d (before) and the one-statement write
(after), alternating per repeat, three repeats at 3 600, 5 400 and
8 100 rps. At 3 600 rps, below the knee, p95 falls from 64.5 ms
(35.7..142.5) to 3.7 ms (2.8..5.4) and p99 from 122 to 13 ms; the API
spends 391 instead of 457 % CPU and PostgreSQL 102 instead of 137 %;
connections waiting for a chat lock drop from 12.8 to 0.15 and none is
idle in transaction any more. Past the knee the peak rises from 4 252
(4 054..4 396) to 4 894 rps (4 494..5 104), +15 %, and at 8 100 rps
from 4 104 to 4 841 (+18 %); the ranges do not overlap. Saturated, the
lock holders now wait for the WAL flush (LWLock WALWrite) instead of
the application.

The switch to the after image for the third repeat failed in compose
while another lab was running tests on the shared VM, so that repeat
was run alone with --first-repeat 3 on the same database; apply now
retries compose up twice and prints its stderr. Two after rows were
still marked disturbed after three reruns and are kept; they are the
lower ones.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 01:37:34 +03:00
ArtyomandClaude Opus 5.5 296e070eba Add the before and after comparison of track 1
make track1-compare builds the image of BEFORE_REV, the last commit
without the one-statement write, builds the current one, and runs 8x1
at 3 600, 5 400 and 8 100 rps with the two images alternating per
repeat, so drift of the shared VM affects both alike. make track1 runs
the host snapshot, both series, the comparison and the plots.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 01:06:49 +03:00
ArtyomandClaude Opus 5.5 22f0e6244d Post a message with one statement under the chat lock
Sending a message used to be BEGIN, a SELECT ... FOR UPDATE of the
chat with its members, a lookup of the idempotency key, the insert CTE
and COMMIT. Three of those round trips ran under the chat row lock, and
each one waited for the replica's event loop, so under load the lock
holders sat idle in transaction while everyone else queued behind
them.

The new MessageLog port has one PostgreSQL adapter that does it all in
a single statement with an implicit transaction: lock the chat row
joined with the sender's membership, stamp the message with the later
of the replica clock and the last message plus one microsecond, claim
the key (overwriting it only if it has expired), insert the message
only when the key was claimed, and move last_message_at. The foreign
key from the key to the message is checked at the end of the statement,
so the key may be written first. When the key is already taken the
adapter reads the stored result with a second statement outside the
lock, which also covers a concurrent duplicate that waited for the
first one; the handler then replays it or answers 409 as before.

The chat aggregate no longer takes part in posting, so its post,
membership and last_message_at members and their unit tests are gone;
Message.posted() builds the event. An invalid text is still answered
404 for an outsider: only then the handler reads the membership. A new
API test pins the clock rule against a real database: a lagging
replica clock gets the last stamp plus one microsecond, a clock ahead
is kept. The concurrency, same-key race, expiry and access tests pass
unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 01:06:49 +03:00
ArtyomandClaude Opus 5.5 edc091dbc4 Choose a one-statement message write as the track 1 improvement
Series B located the bottleneck. At eight replicas the API has 40 % of
its CPU spare and the balancer 60 % of a core, yet three quarters of
the pool connections wait for a chat row lock: the locking SELECT of
the chat averages 14.2 ms and the lock holders sit idle in transaction
while the replica's event loop gets round to the next statement. The
critical section, three round trips long, is what more replicas
lengthen.

The design now records the choice: one CTE that locks the chat with
the membership check, stamps the message no earlier than the last one,
claims the idempotency key and inserts the message, committed by the
statement itself, so the lock never waits for the application. A taken
key is read back by a second statement outside the lock. The price is
that the chat aggregate no longer guards posting; the rules move into
the adapter's SQL and are held by integration tests. The rejected
alternatives and the before/after plan are listed, and the spec gains
the observable requirement of a short critical section.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 01:06:10 +03:00
ArtyomandClaude Opus 5.5 3bdc8fad23 Record the growth series of track 1
One to eight replicas of 1 CPU, 512 MiB and one worker each, three
repeats per rate; the 4x1 point comes from series A. Peak successful
RPS: 1 720 for one replica, 2 936 for two, 4 724 for four and only
4 260 for eight, lower than four. Up to two replicas the API is the
limit (101 % and 201 % of its CPU). At eight replicas the API uses 473
of its 800 % and the balancer 40 % of a core, while PostgreSQL is at
178 of 200 %, 47.5 of the 64 pool connections wait for a chat row lock
and the balancer queue holds 744 requests. The locking SELECT ... FOR
UPDATE of the chat takes 14.2 ms on average, 97 % of all statement
time of the step, and about 7 connections sit idle in transaction:
the lock holders wait for the application between round trips.

A universal scalability law fit over 1, 2, 4 and 8 CPUs gives
sigma 0.014 and kappa 0.038, a peak near five replicas and a decline
after it. No step of this series was rejected as disturbed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 01:05:39 +03:00
ArtyomandClaude Opus 5.5 be19abf364 Record the fixed budget series of track 1
1x4, 2x2 and 4x1 at 4 CPU and 2 GiB of API, workers equal to CPUs,
32 pool connections and 128 balancer slots in every layout, three
repeats from 600 to 6 830 rps. All three keep up with the offered
load up to 4 560 rps; the peaks are 5 375 rps for 1x4 (5 332..5 384),
4 964 for 2x2 (4 894..5 104) and 4 724 for 4x1 (4 686..4 794). At the
peak the API uses 95 % of its four CPUs in every layout, PostgreSQL
82..89 % of its two, the balancer 40 % of one, and about half of the
32 database connections wait for the lock on a chat row.

Smaller replicas cost more: at 3 040 rps the API burns 215 % CPU as
1x4, 243 % as 2x2 and about 280 % as 4x1, and p95 at 4 560 rps grows
from 9 to 69 and 150 ms.

The first attempt was aborted when a neighbour's image build froze the
Docker API; its summary is kept under aborted/. Twelve steps of the
rerun were rejected as disturbed and repeated. A config's SLO capacity
now needs a majority of saturated repeats, since a median is robust to
one disturbed repeat and "any repeat" is not. Series B starts at 450
rps per CPU.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 00:31:43 +03:00
ArtyomandClaude Opus 5.5 67f6d6866e Reject steps disturbed by foreign load on the shared VM
The first run of series A was spoiled by other stands on the same
Docker VM: an image build of another lab is invisible to docker stats,
yet it took CPU from the pinned replicas, doubled their CPU per request
and stalled the Docker API for fifteen minutes in the middle of two
steps.

docker-stats.sh now also samples the VM-wide /proc/stat counters, so a
step records the CPU of the whole VM, the part not used by this
project, the largest gap between samples and its wall time. A step
with more than one foreign core, a gap over 10 s or a k6 run 90 s
longer than its load is written to disturbed.csv and repeated after
the VM has been quiet for 15 s, up to three times. On a quiet VM the
foreign part is 2 % of a core at 600 rps and 25 % at 4 560 rps, which
is Docker networking and the samplers themselves.

The growth series starts at 300 rps per CPU and searches up to 9 000,
so its knees are found with fewer idle steps. The plots fit the
universal scalability law to the growth series.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 22:18:14 +03:00
ArtyomandClaude Opus 5.5 e8804b05d0 Plot track 1 series and tabulate the ceilings
tools/plot_track1.py reads any set of summary.csv files and draws, per
series, successful RPS, p95 and p99 against the offered rate and CPU
per component against its limit, plus two cross-series charts: peak
successful RPS against total API CPU with the linear growth line, and
the utilisation of API, PostgreSQL, balancer and the share of the pool
waiting on a lock at each config's peak. A point is the median of the
repeats, the whiskers are min and max. capacity.csv and capacity.md
hold the SLO capacity, the peak and the resource figures at the peak.

Colours follow the config across charts, every line also has its own
marker and a direct label. matplotlib comes through uv run --with, so
the tool lives in tools/ and stays out of the 3.9 scripts; make plots
runs it, make lint covers it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:35:05 +03:00
ArtyomandClaude Opus 5.5 8ea1996063 Add the series orchestrator and samplers for track 1
scripts/scaling.py applies a replicas x CPU layout through compose
variables: workers equal to the replica CPUs, a pool of 8 per worker,
a balancer queue of 32 per worker so every layout of a series offers
the same concurrency to the API, and the replicas pinned to CPUs 7 and
up, next to PostgreSQL on 0-1, the balancer on 2, Valkey on 3 and k6
on 4-6 with 3 CPU and 1 GiB. For every offered rate it runs k6 three
times and stops a config after two steps saturated in all repeats.

Each step keeps its raw data: k6 summary, docker stats, HAProxy
statistics and pg_stat_activity sampled every second, and the top of
pg_stat_statements. The summary row carries CPU per component,
PostgreSQL active, idle, idle in transaction and lock waiting backends,
the balancer queue and the database size. Several variants alternate
per repeat for before and after runs. host-snapshot.sh records every
container on the host before a series.

make track1-a and track1-b run the fixed budget and the growth series
from an empty database.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:25:23 +03:00
ArtyomandClaude Opus 5.5 107fbbbe15 Let measurements pin each component to its own CPUs
Track 1 compares replica layouts that must get exactly the same CPUs,
and the load generator, balancer, Valkey and PostgreSQL must not steal
cycles from the replicas or from each other. Every service now takes
an optional cpuset from LB_CPUSET, API_CPUSET, PG_CPUSET,
VALKEY_CPUSET and K6_CPUSET. They are empty by default and compose
drops an empty cpuset, so the ordinary stand is unchanged and make
limits still passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:25:23 +03:00
ArtyomandClaude Opus 5.5 d8d551aae4 Propose the scaling cost track as an openspec change
Track 1 of grade 5 asks to compare one, two and four API replicas at a
fixed total CPU and memory budget, to grow resources until a shared
dependency saturates, to show with database, balancer and pool metrics
why more replicas stop helping, and to confirm one improvement with a
before and after measurement.

The hypotheses are stated before any run: at 4 CPU / 2 GiB the 1x4,
2x2 and 4x1 layouts reach the same ceiling within 15 percent; with one
CPU per replica throughput grows almost linearly up to four replicas
and by less than 1.5 times from four to eight, because PostgreSQL
saturates on the chat row lock held across three round trips and on
CPU, while the balancer stays far from its limit.

The new lab3-scaling-cost capability fixes the method: open-model k6
with the level 4 profile, three repeats per point, equal workers, pool
and balancer concurrency across layouts, CPU pinning per component,
raw samples per step, plots from summary files, and an improvement
chosen only after the bottleneck is found.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:16:19 +03:00
ArtyomandClaude Opus 5.5 61c0c28a43 Archive the failover operations change
Grade 4 of lab 3 is closed. A replica on SIGTERM first answers 503 on
readiness for a two second window, longer than the 1.5 s worst
detection, and only then closes its listener and finishes accepted
requests within five seconds; the balancer lets a replica back only
after two passed checks. A test pins the timeout chain to compose,
HAProxy and the stop hook. make detection measured 0.71 s for SIGTERM
(before the process exited at 2.49 s), 0.67 s for a killed process,
2.76 s for a hung one and 1.23 s until no replica was left with
PostgreSQL down. make failover-each stopped and restored each replica
under k6 at 100 rps with no errors and no balancer retries, make
lb-check ran 20 same-key races of 8, make limits checks limits and
isolation against a policy, and make rolling-update replaced the
level 3 image with the current one under load with capacity kept at
two and no errors, migrate running once. The new
lab3-failover-operations spec is synced, and the lab 1 operability
and lab 2 dependency failure specs carry their deltas.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:08:15 +03:00
ArtyomandClaude Opus 5.5 a9aab2727f Record level 4 evidence and extend the report
Evidence from a freshly recreated stand at the current code: the four
detection cases with their declared maxima, failover-each for both
replicas under k6 at 100 rps (no errors, no balancer retries, the
stopped replica never served during its outage), the 20 round
same-key race, the limits and isolation check with a run against
deliberate violations, and two rolling updates under load: from the
level 3 image, where the old replicas without a drain window caused
54 redispatches and 2 response errors that the balancer retried, and
between two current builds with all counters at zero. Level 3 runs
are refreshed on the new code: smoke, stack view, both failovers with
the drain sequence in the replica log, hooks with the longer stop,
persistence, stack fault tests and own tests with the shutdown and
timeout chain output.

The report gains the shutdown order, the check period and worst
detection per failure with measurements, the timeout chain, rows 4.1
to 4.4, level 4 rows of the failure table, the k6 role, the new
variables and the updated limitations; AGENTS.md lists the commands.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:07:51 +03:00
ArtyomandClaude Opus 5.5 6107c1aa34 Show the drain sequence of the stopped replica in failover evidence
The failover evidence printed the last six log lines of the stopped
replica, which no longer reach back to the signal now that shutdown
starts with a two second drain window. It now filters the replica log
since the start of the run for the signal, the drain message, the 503
readiness answers, the listener shutdown and the process exit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:07:44 +03:00
ArtyomandClaude Opus 5.5 e00eb6c878 Keep probe logs out of the ignored log pattern
The repository ignores *.log, so the action log of a load run (with
the rolling update replacement lines) and the k6 output would never
reach evidence/. The probe now writes action.txt and k6/k6.txt.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:52:55 +03:00
ArtyomandClaude Opus 5.5 eb04fc10e3 Give every level 4 check one make target
make limits, make lb-check (now 20 rounds of 8 parallel posts with one
key), make detection, make failover-each and make rolling-update each
write their file in evidence/ and fail on a violated check; make
level4 runs them in order. RATE sets the k6 arrival rate (100 rps by
default). make rolling-update builds the image before the load starts
so the build does not skew the measurement; with FROM_REV=<commit> it
first builds the image tag from git archive of that commit and
recreates the replicas on it, so the update is a real change of
version, and it writes to evidence/rolling-from-<commit>. Lint covers
scripts/limits.py. CI acceptance runs make limits and make detection.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:52:29 +03:00
ArtyomandClaude Opus 5.5 3584501de8 Check that the balancer drops every replica while PostgreSQL is down
The stack test for a stopped PostgreSQL asked for liveness 200 through
the balancer, which only held while the balancer had not noticed the
outage yet. Now the protected endpoints and readiness through the LB
must answer 503 unavailable within 2 s, liveness is checked inside a
replica with docker exec, the balancer must report zero UP servers
within 2 s and answer liveness itself with 503 in the contract error
format, the request id echoed and no X-Instance-Id. After PostgreSQL
starts again every replica must be back in rotation without a restart.

Stand helpers read the number of UP servers from the HAProxy stats,
excluding the backend summary row, and reach a replica by container
id, which survives the renumbering after a rolling update.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:52:29 +03:00
ArtyomandClaude Opus 5.5 572204950a Replace replicas one by one under load
scripts/rolling-update.sh runs migrate once before touching any
replica, then for each old replica scales the service to N+1 without
recreating the others, waits until the new container is healthy and
the balancer sees N+1 UP servers, and only then stops the old one
with docker stop -t 6 (so it drains) and removes it. The log names
the hostname and image of the old and the new replica of every step,
which is what the report quotes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:52:19 +03:00
ArtyomandClaude Opus 5.5 5adb8719d4 Check limits and isolation against a declared policy
scripts/limits.py states which secrets, volumes and networks each
service needs and that only lb publishes ports, on loopback. It
compares that policy with docker compose config (with the bench
profile, so k6 is covered) and with docker inspect of the running
containers: CPU and memory limits declared and applied, no
container_name, no published port on any API replica, a read-only
root filesystem for the API, and which networks are internal. It
prints both tables and every violation and exits non-zero on any.

The stack evidence script no longer reaches replicas through
docker compose exec or port, which assume container number 1 and
break after a rolling update renumbers the replicas.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:52:09 +03:00
ArtyomandClaude Opus 5.5 d5f8a7c25c Measure detection and fail every replica in turn under load
scripts/detection.sh runs four failures one after another and brings
the stand back between them: SIGTERM through docker stop -t 6 (the
balancer must exclude the replica within 1.5 s and before the process
exits after its drain window), docker kill (1.5 s), docker pause for a
hung process (3.5 s, the check timeout counts twice) and stopping
PostgreSQL (all replicas out within 1.5 s, the balancer answers 503
itself). A 0.25 s slack covers the docker CLI and the 50 ms polling.
Each run writes evidence/detection/<case>.json and the script fails
if any case exceeds its declared maximum.

scripts/failover-each.sh stops every replica in turn with stop-one.sh,
keeps it down for 10 s and restores it, all inside one k6 load window,
and collects a summary with RPS, errors, latency, detection, rejoin,
probe recovery and balancer retry counters per replica.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:52:09 +03:00
ArtyomandClaude Opus 5.5 711c9c98dc Add the k6 load profile and the load and detect probe modes
The k6 profile from the prepared kit runs an open model against the
balancer from the edge network: constant arrival rate, 80 % keyed
message posts and 20 % history reads over 20 synthetic users and 10
chats, metrics only for the measure phase, and no setup data (tokens)
in summary.json. scripts/k6.sh runs it with the host uid so the
summary is writable on Linux too.

scripts/probe.py gains three things. check --rounds repeats the
same-key race with a new key per round and counts rounds with
identical bodies, one stored copy and 409 on another body. load runs
an action under k6 while a probe posts every 0.1 s and resends the
same key after a failure and a poller reads the number of UP servers;
it writes the action log, the probe and LB traces, the delta of the
backend retry, redispatch and error counters, and a verdict: dip for
a replica that leaves and returns, replace for a rolling update that
must keep capacity and end on new replicas only. detect runs the
action in the background and polls the balancer at the same time, so
a drain that ends after the exclusion is visible, and checks the
declared maximum, the LB answer after detection and the time to
restore.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:51:59 +03:00
ArtyomandClaude Opus 5.5 502c7d8ace Keep a returning replica out of rotation until its checks pass
The first failover-each run had no errors, yet the balancer counted
19 and 49 redispatches. The HAProxy log showed why: when a stopped
container comes back into Docker DNS, its server-template slot leaves
MAINT (resolution) as UP/READY at once, before any health check, so
requests went to a gunicorn that was not listening yet and were
redispatched to the other replica.

default-server now sets init-state fully-down (HAProxy 3.1+): at start,
on discovery through DNS and on leaving maintenance a server stays DOWN
until rise 2 checks pass. A rerun under 100 rps shows zero retries and
redispatches for both replicas; rejoining takes about a second longer.
The design and the failover spec of the change record the finding.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:51:49 +03:00
ArtyomandClaude Opus 5.5 85ed37914d Pin the timeout chain in the stand and test it
The API service now names every timeout that the chain depends on:
DRAIN_SECONDS, DB_ACQUIRE_TIMEOUT_SECONDS and DB_COMMAND_TIMEOUT_SECONDS
join REQUEST_TIMEOUT_SECONDS and SHUTDOWN_TIMEOUT_SECONDS in compose,
and the image gets the drain default. A unit test reads the compose
defaults, the HAProxy check and timeout settings and the stop hook
timeout, and checks that the chain shrinks inwards: LB server 10 s >
request 8 s > pool wait plus statement 3 s, readiness 0.8 s < check
1 s, drain 2 s > worst detection 1.5 s, shutdown 5 s leaves 3 s for
accepted requests, and shutdown < hook 6 s < stop_grace_period 8 s
< 10 s. It also pins the declared detection maxima: 1.5 s for a 503
or a refused connection and 3.5 s for a hung replica.

The k6 load generator joins the stand under the bench profile: pinned
by digest, edge network only, 2 CPU and 512 MiB, no secrets.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:33:55 +03:00
ArtyomandClaude Opus 5.5 6fb4587ef6 Leave the rotation before closing the listener on SIGTERM
On SIGTERM gunicorn closed the listener at once, so a request already
queued on the closing socket reached the balancer as a dropped
connection after sending: a retry for GET and keyed messages, a 502
for user, chat and login posts. Now each worker first raises a
draining flag and keeps serving for DRAIN_SECONDS (2 s by default,
above the 1.5 s worst detection of a 503): readiness answers 503
unavailable, liveness stays 200, and the balancer takes the replica
out while it is still healthy. Only then uvicorn closes the listener
and finishes accepted requests within SHUTDOWN_TIMEOUT_SECONDS minus
the window, so gunicorn graceful_timeout stays 5 s.

The worker runs a uvicorn Server subclass whose main loop adds the
window after the exit signal; UvicornWorker._serve is overridden to
use it. Tests check readiness under the flag and run real gunicorn on
a free port: during the window ready is 503 and live 200, the port
closes after the window, a registration accepted before SIGTERM and
finished after the port closed still gets 201, and the master exits
with code 0 before the shutdown limit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:33:02 +03:00
ArtyomandClaude Opus 5.5 6e4c387e50 Answer readiness within the balancer check timeout
The readiness ping waited up to one second for a pooled connection and
one more second for SELECT 1, while HAProxy gives a check one second.
With a busy pool the balancer saw a check timeout instead of an
explicit 503. The ping now waits 0.4 s for a connection and 0.4 s for
the query, so the worst answer, 0.8 s, still arrives inside the check.
Other queries keep the pool acquire and command timeouts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:33:01 +03:00
ArtyomandClaude Opus 5.5 1ae4b770ae Propose failover operations as an openspec change
Level 4 of lab 3 asks for health checks, timeouts and SIGTERM handling
that agree with each other, a stated check period and worst detection
time, stopping every replica in turn under load with RPS, errors and
recovery time, a parallel race on one idempotency key, limits and
port and secret isolation, and a rolling update under load with
migrations run once.

The new lab3-failover-operations capability covers those guarantees.
The lab 1 operability delta makes a replica leave the rotation first
on SIGTERM: readiness answers 503 for a drain window longer than the
balancer detection time, and only then the listener closes. The lab 2
dependency failure delta states that the balancer drops every replica
and answers 503 itself while PostgreSQL is down. The design adds the
measurement tooling from the prepared kit: k6 under a bench profile,
probe load and detect modes, detection, failover-each, limits and
rolling update scripts with one make target each.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:30:27 +03:00
ArtyomandClaude Opus 5.5 8d28d45d55 Archive the load-balanced replicas and idempotent messages change
Grade 3 of lab 3 is closed: the stand runs two API replicas of one
compose service behind HAProxy as the only published entry point,
one token reaches both replicas by X-Instance-Id, stopping either
replica with scripts/stop-one.sh keeps writes going within 1.2-1.7 s
without errors or duplicates, eight concurrent posts with one key give
one message, and make test, the persistence scenario with
scripts/restart.sh, the failover scenario, lint, own tests, check,
validate and the lab 2 stack fault tests pass. The new
lab3-load-balancing and lab3-idempotent-messages specs are synced, and
the lab 1 history and operability specs and the lab 2 storage spec
carry their deltas.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:53:26 +03:00
ArtyomandClaude Opus 5.5 957eabdfc3 Record level 3 evidence and fill in the report
Save sanitized outputs from a stand started from a clean state: the
clean clone run, smoke through the balancer, the stack view with the
replica hostnames, ports, networks, limits and the balancer server
states, the probe with two replicas behind one token and the eight
way race on one key, a failover run for each replica with the
container left exited and its graceful shutdown log, the hook
negative cases and the balancer answer without replicas, the
persistence run with StartedAt of every container before and after,
schema, migration rerun, KDF prefix, ignore rules, own tests and the
lab 2 stack fault tests through the balancer, plus the environment.

The report covers the clean clone start, the architecture, the
X-Instance-Id and idempotency decisions, the balancer settings, the
role to service table, the mapping of .env.example to the real
variables, one evidence row per level 3 criterion and the carried
lab 2 minimum, and the failure table for level 3.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:53:11 +03:00
ArtyomandClaude Opus 5.5 0796e291b7 Restart containers in place in the restart hook
After make up, which builds with --build, a plain up in the restart
hook saw a different service hash and recreated migrate and the API
replicas instead of starting the stopped ones, so the replicas came
back with new hostnames. The hook now passes --no-recreate: lb and the
replicas are the same containers restarted, migrate reruns in its old
container, and only volumes and data carry state across the restart.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:47:06 +03:00
ArtyomandClaude Opus 5.5 ae8fb0a48e Record which replica stops and which services restart
scripts/failover-evidence.sh lists the api containers by hostname, the
X-Instance-Id they answer with, before and after make test-failover,
names the container left exited, keeps its last log lines with the
gunicorn graceful shutdown, then restores it and lists the replicas
again. scripts/evidence-stack.sh records the compose config check,
published ports, the missing port of a replica, hostnames, networks,
limits, process users, the balancer view of every server and the
alternating X-Instance-Id through the balancer, plus the environment
and image versions. The persistence and storage evidence scripts now
list every replica and the idempotency key table.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:44:03 +03:00
ArtyomandClaude Opus 5.5 ceb266ed05 Add the stop-one, restart and restore hooks and an LB probe
scripts/stop-one.sh maps TARGET_INSTANCE_ID to the api container whose
hostname matches, stops only it with docker stop -t 6 and leaves it
stopped; an unknown id exits 3 and touches nothing. scripts/restart.sh
replaces the lab 2 hook: it stops lb and the replicas, restarts
PostgreSQL and Valkey with their volumes and brings the stand back
with up --wait, which also reruns the idempotent migrate job.
scripts/restore.sh starts stopped replicas without recreating them
and scripts/lb-wait.sh waits until the balancer statistics show every
replica UP again, so the next failover run sees two replicas.

scripts/probe.py check (stdlib, Python 3.9 syntax) counts X-Instance-Id
over 40 calls with one token and races 8 posts with one key through
the balancer. make restore and make lb-check run them, ruff covers the
probe, and the CI acceptance job runs the failover test and the probe.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:43:52 +03:00
ArtyomandClaude Opus 5.5 dc2015473f Run the API as replicas behind HAProxy
compose.yaml now runs one api service with API_REPLICAS replicas and
no published ports, container name or hostname, behind lb, HAProxy
3.2.26 LTS pinned by digest and the only service with ports, bound to
127.0.0.1 (LB_HTTP_PORT for HTTP, LB_STATS_PORT for read-only stats).
The balancer finds replicas through Docker DNS with a server template,
checks GET /health/ready every second (fall 2, rise 2), balances round
robin without stickiness, retries connection failures on another
replica and allows a retry after a sent request only for GET and for
message posts carrying Idempotency-Key, never for users, chats or
login. Its own 502/503/504 answers use the contract error body with
the forwarded X-Request-Id.

The network is split into edge and an internal backend, every service
gets CPU and memory limits, the API runs read-only with a tmpfs /tmp,
two workers per replica, an 8 s request limit and a 5 s graceful stop
inside an 8 s stop_grace_period. Only api builds the image, migrate
reuses it once before the replicas start. PostgreSQL loads
pg_stat_statements and allows 200 connections. The short TTL instance
of lab 2 published the API around the balancer and is removed; its
expiry check stays as an in-process test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:43:43 +03:00
ArtyomandClaude Opus 5.5 7bf28fc6ed Make message posts idempotent with Idempotency-Key
POST /api/v1/chats/{chat_id}/messages now requires the header, checked
at the HTTP boundary (1-128 of [A-Za-z0-9._:-], else 400). The handler
keeps the lab 2 order under the chat row lock: membership, then text,
then the receipt for (chat, sender, key) younger than
IDEMPOTENCY_TTL_SECONDS. A receipt with the same fingerprint replays
the stored message, a different one is 409 conflict; without a
receipt one statement inserts the message, the key row and the chat
mark, so both commit or neither does. Concurrent retries wait on the
same row lock and then read the committed receipt. The fingerprint is
SHA-256 of the canonical JSON of the parsed body, so layout does not
matter, and only 201 answers are ever stored.

A lock timeout now counts as a dependency failure and answers 503
instead of 500, so a retry with the same key is safe.

Own tests cover replay, a different layout of the same JSON, conflict,
missing and malformed keys, rejected attempts that leave the key free,
scope by user and chat, revalidation after logout and for outsiders,
expiry, and eight concurrent retries through two app instances
sharing one PostgreSQL. Every earlier message post sends a fresh key.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:36:15 +03:00
ArtyomandClaude Opus 5.5 a18e7b320b Add the idempotency key table as an additive migration
Migration 0002 stores one row per accepted message key: the chat, the
sender, the key, the SHA-256 fingerprint of the request and the
message. The primary key (chat_id, user_id, key) is the key scope of
the contract, the composite foreign key keeps the row inside the
messaging tables, as messages already do, and CHECK constraints repeat
the key format and the fingerprint length. Nothing existing changes,
so the lab 2 code keeps running on the new schema.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:33:11 +03:00
ArtyomandClaude Opus 5.5 bf0f0b4c07 Name the serving replica in every response
The platform settings default the instance id to the container
hostname, computed once per process, so every gunicorn worker of a
replica reports the same value and it matches docker ps. The request
pipeline stamps X-Instance-Id next to X-Request-Id on every response
it sends, and the unhandled error handler adds it itself, because the
500 answer is produced outside the pipeline.

Own tests cover 200, 204, 400, 401, 404, the hidden 405, 413 and 500
with a fixed instance id, and the hostname default. The application
version follows the contract version 1.3.0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:32:36 +03:00
ArtyomandClaude Opus 5.5 9db7a24362 Propose load-balanced replicas and idempotent messages as an openspec change
Plan the grade 3 cycle of lab 3 on top of the carried-over lab 2
solution: run the API as several replicas of one compose service
behind HAProxy as the only published entry point, stamp every API
response with X-Instance-Id taken from the container hostname, require
Idempotency-Key on message posts and keep the key and the message in
one PostgreSQL transaction, and add explicit stop-one, restart and
restore hooks.

The design fixes the balancer settings and retry policy, the
idempotency table, the lookup order under the chat row lock, the
request fingerprint and the shutdown budget. Delta specs add the
lab3-load-balancing and lab3-idempotent-messages capabilities and
modify message posting, request identification, shutdown, container
delivery, migrations, restart and the reproducible stand.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:31:41 +03:00
ArtyomandClaude Opus 5.5 416ce4f9db Carry over the lab 2 solution
Bring the application, own tests, Dockerfile, toolchain, compose
stack, migrations, secret generator, restart hook, evidence scripts and
CI job from the lab 2 solution commit 8cddb09 unchanged, merge the lab
2 Makefile targets into the lab 3 skeleton Makefile without touching
its targets, and append the lab 2 solution variables to the skeleton
.env.example. The package version follows the contract version 1.3.0.

This is the starting point of the lab 3 cycle: lint, own tests and the
offline skeleton check pass before any lab 3 change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:26:15 +03:00
ArtyomandClaude Opus 5.5 0ec2f9dae3 Carry over the agent context and openspec history from lab 2
Bring the Claude Code settings, the lint hook, the opsx commands and
skills, the architecture notes, the lab 1 and lab 2 specs and the
archived lab 1 and lab 2 changes into the lab 3 skeleton, so lab 3
continues the same spec history instead of starting from scratch.

Rewrite AGENTS.md and the openspec context and rules for lab 3:
HAProxy in front of several stateless API replicas, X-Instance-Id on
every response, a mandatory Idempotency-Key for messages shared by all
replicas through PostgreSQL, explicit stop-one and restart hooks, Out
of scope moved to labs 4-7 plus Swarm and HA storage, and rules on
retry policy, timeout chains and replica-local state.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:23:53 +03:00
Mikhail Verkovykh 0ab3a341aa Подготовить skeleton лабораторной 3 по мессенджеру 2026-09-10 18:01:35 +03:00
3319 changed files with 299982 additions and 58598 deletions
-2
View File
@@ -42,5 +42,3 @@ compose*.yaml
lab.json
requirements-dev.txt
.claude
.verification/
+9 -58
View File
@@ -11,24 +11,13 @@ SESSION_STORE=postgres
CACHE_URL_FILE=/run/secrets/cache_url
API_REPLICAS=2
IDEMPOTENCY_TTL_SECONDS=86400
S3_ENDPOINT=http://s3:9000
S3_REGION=us-east-1
S3_BUCKET=messenger-attachments
S3_ACCESS_KEY_ID_FILE=/run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE=/run/secrets/s3_secret_access_key
MAX_UPLOAD_BYTES=10485760
MAX_IMAGE_PIXELS=40000000
BROKER_URL_FILE=/run/secrets/broker_url
WORKER_CONCURRENCY=2
JOB_MAX_ATTEMPTS=5
JOB_TIMEOUT_SECONDS=60
STAGING_RETENTION_SECONDS=86400
# Solution-specific: published LB ports, replica limits, workers and pool per replica,
# LB queue per replica, drain window before closing the listener on SIGTERM, PostgreSQL
# limits, load generator limits (profile bench), cursor key file. API_PORT/SHORTTTL_PORT of lab2 are gone:
# limits, load generator limits (profile bench), CPU pinning for track 1 measurements (empty = not
# pinned), cursor key file. API_PORT/SHORTTTL_PORT of lab2 are gone:
# only the load balancer is published.
LB_HTTP_PORT=18085
LB_STATS_PORT=25085
LB_HTTP_PORT=8080
LB_STATS_PORT=8404
API_CPUS=1
API_MEMORY=512m
WEB_CONCURRENCY=2
@@ -40,47 +29,9 @@ PG_CPUS=2
PG_MEMORY=1g
K6_CPUS=2
K6_MEMORY=512m
CURSOR_SECRET_FILE=/run/secrets/cursor_secret
# Solution-specific for lab4: S3 client timeouts (connect + read, two attempts stay below
# REQUEST_TIMEOUT_SECONDS=8), parallel image checks per worker, RustFS memory limit.
S3_CONNECT_TIMEOUT_SECONDS=1
S3_READ_TIMEOUT_SECONDS=2
IMAGE_CHECK_CONCURRENCY=2
S3_MEMORY=512m
# Solution-specific for lab4 grade 4: upload bodies buffered at once per worker, grace period
# before an abandoned pending upload is swept, sweep interval and rows per sweep pass.
UPLOAD_CONCURRENCY=3
UPLOAD_GRACE_SECONDS=900
UPLOAD_SWEEP_INTERVAL_SECONDS=30
UPLOAD_SWEEP_BATCH=100
# Fixed glibc mmap threshold for the API: buffers above 128 KiB are returned to the system on free.
MALLOC_MMAP_THRESHOLD_=131072
# Solution-specific for lab5: RabbitMQ management UI on 127.0.0.1, backoff base of a transient
# retry (doubles per attempt up to 60 s), relay poll and sweep intervals, worker limits and the
# PostgreSQL pool of relay and worker. FAULT_POINT is set only by infra/compose.faults.yaml.
BROKER_MANAGEMENT_PORT=25086
JOB_BACKOFF_BASE_SECONDS=2
OUTBOX_POLL_SECONDS=1
SWEEP_INTERVAL_SECONDS=5
WORKER_CPUS=2
WORKER_MEMORY=1g
PIPELINE_DB_POOL_MAX_SIZE=4
# Solution-specific for lab5 grade 4: render time per job (the render process is killed after it),
# address space of one render process (WORKER_MEMORY >= WORKER_CONCURRENCY * RENDER_MEMORY_MB plus
# ~120 MiB of the worker itself), jobs per render process before it is replaced.
RENDER_TIMEOUT_SECONDS=30
RENDER_MEMORY_MB=448
RENDER_TASKS_PER_CHILD=100
# Minimal client transfer rate, bytes/s: every body byte moved in either direction extends the
# request deadline by 1/rate, so slow but steady uploads and downloads outlive REQUEST_TIMEOUT_SECONDS.
MIN_TRANSFER_RATE_BYTES=131072
# Solution-specific for lab4 track 2: limits of the one-off reconciler (profile reconcile), its S3
# read timeout for listing pages, optional CPU pinning of the measured services (empty = no pinning)
# and the CPU limit of RustFS (make reconcile-cost raises it with its memory for the 100k point).
RECONCILE_CPUS=1
RECONCILE_MEMORY=256m
RECONCILE_S3_READ_TIMEOUT_SECONDS=10
RECONCILE_CPUSET=
S3_CPUSET=
API_CPUSET=
LB_CPUSET=
PG_CPUSET=
S3_CPUS=1
VALKEY_CPUSET=
K6_CPUSET=
CURSOR_SECRET_FILE=/run/secrets/cursor_secret
+1 -1
View File
@@ -1,4 +1,4 @@
## Лабораторная 5
## Лабораторная 3
Автор / группа: TODO
Целевая оценка (3/4/5): TODO
+2 -42
View File
@@ -19,7 +19,7 @@ jobs:
- run: make student-test VENV=.venv
acceptance:
name: Стенд Compose с RabbitMQ, relay и worker за HAProxy, smoke, pipeline, отказ реплики, рестарт и отказы зависимостей
name: Стенд Compose за HAProxy, smoke, отказ реплики, рестарт и отказы зависимостей
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
@@ -34,28 +34,6 @@ jobs:
- name: Публичный smoke через балансировщик
run: make test
- name: Топология брокера, consumer с prefetch и права клиентов
run: make topology
- name: Уровни «3» и «4» pipeline — worker остановлен, рестарт брокера, дубль, S3, попытки, варианты и пределы, падения в окнах 2–5, аренда, replay, DLQ, jitter, retention
run: |
make suite-preflight
make level4
- name: Сообщение задания в очереди, 409 до ready и SIGTERM relay и worker
run: make pipeline-evidence
- name: Вложение после остановки принявшей его реплики и при недоступном S3
run: |
make scenario SCENARIO=failover-upload
make scenario SCENARIO=s3-outage
- name: Приватный bucket, ограниченный ключ API и повторный bootstrap
run: make s3-check
- name: Негативный suite вложений, компенсация, уборка после grace, обрыв и зависание загрузки
run: make upload-faults
- name: Несколько реплик на одном токене и гонка одного ключа
run: make lb-check
@@ -82,8 +60,7 @@ jobs:
- name: Наружу опубликован только балансировщик
run: |
docker compose ps --format '{{.Service}} {{.Ports}}' | awk '$2 != "" && $1 != "lb" && $1 != "broker" {print; bad=1} END {exit bad}'
docker compose ps --format '{{.Service}} {{.Ports}}' | awk '$1 == "broker" && $2 !~ /^127\.0\.0\.1:/ {print; bad=1} END {exit bad}'
docker compose ps --format '{{.Service}} {{.Ports}}' | awk '$2 != "" && $1 != "lb" {print; bad=1} END {exit bad}'
- name: Процессы работают не от root
run: |
@@ -91,23 +68,6 @@ jobs:
docker top "$id" -eo pid,uid,comm | awk 'NR>1 && $2==0 {print; bad=1} END {exit bad}' || exit 1
done
- name: Одноразовые сервисы не от root
run: |
for service in migrate s3-init; do
user=$(docker inspect -f '{{.Config.User}}' "$(docker compose ps -aq "$service")")
echo "$service: $user"
case "${user%%:*}" in ""|0|root) exit 1 ;; esac
done
- name: Корректная остановка relay и worker по SIGTERM
run: |
start=$(date +%s)
docker compose stop -t 10 relay worker
elapsed=$(( $(date +%s) - start ))
codes=$(docker inspect -f '{{.State.ExitCode}}' $(docker compose ps -aq relay worker) | sort -u)
echo "остановка за ${elapsed} с, коды завершения ${codes}"
test "$elapsed" -le 10 && test "$codes" = "0"
- name: Корректная остановка реплик API по SIGTERM
run: |
start=$(date +%s)
-2
View File
@@ -38,5 +38,3 @@ coverage/
# Локальные настройки агента
.claude/settings.local.json
.verification/
+80 -61
View File
@@ -1,90 +1,109 @@
## Что это за репозиторий
Skeleton лабораторной 5 курса «Мессенджер под нагрузкой» (МТУСИ, 7 независимых лаб) и решение в ветке `solution/bpi2303-kostylev`. Skeleton даёт задание (`README.md`), нормативный контракт (`contracts/`) и публичные black-box проверки (`tests/`). Реализация API, worker, relay, Dockerfile, compose-стенд, конфигурация балансировщика, S3 и брокера, миграции, хуки отказов и собственные тесты — студенческие; решение лабы 5 построено переносом решения лабы 4.
Skeleton лабораторной 3 курса «Мессенджер под нагрузкой» (МТУСИ, 7 независимых лаб) и решение в ветке `solution/bpi2303-kostylev`. Skeleton даёт задание (`README.md`), нормативный контракт (`contracts/`) и публичные black-box проверки (`tests/`). Реализация API, Dockerfile, compose-стенд, конфигурация балансировщика, миграции, хуки отказов и собственные тесты — студенческие; решение лабы 3 построено переносом решения лабы 2.
Лаба 5: приём вложения отделён от тяжёлой обработки. `POST /api/v1/chats/{chat_id}/attachments` проверяет файл как в лабе 4, кладёт оригинал в приватный префикс `staging/` bucket **RustFS** и одной транзакцией PostgreSQL фиксирует вложение `queued`, задание (`jobs`) и строку **transactional outbox**; только после этого отвечает `202` со снимком `queued`. Отдельный сервис `relay` публикует outbox в **RabbitMQ 4.1** (quorum queue, DLX, publisher confirms, `mandatory`, persistent), отдельный сервис `worker` (тот же образ, свой entrypoint) потребляет очередь через **FastStream** поверх aio-pika с ручным `ack` строго после `COMMIT` результата, захватывает задание арендой (`lease_token`), публикует оригинал в `originals/` и строит варианты Pillow. Источник истины состояний — PostgreSQL, брокер — транспорт и будильник. Остаётся всё из лаб 1–4: HAProxy перед ≥2 репликами, `X-Instance-Id`, `Idempotency-Key`, Basic → Bearer, хуки остановки реплики и рестарта стенда.
Лаба 3: несколько взаимозаменяемых реплик API за единой точкой входа — **HAProxy**. Реплики не держат состояния: PostgreSQL остаётся источником истины для пользователей, сессий, чатов и сообщений, Valkey — общий кэш сессий. Каждый ответ API несёт `X-Instance-Id` обслужившей реплики, отправка сообщения требует `Idempotency-Key`, а повтор с тем же ключом через любую реплику не создаёт дубль. Остановка одной реплики не прерывает чтение и запись дольше 10 с.
Текущий доступ к PostgreSQL — **SQLAlchemy 2.0.45 async**, `postgresql+asyncpg`, `async_sessionmaker`, ORM/Core и `DeclarativeBase`; миграции — **Alembic 1.16.5**, каталог `alembic/versions/`, async `alembic/env.py`. Compose-service `migrate` выполняет `alembic upgrade head` под advisory lock с проверкой legacy-схемы. Сырой asyncpg и прежние SQL-миграции не являются текущей архитектурой.
Автор решения — Костылев Артём, БПИ2303. Уровни «3» → «4» → трек «5» реализованы последовательно,
после них внесены исправления ревью. Код итогового ревью — `8c9ec653ed9d0722bccecd4690def220a803c969`;
активных OpenSpec changes нет. Финальная validation выполнена на checkout `b063c8780eabf63cdb9da3802f82440750308d6b`,
с тем же кодом review. Публикация требует отдельного сообщения координатора. Границы доказательств:
- `evidence/load-validation.md` — аудит применимости: 135 прежних опытов приняты, полный трек повторять запрещено.
- `evidence/final-validation/` — чистый clone, 372 student-tests, 12 stack-tests, публичные сценарии,
реальные регрессии кэша/NUL/Basic/HAProxy, detection, failover/rolling и 9 текущих after 8×1.
2400/3600 RPS проходят прежние границы, 5400 насыщен по drops; HTTP-ошибок нет.
Стенд остановлен, volumes сохранены. Изменения итогового коммита — только docs/evidence.
- `evidence/track1-sqlalchemy/revalidation/` — уровни «3»/«4» **до исправлений ревью**, код `3fb741e`;
API того прогона идентичен `57744c5`. Позднее изменены сессии, валидация и шаблоны HAProxy.
- `evidence/track1-sqlalchemy/` — 135 измерений с тремя повторами и 12 графиков;
baseline `5c8f605`, after `57744c5`; это измерения до ревью, не производительность текущего HEAD.
- `evidence/grade3-*.txt`, `grade4-*.txt` — этапы SQLAlchemy до оптимизации;
`evidence/track1/` и файлы, отмеченные в индексе как raw asyncpg, — исторический код.
- `evidence/review-fixes/`, `review-nul-inputs/`, `review-non-ascii-basic/` — исправления сессий/LB,
NUL и не-ASCII Basic, offline-доказательства и обязательные сценарии для финального агента.
- `evidence/report-after-review.md` — проверки текущих документов и оставшийся minor;
`evidence/report-checks.txt` и `report-audit.txt` — первый отчёт `4513d90` до ревью.
- `openspec/config.yaml` сохраняет контекст прежних измерений `3fb741e`; текущий этап не имеет
права менять конфиги/OpenSpec. Для актуального статуса использовать REPORT.md и этот файл.
## Команды
```sh
make check # offline: структура skeleton, JSON, локальные $ref, синтаксис Python, версия 1.5.0
make validate PYTHON=.venv/bin/python # полная валидация OpenAPI 3.1 и jobs.schema.json (venv с requirements-dev.txt)
make check # offline: структура skeleton, JSON, локальные $ref, синтаксис Python, X-Instance-Id в контракте
make validate # полная валидация OpenAPI 3.1: openapi-spec-validator из .venv (make install); PYTHON= — свой интерпретатор
make install # uv sync --group dev
make lint # ruff check + ruff format --check + ty check; обязательно перед commit
make student-test # собственные тесты (одноразовые PostgreSQL, Valkey, RustFS и RabbitMQ в Docker)
make student-test # собственные тесты (поднимают одноразовые PostgreSQL и Valkey в Docker)
make student-test PYTEST_ARGS='-v -k drain' # выборка тестов с именами
openspec validate --all --strict
make secrets # локальные secrets/* (PostgreSQL, Valkey, S3 root/API/worker, RabbitMQ) и .env
make up # docker compose build, затем up -d --wait (relay и worker пересоздаются на новом образе): хранилища и broker → migrate, s3-init → api ×2, relay, worker → lb
make test # публичный smoke через LB (BASE_URL=http://127.0.0.1:18085)
make test-persistence ACTION_SCRIPT=scripts/restart.sh # рестарт lb, api, relay, worker и postgres, valkey, s3, broker с volumes
make secrets # локальные secrets/* и .env из .env.example (не коммитятся)
make up # docker compose up -d --build --wait: postgres, valkey, migrate, api ×API_REPLICAS, lb; затем ждёт все реплики UP в LB
make test # публичный smoke через LB (BASE_URL по умолчанию — docker compose port lb 8080, без стенда http://localhost:8080)
make test-persistence ACTION_SCRIPT=scripts/restart.sh # рестарт lb, api, postgres, valkey с сохранением volumes
make test-failover ACTION_SCRIPT=scripts/stop-one.sh # остановка реплики TARGET_INSTANCE_ID под запросами
make test-faults # тесты стенда лабы 2 через LB (make faults-evidence → evidence/faults.txt)
make restore # вернуть остановленные реплики и дождаться их в ротации LB
make worker-stop / worker-start # демонстрация «worker недоступен»
make queues # очереди брокера: ready, unacked, consumers
make level3 # сценарии уровня «3» suite (scripts/pipeline_suite.py) → evidence/level3.json
make level4 # сценарии уровней «3» и «4» suite и инварианты → evidence/level4.json
make track2 TRACK2_OUTPUT=evidence/local-track2.json # 13 сценариев и 8 инвариантов трека 2
make reprocess ID=… PIPELINE_VERSION=2 # новая версия обработки доступного исходника
make failed / make replay ID=… / make dlq # реестр failed, явный replay, просмотр DLQ (python -m messenger.admin, rabbitmqadmin)
make down # остановка своего project без удаления постоянных volumes
make lb-check # ≥2 X-Instance-Id на одном токене и 20 гонок одного Idempotency-Key
make retry-check # потерянный ответ (запись ждёт блокировку чата) на живой и на убитой реплике, повтор тем же ключом
make limits # уровень 4: лимиты, порты, секреты, volumes, сети против политики scripts/limits.py
make detection # уровень 4: время обнаружения SIGTERM/kill/pause/PostgreSQL против заявленного максимума
make failover-each # уровень 4: каждая реплика по очереди под k6 (RATE=100) → evidence/failover-each
make rolling-update [FROM_REV=<коммит>] # уровень 4: замена реплик по одной под k6, migrate один раз
make track1 # трек 1: снимок хоста, серии A (1x4/2x2/4x1) и B (1x1/2x1/8x1), до/после на 8x1, контроль, графики; volumes сохраняются
make track1-control # трек 1: пул 4/8/16 и maxconn 16/32/64 на 8x1; 8x0.5 против 4x1 на одних CPU
make plots # трек 1: графики и сводка потолков из evidence/track1-sqlalchemy/*/summary.csv
make down # остановка без удаления данных (с профилем bench); make reset-data — удаление volumes
```
`tests/*.py` и `scripts/*.py` — только stdlib, синтаксис Python 3.9 (`make check` разбирает их системным `python3`). Проверки сами не поднимают стенд. Хуки `scripts/stop-one.sh` и `scripts/restart.sh` запускаются тестом без shell, поэтому исполняемые и с shebang. Данные не удаляются автоматически: каждый прогон smoke создаёт новых синтетических пользователей и вложения.
`tests/*.py` и `scripts/*.py` — только stdlib, синтаксис Python 3.9 (`make check` разбирает их системным `python3`). Проверки сами не поднимают стенд. Хуки `scripts/stop-one.sh` и `scripts/restart.sh` запускаются тестом без shell, поэтому исполняемые и с shebang. Данные не удаляются автоматически: каждый прогон smoke создаёт новых синтетических пользователей.
Стенд запускать только в project `mtusi2026lab3` основного репозитория либо временном
`COMPOSE_PROJECT_NAME` с префиксом `hlpipe-lab3-` и working_dir под основным репозиторием
или `/Users/artemkostylev/projects/learning/HL/labs/.pipeline/.runs/labs`, с `LB_HTTP_PORT=18083`,
`LB_STATS_PORT=23083` и `NO_PROXY=localhost,127.0.0.1,::1` (также `no_proxy`). Остальные
host-порты — 23000–23999. Проверять занятость до запуска; чужие стенды не останавливать.
Полный трек требует 16 vCPU Docker VM. `make detection` перезаписывает свой каталог evidence:
для дополнительного прогона сохранять старое сырьё и задавать новый путь скрипту.
## Архитектура skeleton
- `lab.json` — единственный источник номера лабы (5). От него зависят `scripts/check_contract.py` (`info.version == 1.5.0`) и `tests/client.py` (ветки `LAB>=5`: upload отвечает `202 queued`, сообщение с вложением сразу, ожидание `ready` до 60 с от второго участника, ровно один вариант `thumbnail` 256×160 настоящим JPEG).
- `tests/` байт в байт как в лабе 4; поведение переключает только `lab.json`. `tests/resilience.py` после хука рестарта запрашивает метаданные и оригинал один раз с таймаутом 2 с.
- `contracts/jobs.schema.json` — формат сообщения брокера (7 обязательных полей, без bytes, URL, ключей S3 и filename); `contracts/attachment-states.md` — машина состояний и окна отказа.
- `contracts/infrastructure.json` — роли `api`, `postgres`, `load_balancer`, `s3`, `broker` (persistent volume, at-least-once), `worker` (отдельный процесс). `compose.example.yaml` — только worksheet ролей.
- `lab.json` — единственный источник номера лабы (3). От него зависят `scripts/check_contract.py` (`info.version == 1.3.0`, `X-Instance-Id` во всех ответах) и `tests/client.py` (с lab3 проверяет непустой `X-Instance-Id` на любом ответе и шлёт `Idempotency-Key` в каждом POST сообщения).
- `tests/resilience.py --mode failover`: `TARGET_INSTANCE_ID` — реплика, обслужившая первое сообщение; 40 `GET /users/me` одним токеном должны дать ≥2 `X-Instance-Id`; hook возвращается <10 с с кодом 0; первая запись после hook — не позже 10 с от его старта; ≥5 записей после; target отсутствует в последних 5 ответах; история без дублей и байт-в-байт равна подтверждённым ответам. Таймаут клиента 2 с, после ошибки повтор с тем же ключом и текстом.
- `tests/resilience.py --mode persistence`: hook ≤90 с, затем `/health/ready` и `/users/me` прежним токеном до 90 с, сверка истории.
- `contracts/infrastructure.json` — роли `api` (≥2, без локального состояния), `postgres` (persistent volume), `load_balancer` (единственная публичная точка, readiness backend), опциональный `redis_or_valkey`. `compose.example.yaml` — только worksheet ролей.
## Ключевые правила контракта (lab5)
## Ключевые правила контракта (lab3)
- Upload: те же лимиты и коды, что в лабе 4 (`413`, `415`, `400`, `404`), но успех — `202 Attachment` со снимком `status=queued`, `variants=[]`, `error_code=null`; снимок берётся из записанного объекта, а не перечитывается после публикации. `202` — только после записи оригинала в `staging/` и `COMMIT` вложения, задания и outbox.
- Состояния: `queued → processing → ready | failed`; `processing → queued` при временной ошибке или истёкшей аренде; `failed → queued` только явным replay. `ready` не откатывается. `ready` выставляется, только когда записаны оригинал и все обещанные уровнем варианты.
- `/content` для `queued`, `processing`, `failed` — `409 conflict` для любого `variant`, включая `original`; неизвестный `variant` — `400`; отсутствующий вариант `ready`-вложения — `404`; посторонний — `404`. Варианты: `thumbnail` JPEG вписан в 256×256 без увеличения, EXIF orientation применяется, alpha на белом, размеры вниз, минимум 1 px.
- Сообщение: вложение `queued`/`processing` прикрепляется (`201`), `failed` — `409`, вложение другого чата или неизвестное — `404`.
- Сообщение брокера — ровно поля `jobs.schema.json`; `job_id` не меняется при redelivery, `attempt` растёт только при управляемом повторе через outbox. Ключ эффекта — `(attachment_id, pipeline_version, variant)`.
- Сохраняются правила lab4 (сырые байты, тип по содержимому, скачивание только через API без redirect), lab3 (`X-Instance-Id`, `Idempotency-Key`, LB не повторяет upload), lab2 (`Basic` только на входе, `Bearer` дальше, `WWW-Authenticate` по пути) и lab1 (формат ошибки, посторонний → `404`, keyset `(created_at, id)`, время RFC 3339 с `Z`).
- `X-Instance-Id` и `X-Request-Id` — на каждом ответе API, включая health, `204` logout, `404/405` неизвестного маршрута и `500`. `X-Instance-Id` один на реплику (контейнер), стабилен на жизнь процесса и одинаков у всех воркеров gunicorn. Ответы, сгенерированные самим LB, его могут не иметь.
- `POST /api/v1/chats/{chat_id}/messages` требует `Idempotency-Key` (`^[A-Za-z0-9._:-]{1,128}$`); нет или неверный — `400`. Область ключа — пользователь + метод + путь + ключ, результат хранится ≥24 ч (`IDEMPOTENCY_TTL_SECONDS`). Повтор того же JSON — тот же `201` и тот же `Message` (включая строку `created_at`), другой JSON — `409`. Повтор заново проверяет сессию (`401`) и доступ (`404`). Сохраняется только `201`; запись сообщения и ключа — одна транзакция PostgreSQL.
- LB и клиент не повторяют прозрачно `POST /users` и `POST /chats`. Балансировка без sticky-сессий.
- Сохраняются правила lab2: `Basic` только на `POST /api/v1/auth/sessions`, иначе `Bearer`; `WWW-Authenticate` по пути; `X-User-Id` игнорируется; logout `204`, повтор `401`; абсолютный TTL сессии. И lab1: формат ошибки `{"error":{code,message,request_id}}`, посторонний → `404`, keyset `(created_at, id)`, `limit` 1–100, cursor привязан к чату, время RFC 3339 с `Z`.
## Правила работы с репозиторием
- Не менять и не ослаблять `contracts/`, `tests/`, `README.md`, `COURSE.md`, рубрику. Расхождение оформляется вопросом преподавателю.
- Код — `src/messenger/`, миграции — `alembic/versions/`, конфигурация LB, нагрузки, S3 и брокера — `infra/`, хуки и обвязка стенда — `scripts/`, собственные тесты — `student-tests/`, обезличенные доказательства — `evidence/`.
- Код — `src/messenger/`, миграции — `alembic/versions/*.py`, конфигурация LB и нагрузки — `infra/`, хуки и обвязка стенда — `scripts/`, собственные тесты — `student-tests/`, обезличенные доказательства — `evidence/` (индекс — `evidence/README.md`, первая строка каждого файла — дата и коммит прогона).
- PostgreSQL — SQLAlchemy 2.0.45 async (`async_sessionmaker`, `DeclarativeBase`), Alembic 1.16.5; версии закреплены в `pyproject.toml` и `uv.lock`, asyncpg только драйвер. ORM/Core остаются в инфраструктуре, домен — dataclass, pydantic — на HTTP-границе. Миграции — только `alembic upgrade head` в one-shot сервисе `migrate`. Старую схему принимает `alembic/legacy.py` после проверки журнала и каталога PostgreSQL; безусловный `stamp` запрещён. Старый журнал сохраняется, volumes не удаляются.
- Без комментариев в коде; отступы — пробелы (`[tool.ruff.format]`); Python 3.14 без `from __future__ import annotations` и `TYPE_CHECKING`.
- Порты реплик API, S3, AMQP наружу не публикуются: только LB и UI брокера на `127.0.0.1`. API к брокеру не подключается. Клиенты брокера не объявляют очереди: топология — в `infra/rabbitmq/definitions.json`, права configure есть только у оператора.
- `REPORT.md`: команды из чистого clone, архитектура, таблица «критерий → команда → наблюдение → файл», таблица отказов. PASS без команды не доказательство; для отказов — какой контейнер реально остановлен (`docker ps -a`).
- Ветка `solution/bpi2303-kostylev`, коммиты на английском, перед каждым — `make lint`. Никогда не коммитить `.env`, `secrets/`, токены, ключи S3, пароли брокера, volumes, дампы.
- Проверять документацию используемых библиотек через Context7. Изменения реализации проходят OpenSpec propose/apply/archive. `openspec/config.yaml` уже описывает SQLAlchemy/Alembic и ЛР5; оставшееся в design правило `201` относится к прежней загрузке ЛР4, нормативный успех ЛР5 — `202`.
- Автор работы — Костылев Артём, БПИ2303. Не приписывать авторство Claude. Публикация только в private MTUCI-Labs и только назначенным агентом фазы PUBLICATION; вне неё не делать push и не отправлять Telegram.
## Порядок этапов и Docker
- Только runner управляет агентами: один переносит основу и последовательно реализует 3 → 4 → 5 с evidence, затем отдельные review → fix только major → re-review, отчёт, один агент нагрузки и публикации. Review/отчёт предыдущей лабы могут идти параллельно реализации следующей. Самостоятельно фоновых агентов не запускать, runner/state не менять.
- Major — доказанная угроза безопасности, потеря/повреждение данных, недоступность сервиса или провал обязательного сценария. Остальное — minor: сохранять в findings и ограничениях REPORT, не исправлять без назначения и не блокировать этапы. Обязательный FAIL нельзя объявлять PASS.
- Менять только назначенную лабу; другую читать лишь через предоставленный snapshot. Перед итоговым review переносить применимые поздние исправления. Сохранять WIP; `reset/clean` и удаление пользовательских данных запрещены.
- Разрешены основной project `mtusi2026lab5` в основном репозитории или `hlpipe-lab5-*` с working_dir под ним либо `/Users/artemkostylev/projects/learning/HL/labs/.pipeline/.runs/labs`. API — 18085, остальные порты — 25000–25999; текущие значения stats/UI — 25085/25086. Учитывать Compose env, не печатать секреты.
- Обычные Docker-этапы работают совместно на отдельных project/портах/volumes; финальная нагрузка получает эксклюзивный доступ через runner. Останавливать только свой стенд, в конце — без `-v`. Фон и условия замеров фиксировать; пригодные серии сохранять, повторять только отсутствующие, затронутые или непригодные.
- Отчётная фаза меняет только `REPORT.md`, `AGENTS.md`, `evidence/**/*.md`; не меняет код, конфиги, OpenSpec и сырые измерения. Docker, динамические/нагрузочные прогоны и полный `make student-test` в этой фазе запрещены; лёгкие check/validate/lint и OpenSpec validation допустимы.
## Состояние доказательств после ревью
Индекс — [evidence/README.md](evidence/README.md), отчёт — [REPORT.md](REPORT.md). Текущий код после исправления памяти и независимого re-review — `6096aa7948b8b666aa3892397a96d044a55f02c6` (M). Последняя полная динамическая квалификация — `0e921116610d289716af0a40ec823376eb8bbcbc` (D); предыдущее review — `42575281b352d35bda17ee7601bae5548a996e91` (R). От D к R менялись renderer PNG и policy RabbitMQ; от R к M из runtime изменён только renderer. Старые PASS не обозначать прогоном M.
Validation на `b30e2f6a84419d1ca64525ffb40df92183cccaef` (V, runtime = R) завершила реальный PostgreSQL outage: 12 requeue → ready без replay, policy `delivery-limit=-1` на сохранённом volume, DLQ не выросла. HTTP PNG: 8 fixtures / 48 сравнений вариантов v1/v2 PASS. Delivery subset: 12 сценариев / 8 инвариантов PASS. Student-test: 980 passed / 1 Linux-only skip. Исторический аудит — [evidence/load-validation.md](evidence/load-validation.md), результаты V — [evidence/final-validation-2026-11-07/RESULTS.md](evidence/final-validation-2026-11-07/RESULTS.md).
**Прежний major закрыт динамически на F `74c33160ac01618f70e9e1c6a121a5e45c3c7155` (runtime = M).** Четыре исходных PNG 8000×5000 и grayscale16 без tRNS прошли Linux с импортами worker при 448 MiB/30 с и HTTP 202 → ready в v1/v2. Два одновременных HTTP jobs: peak 844.41 MiB / 1 GiB, OOM=0. Boundary: 8 сценариев / 8 инвариантов, 11 ID / 59 объектов; малая HTTP-матрица: 8 fixtures / 48 сравнений / 8 инвариантов, 8 ID / 56 объектов. Принудительные лимиты, обычное задание после каждого отказа и cleanup после sweep — PASS. Student-test: 984 passed / 2 Linux-only skipped / 19 deselected; Linux-эквивалент выполнен отдельно. Исходные FAIL V сохранены, лимиты не увеличивались.
Актуальные команды и сырьё — [evidence/final-memory-validation-2026-11-07/RESULTS.md](evidence/final-memory-validation-2026-11-07/RESULTS.md), аудит применимости — [evidence/load-validation.md](evidence/load-validation.md). Полные track2, PostgreSQL outage и delivery subset приняты без повторения: соответствующий runtime не менялся. Стенд F остановлен, четыре volumes сохранены. Validation завершена; push/Telegram не выполнялись, публикация требует отдельного сообщения координатора с точным SHA. Документационные коммиты после F не меняют измеренный runtime.
Семь minor последнего review сохраняются в REPORT: crash_publish_confirm фактически после confirm до COMMIT outbox; limits.py имеет ложный FAIL порядка secrets; reconcile_track.py ожидает upload 201 ЛР4; failed upgrade v2 нельзя отдельно повторить при ready v1; UTC/elapsed run2/3 расходятся; design-правило config сохранило 201; JobMessage допускает преобразования типов и null вне jobs.schema.json. Последний minor воспроизведён reviewer в четырёх offline-пробах (bus/ack вместо reject); штатный relay формирует корректные сообщения, критичного последствия не установлено. Происхождение — [аудит обновления отчёта](evidence/report-memory-audit-2026-11-07.md). Корневые старые evidence до SQLAlchemy остаются историей.
- Порты реплик API наружу не публикуются: только LB. Нет `container_name`, `hostname` и общего `INSTANCE_ID` у сервиса `api`.
- `REPORT.md`: команды из чистого clone, архитектура, таблица «критерий → команда → наблюдение → файл», таблица отказов, трек 1, ответы на вопросы защиты. PASS без команды не доказательство; для отказов — какой контейнер реально остановлен (`docker ps -a`). Цифры в отчёте берутся только из файлов `evidence/`.
- Старые результаты сохранять с SHA и пометкой версии кода; не смешивать raw asyncpg, baseline SQLAlchemy и SQLAlchemy с кэшем компиляции. Микробенчмарк компиляции не заменяет HTTP-замер, пик RPS не равен пределу SLO. Rolling с no-op миграцией не доказывает произвольный DDL под нагрузкой.
- Ветка `solution/bpi2303-kostylev`, коммиты на английском, перед каждым — `make lint`. Никогда не коммитить `.env`, `secrets/`, токены, volumes, дампы.
- ЛР1/ЛР2 заморожены; менять только текущую лабораторную, WIP и volumes сохранять. Текущую `.pipeline` и её state не менять. Публикация — отдельный этап, только private в MTUCI-Labs; Telegram отправляет runner.
- Порядок runner: один агент переноса/реализации уровней 3→4→5 и evidence → review (fix и re-review при major) → отдельный report → один агент нагрузки и публикации. Ревью/отчёт предыдущей лабы могут идти параллельно реализации следующей; агенты не запускают собственные цепочки. Другие репозитории читать только через предоставленные snapshots, поздние исправления переносить до итогового ревью.
- Исправлять только критичное: major — доказанная угроза безопасности, потери/повреждения данных, недоступность или неработающий обязательный сценарий. Остальное — minor: сохраняется в findings и ограничениях REPORT, не требует fix и не блокирует следующие этапы. Провал обязательной проверки нельзя объявлять успехом.
- На этапе report разрешены только REPORT.md, AGENTS.md и `evidence/**/*.md`; без Docker, динамических/нагрузочных прогонов и тяжёлых тестов. Существующие измерения и конфиги не менять; `make student-test` относится к следующей разрешённой фазе.
## Уровни оценки
Цель — **«5»**, трек 2 «Доказанная идемпотентность pipeline». Критерии подтверждены совокупностью сохранённых серий D/V и адресной validation F; прежний major закрыт, семь minor сохранены. Таблица доказательств и границы применимости — в `REPORT.md`.
Цель — **«5»** с треком 1 «Цена масштабирования»; таблица доказательств — в `REPORT.md`.
Прежние 135 измерений приняты аудитом; исправленный HEAD имеет отдельные динамические
проверки в final-validation. Полную матрицу не переснимать: повторять только затронутые/невалидные
сценарии с причиной в evidence/load-validation.md. Смена SHA документов сама по себе не причина. Открытый minor: одиночный суррогат в строке JSON может
дать 500 при fingerprint вместо 400; финальное ревью воспроизвело это offline, исправление не требуется этим этапом.
- **3** — `202 queued` после staging и `COMMIT` задания с outbox; отдельные `broker`, `relay`, `worker` в стенде; durable quorum queue, publisher confirm, ручной `ack` после `COMMIT`; состояния `queued/processing/ready/failed`, оригинал в `originals/` и thumbnail; ограниченный retry (`JOB_MAX_ATTEMPTS=5`) и `failed` с `error_code`; повтор job без второго набора вариантов; worker остановлен → upload → сообщение → рестарт API и broker → worker → `ready`; минимум lab4 сохранён.
- **4** — crop и optimized WebP, лимиты worker по байтам, пикселям, времени и памяти, crash между `COMMIT` и publish, backoff с jitter и классификация ошибок, DLQ и replay, crash после записи объекта до `ack`, lease и fencing для зависшего `processing`, prefetch, очистка staging.
- **5** — трек 2: `pipeline_version` 1 и 2, монотонная публикация, автоматический fault-injection (дубль, reorder, crash до и после side effect, устаревшая job) и проверка инвариантов одной командой.
- **3** — стенд в `compose.yaml`: HAProxy → ≥2 реплики API → общие PostgreSQL и Valkey; разные реплики на одном токене; остановка любой реплики с восстановлением ≤10 с; идемпотентная отправка, включая конкурентный повтор и смену реплики; `make test`, `make test-persistence ACTION_SCRIPT=scripts/restart.sh`, `make test-failover ACTION_SCRIPT=scripts/stop-one.sh`; минимум lab2 сохранён.
- **4** — согласованные readiness, таймауты и `SIGTERM`, период проверок и время обнаружения; поочерёдная остановка каждой реплики под нагрузкой с RPS, ошибками и временем восстановления; лимиты CPU/RAM, API без опубликованных портов; rolling update под нагрузкой и однократные совместимые миграции.
- **5** — трек 1: 1/2/4 реплики при фиксированном бюджете CPU/RAM и серия с ростом ресурсов, saturation point БД/LB/пула, графики и одно подтверждённое улучшение.
+1 -1
View File
@@ -1,4 +1,4 @@
# Памятка преподавателю — лабораторная 5
# Памятка преподавателю — лабораторная 3
## Что подготовлено
+38 -126
View File
@@ -10,26 +10,20 @@ COMPOSE ?= docker compose
RATE ?= 100
FROM_REV ?=
SHIFT ?= 0
SCRIPTS_PY = scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py \
scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py \
scripts/make_images.py scripts/pipeline_suite.py scripts/pipeline_evidence.py
SCENARIO ?= failover-upload
SCRIPTS_PY = alembic scripts/probe.py scripts/limits.py scripts/scaling.py tools
TRACK1 ?= evidence/track1-sqlalchemy
SCALING = $(PYTHON) scripts/scaling.py
BEFORE_REV ?= 5c8f605cd3bd40245109348463d3e7111379b2d3
BASELINE = --env API_IMAGE=mtusi-messenger:$(BEFORE_REV)
PSQL = $(COMPOSE) exec -T postgres psql -U messenger -d messenger -X -q
TRACK2_OUTPUT ?= evidence/track2.json
TRACK2_SCENARIOS = smoke_pipeline,duplicate_after_ready,crash_commit_publish,crash_publish_confirm,crash_after_s3_before_commit,crash_after_commit_before_ack,lease_expiry_fencing,crash_before_side_effect,worker_kill_mid_task,duplicate_in_flight,reorder_attempts,reorder_versions,stale_after_new_version
PIPELINE_VERSION ?= 2
PYTEST_ARGS ?=
.PHONY: help check validate test load test-persistence test-failover \
install lint format typecheck student-test \
secrets up down reset-data migrate ps logs restore lb-check failover-evidence stack-evidence \
persistence-evidence storage-evidence cache-evidence test-faults seed explain \
limits detection failover-each rolling-update lab4-level4 \
scenario s3-check attachments-evidence upload-faults upload-memory \
images queues topology worker-stop worker-start broker-restart suite-preflight level3 \
pipeline-evidence level4 replay failed dlq \
faults-evidence retry-check reconcile reconcile-demo reconcile-cost load-evidence \
clean-clone-evidence overload-leftovers
persistence-evidence storage-evidence cache-evidence test-faults faults-evidence seed explain retry-check \
limits detection failover-each rolling-update level4 \
track1 track1-images track1-host track1-a track1-b track1-compare track1-control plots
.NOTPARALLEL:
@@ -38,23 +32,9 @@ help:
@echo "install/lint/typecheck/student-test: студенческий тулчейн"
@echo "secrets/up/down/migrate/ps/logs: стенд Docker Compose; reset-data: намеренное удаление volumes"
@echo "restore: вернуть остановленные реплики; lb-check: реплики на одном токене и 20 гонок одного ключа"
@echo "lab4-level4 = limits, lb-check, detection, failover-each, rolling-update [FROM_REV=<коммит>]; RATE=$(RATE) rps"
@echo "scenario SCENARIO=failover-upload|s3-outage → evidence/<сценарий>.json; s3-check; attachments-evidence"
@echo "upload-faults: негативный suite, компенсация, уборка, обрыв и зависание загрузки против стенда"
@echo "upload-memory: пик памяти реплик при 10/20/40 одновременных загрузках 1 и 10 MiB → evidence/upload-memory/"
@echo "lab5: queues, topology, worker-stop/worker-start, broker-restart, images, suite-preflight, level3, level4"
@echo "lab5 оператор: failed (реестр), replay ID=<attachment_id>, dlq (просмотр DLQ без извлечения)"
@echo "retry-check: потерянный ответ на живой и на убитой реплике, повтор тем же ключом"
@echo "lab4-level4 = limits, lb-check, detection, failover-each, rolling-update [FROM_REV=<коммит>]; RATE=$(RATE) rps"
@echo "scenario SCENARIO=failover-upload|s3-outage → evidence/<сценарий>.json; s3-check; attachments-evidence"
@echo "upload-faults: негативный suite, компенсация, уборка, обрыв и зависание загрузки против стенда"
@echo "upload-memory ARGS='--out evidence/upload-memory/runN': пик памяти реплик при 10/20/40 загрузках 1 и 10 MiB"
@echo "load-evidence: tests/load.py ×3 клиентом-контейнером в сети edge стенда → evidence/load.txt, evidence/load/"
@echo "overload-leftovers: после upload-memory — намерения от 503, сверка и уборка после grace → evidence/overload-leftovers.txt"
@echo "clean-clone-evidence: блок «Запуск из чистого clone» в свежем клоне рядом → evidence/clean-clone.txt"
@echo "reconcile ARGS='--apply --deep': сверка PostgreSQL ↔ S3 (без ARGS — dry-run); трек 2:"
@echo "reconcile-demo: внедрение ошибок, dry-run/apply/повтор, безопасность, нагрузка → evidence/reconcile/"
@echo "reconcile-cost: стоимость обхода 1k/10k/30k/100k объектов × 3 прогона на чистом стенде → evidence/reconcile/cost/"
@echo "level4 = limits, lb-check, detection, failover-each, rolling-update [FROM_REV=<коммит>]; RATE=$(RATE) rps"
@echo "track1 = track1-host, track1-a (1x4/2x2/4x1), track1-b (1x1/2x1/8x1), track1-compare [BEFORE_REV=$(BEFORE_REV)], track1-control, plots → $(TRACK1)"
@echo "BASE_URL=$(BASE_URL) (порт lb из docker compose); failover-evidence SHIFT=1 — сдвиг round robin на вторую реплику"
check:
@@ -98,19 +78,17 @@ student-test:
secrets:
sh scripts/secrets.sh
sh scripts/broker-secrets.sh
up: secrets
$(COMPOSE) build
$(COMPOSE) up -d --wait
$(COMPOSE) up -d --build --wait
scripts/lb-wait.sh
down:
$(COMPOSE) --profile bench --profile tools --profile reconcile down
$(COMPOSE) --profile bench down
reset-data:
@echo "Удаляются volumes с учебными данными стенда"
$(COMPOSE) --profile bench --profile tools --profile reconcile down -v --remove-orphans
$(COMPOSE) --profile bench down -v --remove-orphans
migrate:
$(COMPOSE) run --rm migrate
@@ -151,44 +129,39 @@ endif
--out evidence/rolling$(if $(FROM_REV),-from-$(FROM_REV)) \
-- scripts/rolling-update.sh
lab4-level4: limits lb-check detection failover-each rolling-update
level4: limits lb-check detection failover-each rolling-update
scenario:
$(PYTHON) scripts/lab4_scenarios.py $(SCENARIO) --base-url "$(BASE_URL)" --restore --output evidence/$(SCENARIO).json
scripts/lb-wait.sh
track1: track1-images track1-host track1-a track1-b track1-compare track1-control plots
s3-check:
sh scripts/s3-check.sh > evidence/s3-security.txt; status=$$?; cat evidence/s3-security.txt; exit $$status
track1-images:
git archive "$(BEFORE_REV)" | docker build -q -t mtusi-messenger:$(BEFORE_REV) -
$(COMPOSE) build -q api
attachments-evidence:
sh scripts/evidence-attachments.sh
track1-host:
sh scripts/host-snapshot.sh $(TRACK1)/host.txt
upload-faults:
{ echo "date: $$(date -u +%FT%TZ) commit: $$(git rev-parse --short HEAD) BASE_URL=$(BASE_URL)"; \
BASE_URL="$(BASE_URL)" $(VENV)/bin/pytest -m stack -v -s --no-header --tb=short \
-p no:cacheprovider student-tests/stack/test_attachment_faults.py; } > evidence/upload-faults.txt; \
status=$$?; cat evidence/upload-faults.txt; exit $$status
track1-a:
$(SCALING) --out $(TRACK1)/series-a --configs 1x4,2x2,4x1 --api-memory-total 2048 \
--start-per-cpu 150 $(BASELINE)
upload-memory:
$(PYTHON) scripts/upload_memory.py --base-url "$(BASE_URL)" $(ARGS)
track1-b:
$(SCALING) --out $(TRACK1)/series-b --configs 1x1,2x1,8x1 --api-memory 512 \
--start-per-cpu 300 --search 1.5:9000 $(BASELINE)
load-evidence:
sh scripts/load-evidence.sh
track1-compare: track1-images
$(SCALING) --out $(TRACK1)/compare --configs 8x1 --api-memory 512 --rates 2400,3600,5400 \
--variant before:API_IMAGE=mtusi-messenger:$(BEFORE_REV) --variant after
overload-leftovers:
sh scripts/overload-leftovers.sh
track1-control:
$(SCALING) $(BASELINE) --out $(TRACK1)/control-pool --configs 8x1 --api-memory 512 --rates 5400 \
--variant pool4:DB_POOL_MAX_SIZE=4 --variant pool8 --variant pool16:DB_POOL_MAX_SIZE=16 \
--variant maxconn16:API_SERVER_MAXCONN=16 --variant maxconn64:API_SERVER_MAXCONN=64
$(SCALING) $(BASELINE) --out $(TRACK1)/control-cpus --configs 4x1,8x0.5 --api-memory-total 2048 --rates 3040,4560 \
--variant pinned
clean-clone-evidence:
sh scripts/clean-clone.sh
reconcile:
$(COMPOSE) run --rm --no-deps reconciler $(ARGS)
reconcile-demo:
$(PYTHON) scripts/reconcile_track.py demo --base-url "$(BASE_URL)" $(ARGS)
reconcile-cost:
$(PYTHON) scripts/reconcile_track.py cost $(ARGS)
plots:
MPLBACKEND=Agg $(UV) run --no-project --python 3.14 --with 'matplotlib==3.11.2' \
python tools/plot_track1.py --out $(TRACK1)/plots $(wildcard $(TRACK1)/*/summary.csv)
failover-evidence:
sh scripts/failover-evidence.sh "$(BASE_URL)" "$(SHIFT)"
@@ -224,64 +197,3 @@ explain: seed
echo "\$$ make explain # scripts/sql/explain-history.sql"; \
$(PSQL) -f - < scripts/sql/explain-history.sql; } > evidence/explain-history.txt
cat evidence/explain-history.txt
images:
$(PYTHON) scripts/make_images.py
queues:
sh scripts/broker.sh queues
topology:
{ echo "date: $$(date -u +%FT%TZ) commit: $$(git rev-parse --short HEAD)"; \
echo "$$ rabbitmqctl list_queues -p messenger name type durable policy messages"; \
$(COMPOSE) exec -T broker rabbitmqctl -q list_queues -p messenger name type durable policy messages; \
echo "$$ rabbitmqctl list_consumers -p messenger"; \
$(COMPOSE) exec -T broker rabbitmqctl -q list_consumers -p messenger queue_name ack_required prefetch_count; \
echo "$$ rabbitmqctl list_permissions -p messenger"; \
$(COMPOSE) exec -T broker rabbitmqctl -q list_permissions -p messenger; \
echo "$$ rabbitmqctl list_users"; \
$(COMPOSE) exec -T broker rabbitmqctl -q list_users; \
echo "$$ docker compose ps"; \
$(COMPOSE) ps --format '{{.Service}}\t{{.Status}}\t{{.Ports}}'; } > evidence/topology.txt
cat evidence/topology.txt
worker-stop:
sh scripts/worker.sh stop
worker-start:
sh scripts/worker.sh start
broker-restart:
sh scripts/broker.sh restart
suite-preflight:
BASE_URL="$(BASE_URL)" $(PYTHON) scripts/pipeline_suite.py --preflight
level3:
BASE_URL="$(BASE_URL)" $(PYTHON) scripts/pipeline_suite.py --level 3 --output evidence/level3.json
level4:
BASE_URL="$(BASE_URL)" $(PYTHON) scripts/pipeline_suite.py --level 4 --output evidence/level4.json
failed:
$(COMPOSE) run --rm --no-deps -T worker python -m messenger.admin failed
replay:
@test -n "$(ID)" || (echo "Set ID=<attachment_id>"; exit 2)
$(COMPOSE) run --rm --no-deps -T worker python -m messenger.admin replay --attachment-id "$(ID)"
dlq:
sh scripts/broker.sh dlq
pipeline-evidence:
$(PYTHON) scripts/pipeline_evidence.py --base-url "$(BASE_URL)" > evidence/pipeline-stand.txt; \
status=$$?; cat evidence/pipeline-stand.txt; exit $$status
.PHONY: track2 reprocess
track2:
BASE_URL="$(BASE_URL)" $(PYTHON) scripts/pipeline_suite.py --level 5 --only "$(TRACK2_SCENARIOS)" --output "$(TRACK2_OUTPUT)"
reprocess:
@test -n "$(ID)" || (echo "Set ID=<attachment_id>"; exit 2)
$(COMPOSE) run --rm --no-deps -T worker python -m messenger.admin reprocess --attachment-id "$(ID)" --pipeline-version "$(PIPELINE_VERSION)"
+32 -33
View File
@@ -1,6 +1,6 @@
# Лабораторная 5. Очереди и асинхронная обработка изображений
# Лабораторная 3. Балансировка и отказ API-реплики
Отделить приём файла от тяжёлой обработки, корректно переживать повторную доставку и остановку worker.
Запустить несколько взаимозаменяемых экземпляров API за единой точкой входа и сохранить корректность при остановке одного из них.
Это **skeleton**, а не готовый сервер. Здесь есть задание, контракт и публичные проверки. Реализацию приложения, Dockerfile и требуемую инфраструктуру пишет студент. Язык и framework свободные; UI не обязателен.
@@ -8,42 +8,42 @@
## Откуда начинаем
Нужен минимум lab4. POST upload теперь возвращает 202 queued, а готовность выясняется через метаданные. В HTTP upload клиент по-прежнему передаёт все байты.
Нужен минимум lab2: PostgreSQL, общие действующие сессии, миграции. Добавляются X-Instance-Id и обязательный Idempotency-Key при отправке сообщения.
**Архитектура:** API → устойчивый staging + запись задания → брокер → worker → конечный S3 + PostgreSQL. Клиент опрашивает статус.
**Архитектура:** Клиент → LB → API × 2+ → общие PostgreSQL и хранилище сессий. Compose или Swarm — на выбор.
## Что сделать
1. Выберите RabbitMQ, Redis/Valkey Streams, NATS JetStream или другой брокер с persistence и подтверждениями. Обычный Pub/Sub без хранения не подходит.
2. Разделите приём, durable staging, публикацию задания, обработку и фиксацию результата. Прочитайте contracts/attachment-states.md.
3. Реализуйте worker отдельно от API: публикация оригинала в конечный S3-prefix и thumbnail 256×256 по правилам контракта.
4. Добавьте конечные состояния, ограниченные повторы и идемпотентную запись результатов. Не помещайте bytes или credentials в сообщения брокера.
5. Остановите worker, загрузите файл, перезапустите API/worker, убедитесь в завершении принятой задачи. Затем проверьте повторную доставку.
1. Выберите Compose + Nginx/HAProxy/Traefik либо Docker Swarm. Для выбранного варианта дайте одну последовательность запуска.
2. Уберите локальные сессии/другие данные, от которых зависит корректность, из API. Миграции выполняйте отдельно и согласованно, а не гонкой всех реплик.
3. Настройте балансировщик, проверки готовности, DNS/обновление backend-адресов и graceful shutdown. Не закрепляйте клиента sticky session как способ исправить раздельное состояние.
4. Реализуйте общую идемпотентность отправки сообщения. Проверьте одинаковый ключ одновременно через разные реплики.
5. Подготовьте явный action script, останавливающий одну выбранную API-реплику, и продемонстрируйте чтение/запись во время отказа.
## Оценка «3»: работающий минимум
- Весь минимум lab4 сохранён с переходом upload 201 → 202. API возвращает 202 только после устойчивого приёма оригинала и сохранения задания; остановка API после ответа не теряет работу.
- В общем Compose/Swarm есть отдельные broker и worker. Сообщения durable, доставка at least once, подтверждение только после устойчивого эффекта. Недоступность worker не заставляет API выполнять resize синхронно.
- Состояния queued/processing/ready/failed доступны в API. Worker сохраняет оригинал и один thumbnail; original неизменен, thumbnail соответствует контракту. Публичный smoke проходит.
- Есть ограниченный retry (конкретное число попыток в конфигурации, например 5), финальное failed с безопасным error_code. Повтор одной job не создаёт второго набора вариантов.
- Демонстрация: worker остановлен → upload 202 → сообщение с вложением создано → API перезапущен → worker запущен → ready и доступный thumbnail. Принятое задание не исчезает при рестарте broker с volume.
- Весь стенд описан в compose.yaml или stack.yaml, запускается документированной командой. Не менее двух API-реплик, одна публичная точка входа, одна общая БД и общее хранилище сессий.
- Разные запросы одного пользователя обслуживаются разными репликами без повторного входа. Это видно по X-Instance-Id. Доступ через LB — основной проверяемый маршрут.
- Можно остановить любую одну API-реплику при работающей другой: не позднее 10 секунд продолжаются новые чтения/записи, подтверждённые сообщения доступны. Краткие ошибки/оборванные запросы во время обнаружения отказа измеряются и допустимы.
- Повтор POST сообщения с тем же Idempotency-Key не создаёт дубль, включая конкурентный повтор и переключение реплики. Неподтверждённую запись клиент может безопасно повторить с тем же ключом.
- make test, persistence и failover-сценарий проходят; для failover проверяется наличие ≥2 наблюдаемых реплик. Для Swarm сборка образа и его доступность узлам описаны отдельно.
## Оценка «4»: уровень стажёра/джуна
Весь уровень 3 **и все** пункты ниже:
- Добавлены crop 128×128 и optimized WebP по контракту, параметры качества/удаления метаданных описаны. Невалидные/огромные изображения ограничены по bytes, пикселям, времени и памяти worker.
- Разрыв между PostgreSQL и broker закрыт transactional outbox или другой доказанной схемой восстановления намерения. Внедрён crash между COMMIT и publish; после восстановления задача доходит до ready без ручного создания нового задания.
- Retry использует backoff с jitter, различает постоянные/временные ошибки; есть DLQ/реестр failed и безопасный явный replay. «Повторить» не означает безусловно исполнить дубль уже успешной job.
- Проверена остановка worker после записи объекта, но до ack; повтор завершает тот же результат. Есть timeout/lease для зависшего processing, ограниченная конкуренция/prefetch и очистка staging после безопасной фиксации.
- Health/readiness, timeouts и прекращение приёма при SIGTERM настроены согласованно; LB исключает недоступный backend. Укажите период проверок и максимальное время обнаружения отказа.
- Последовательно остановите каждую API-реплику (восстанавливая предыдущую); соберите RPS, ошибки и время восстановления. Общая идемпотентность проверяется параллельными запросами с одним ключом.
- Контейнеры имеют CPU/RAM-лимиты, API не публикуют порты наружу в обход LB; секреты и хранилища общие только там, где нужно. Нет container_name, мешающего масштабированию одного Compose-service.
- Rolling update или эквивалентная поочерёдная замена проверена под нагрузкой. Миграции запускаются однократно и совместимы с работающей версией либо ограничение честно описано.
## Оценка «5»: исследование и доказанный результат
Весь уровень 4 **и один законченный трек на выбор**. Приведите гипотезу, повторяемую методику, результаты и ограничения.
**Трек 1. Очередь под перегрузкой.** Сравните ≥3 настройки concurrency/prefetch на разных размерах изображений. Измерьте throughput, queue lag, p95 времени до ready и RSS; покажите backpressure и ограничение количества принятой работы. Докажите восстановление после перегрузки, объясните компромисс latency/ресурсы.
**Трек 1. Цена масштабирования.** При фиксированном суммарном CPU/RAM сравните 1/2/4 API-реплики на одном профиле нагрузки; затем проведите серию с ростом ресурсов. Найдите saturation point БД/LB/пула и покажите, почему рост реплик перестаёт помогать. Дайте исходные данные, графики и одно подтверждённое улучшение.
**Трек 2. Доказанная идемпотентность pipeline.** Постройте versioned pipeline и автоматический fault-injection для дубля, reorder, crash до/после side effect и устаревшей job после новой версии. Зафиксируйте инварианты, покажите отсутствие потери подтверждённых задач и перезаписи нового результата старым. Обсудите почему это не exactly-once доставка.
**Трек 2. Отказ узла.** Расширьте стенд до нескольких узлов, покажите перенос API при потере узла и разберите оставшиеся SPOF (LB, DB, storage). Для заявленной защищённой зависимости покажите реальный failover и сохранность данных. Если DB не реплицирована, не заявляйте выживание её узла.
## Как начать и проверить
@@ -79,27 +79,26 @@ Smoke не выставляет оценку автоматически. Про
## Сценарий защиты
1. Запустить make test: API возвращает 202, GET доходит до ready, оригинал и настоящий JPEG thumbnail проверяются.
2. Остановить worker своим скриптом, загрузить файл, убедиться, что content отвечает 409, а другие сообщения продолжают работать.
3. Перезапустить API/broker с сохранением volumes, запустить worker; та же запись должна стать ready. Зафиксировать порядок действий и ID без токенов.
4. Для 4: воспроизвести crash после side effect до ack, разрыв COMMIT/publish и replay failed; для 5 — выбранный эксперимент.
1. Запустить стенд и показать ≥2 разных X-Instance-Id через LB с одной сессией.
2. Запустить make test-failover ACTION_SCRIPT=scripts/stop-one.sh. Скрипт теста выполняет запросы во время action, учитывает ошибки и повторяет сообщения с исходным ключом.
3. Показать, какой контейнер действительно остановлен; заголовок сам по себе не доказательство разных процессов. Повторить для второй реплики после восстановления первой.
4. Проверить restart всего стенда с сохранением volumes и неизменным токеном. Для 4/5 — соответствующие эксперименты.
## Что должно быть в PR
Worker, брокер в общем стенде, схема jobs, migrations/outbox по уровню, конфигурация retry, тесты сбоев и отчёт.
compose.yaml или stack.yaml, конфигурация LB, scripts/restart.sh, scripts/stop-one.sh, Dockerfile, миграции, результаты отказов.
Заполните `REPORT.md` и [шаблон PR](.github/pull_request_template.md), укажите целевую оценку/трек. Не меняйте обязательный контракт и публичные tests ради зелёного результата. Сохраняйте возможности предыдущих лабораторных в пределах [объявленных переходов](COURSE.md).
## Вопросы на защите
- Чем publisher confirm отличается от consumer ack?
- Что именно уже сохранено в момент 202?
- Почему повторная доставка нормальна и где находится ключ идемпотентности job?
- Как отличить зависший worker от долго работающего и не запустить два неконтролируемых дубля?
- Чем отказ процесса отличается от отказа узла?
- Почему depends_on/healthcheck сами по себе не являются балансировщиком?
- Какие POST можно повторять после timeout и что защищает Idempotency-Key?
- Что сломается при увеличении числа API, если сессии находятся в локальной памяти?
## Первичные материалы
- [RabbitMQ reliability](https://www.rabbitmq.com/docs/reliability)
- [Publisher confirms and acknowledgements](https://www.rabbitmq.com/docs/confirms)
- [Redis Streams](https://redis.io/docs/latest/develop/data-types/streams/)
- [NATS JetStream](https://docs.nats.io/nats-concepts/jetstream)
- [Compose services](https://docs.docker.com/reference/compose-file/services/)
- [Swarm routing mesh](https://docs.docker.com/engine/swarm/ingress/)
- [Swarm services](https://docs.docker.com/engine/swarm/services/)
+845 -167
View File
File diff suppressed because it is too large Load Diff
+1 -22
View File
@@ -6,38 +6,17 @@ from legacy import adopt_legacy
from sqlalchemy import Connection, func, make_url, pool, select
from sqlalchemy.ext.asyncio import create_async_engine
from messenger.attachments.infrastructure.postgres import (
AttachmentRow,
JobEventRow,
JobRow,
OutboxRow,
VariantRow,
)
from messenger.identity.infrastructure.postgres import SessionRow, UserRow
from messenger.messaging.infrastructure.postgres import (
ChatMemberRow,
ChatRow,
IdempotencyKeyRow,
MessageAttachmentRow,
MessageRow,
)
from messenger.shared.config import PlatformSettings
LOCK_KEY = 20_260_002
MODELS = (
UserRow,
SessionRow,
ChatRow,
ChatMemberRow,
MessageRow,
IdempotencyKeyRow,
AttachmentRow,
JobRow,
OutboxRow,
JobEventRow,
VariantRow,
MessageAttachmentRow,
)
MODELS = (UserRow, SessionRow, ChatRow, ChatMemberRow, MessageRow, IdempotencyKeyRow)
target_metadata = UserRow.metadata
config = context.config
-6
View File
@@ -9,12 +9,6 @@ from sqlalchemy.schema import CreateSchema
LEGACY = {
"0001_init": "9a13618f078ae9056431391e2e5197a31b981ae1fff047a9b3421ac750d3fd0d",
"0002_idempotency_keys": "d9159a35553249a1e51af5f2e4e7c39b014e97593c49748e13f3799839063dd0",
"0003_attachments": "c4417f560538a083efba1c67f660147dc94532fcbb2fa91f3f3e8258a0f01f2e",
"0004_jobs": "c3887fa7a886ae8153a216aa43fd4c3c6190a2074e8ce20c3e77f29bfb01550e",
"0005_failed_registry": "02b80061a08a6929feb092148c453ad970e3d001901c651b5a73b90b1e40f145",
"0006_attachment_quarantine": (
"541f5399d87fac3584c4102b4e0bf92c23cd372cc20a7d56dad03e771773202c"
),
}
SNAPSHOT = text("""
WITH relations AS (
-77
View File
@@ -1,77 +0,0 @@
import sqlalchemy as sa
from alembic import op
revision = "0003_attachments"
down_revision = "0002_idempotency_keys"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.drop_constraint("messages_text_check", "messages", type_="check")
op.create_check_constraint("messages_text_check", "messages", "char_length(text) <= 4000")
op.create_table(
"attachments",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("chat_id", sa.Uuid(), nullable=False),
sa.Column("owner_id", sa.Uuid(), nullable=False),
sa.Column("filename", sa.Text(), nullable=False),
sa.Column("content_type", sa.Text(), nullable=False),
sa.Column("size_bytes", sa.BigInteger(), nullable=False),
sa.Column("sha256", sa.Text(), nullable=False),
sa.Column("width", sa.Integer(), nullable=False),
sa.Column("height", sa.Integer(), nullable=False),
sa.Column("object_key", sa.Text(collation="C"), nullable=False),
sa.Column("status", sa.Text(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("object_key", name="attachments_object_key_key"),
sa.UniqueConstraint("id", "chat_id", name="attachments_chat_key"),
sa.ForeignKeyConstraint(
["chat_id", "owner_id"],
["chat_members.chat_id", "chat_members.user_id"],
name="attachments_owner_fkey",
),
sa.CheckConstraint(
"char_length(filename) BETWEEN 1 AND 255", name="attachments_filename_check"
),
sa.CheckConstraint(
"content_type IN ('image/png', 'image/jpeg')", name="attachments_content_type_check"
),
sa.CheckConstraint(
"size_bytes BETWEEN 1 AND 10485760", name="attachments_size_bytes_check"
),
sa.CheckConstraint("sha256 ~ '^[0-9a-f]{64}$'", name="attachments_sha256_check"),
sa.CheckConstraint("width > 0", name="attachments_width_check"),
sa.CheckConstraint("height > 0", name="attachments_height_check"),
sa.CheckConstraint("status IN ('pending', 'ready')", name="attachments_status_check"),
)
op.create_index(
"attachments_pending_idx",
"attachments",
["created_at"],
postgresql_where=sa.text("status = 'pending'"),
)
op.create_table(
"message_attachments",
sa.Column("message_id", sa.Uuid(), nullable=False),
sa.Column("position", sa.SmallInteger(), nullable=False),
sa.Column("chat_id", sa.Uuid(), nullable=False),
sa.Column("attachment_id", sa.Uuid(), nullable=False),
sa.PrimaryKeyConstraint("message_id", "position"),
sa.UniqueConstraint("message_id", "attachment_id"),
sa.CheckConstraint("position BETWEEN 0 AND 9", name="message_attachments_position_check"),
sa.ForeignKeyConstraint(["message_id"], ["messages.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(
["attachment_id", "chat_id"],
["attachments.id", "attachments.chat_id"],
name="message_attachments_attachment_fkey",
ondelete="RESTRICT",
),
)
op.create_index("message_attachments_attachment_idx", "message_attachments", ["attachment_id"])
def downgrade() -> None:
raise RuntimeError("Downgrade would delete application data")
-146
View File
@@ -1,146 +0,0 @@
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0004_jobs"
down_revision = "0003_attachments"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.drop_constraint("attachments_status_check", "attachments", type_="check")
op.create_check_constraint(
"attachments_status_check",
"attachments",
"status IN ('pending', 'queued', 'processing', 'ready', 'failed')",
)
for name in ("staging_key", "original_key"):
op.add_column("attachments", sa.Column(name, sa.Text(collation="C"), nullable=True))
op.create_unique_constraint(f"attachments_{name}_key", "attachments", [name])
op.add_column(
"attachments",
sa.Column("published_version", sa.Integer(), nullable=False, server_default="0"),
)
op.create_check_constraint(
"attachments_published_version_check", "attachments", "published_version >= 0"
)
op.add_column("attachments", sa.Column("error_code", sa.Text(), nullable=True))
op.create_check_constraint(
"attachments_error_code_check",
"attachments",
"error_code IN ('invalid_image', 'processing_failed', 'retry_exhausted')",
)
op.alter_column("attachments", "object_key", nullable=True)
attachments = sa.table(
"attachments",
sa.column("status"),
sa.column("object_key"),
sa.column("original_key"),
sa.column("staging_key"),
sa.column("published_version"),
)
op.execute(
attachments.update()
.where(attachments.c.status == "ready")
.values(original_key=attachments.c.object_key, published_version=1)
)
op.execute(
attachments.update()
.where(attachments.c.status == "pending")
.values(staging_key=attachments.c.object_key)
)
op.create_index(
"attachments_staged_idx",
"attachments",
["updated_at"],
postgresql_where=sa.column("staging_key").is_not(None),
)
op.create_table(
"attachment_variants",
sa.Column("attachment_id", sa.Uuid(), sa.ForeignKey("attachments.id"), nullable=False),
sa.Column("pipeline_version", sa.Integer(), nullable=False),
sa.Column("name", sa.Text(), nullable=False),
sa.Column("object_key", sa.Text(collation="C"), nullable=False),
sa.Column("content_type", sa.Text(), nullable=False),
sa.Column("size_bytes", sa.BigInteger(), nullable=False),
sa.Column("width", sa.Integer(), nullable=False),
sa.Column("height", sa.Integer(), nullable=False),
sa.Column("sha256", sa.Text(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("attachment_id", "pipeline_version", "name"),
sa.UniqueConstraint("object_key", name="attachment_variants_object_key_key"),
sa.CheckConstraint(
"pipeline_version >= 1", name="attachment_variants_pipeline_version_check"
),
sa.CheckConstraint(
"name IN ('thumbnail', 'crop', 'optimized')", name="attachment_variants_name_check"
),
sa.CheckConstraint(
"content_type IN ('image/jpeg', 'image/png', 'image/webp')",
name="attachment_variants_content_type_check",
),
sa.CheckConstraint("size_bytes >= 1", name="attachment_variants_size_bytes_check"),
sa.CheckConstraint("width >= 1", name="attachment_variants_width_check"),
sa.CheckConstraint("height >= 1", name="attachment_variants_height_check"),
sa.CheckConstraint("sha256 ~ '^[0-9a-f]{64}$'", name="attachment_variants_sha256_check"),
)
op.create_table(
"jobs",
sa.Column("job_id", sa.Uuid(), primary_key=True),
sa.Column("attachment_id", sa.Uuid(), sa.ForeignKey("attachments.id"), nullable=False),
sa.Column("pipeline_version", sa.Integer(), nullable=False),
sa.Column("status", sa.Text(), nullable=False),
sa.Column("attempt", sa.Integer(), nullable=False),
sa.Column("claims", sa.Integer(), nullable=False, server_default="0"),
sa.Column("lease_token", sa.Uuid(), nullable=True),
sa.Column("lease_until", sa.DateTime(timezone=True), nullable=True),
sa.Column("next_attempt_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_error", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.UniqueConstraint("attachment_id", "pipeline_version", name="jobs_effect_key"),
sa.CheckConstraint("pipeline_version >= 1", name="jobs_pipeline_version_check"),
sa.CheckConstraint(
"status IN ('queued', 'processing', 'done', 'failed', 'superseded')",
name="jobs_status_check",
),
sa.CheckConstraint("attempt >= 1", name="jobs_attempt_check"),
sa.CheckConstraint("claims >= 0", name="jobs_claims_check"),
)
op.create_index(
"jobs_lease_idx",
"jobs",
["lease_until"],
postgresql_where=sa.column("status") == "processing",
)
op.create_table(
"outbox",
sa.Column("id", sa.BigInteger(), sa.Identity(always=True), primary_key=True),
sa.Column("job_id", sa.Uuid(), sa.ForeignKey("jobs.job_id"), nullable=False),
sa.Column("payload", postgresql.JSONB(), nullable=False),
sa.Column("available_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("published_at", sa.DateTime(timezone=True), nullable=True),
)
op.create_index(
"outbox_pending_idx",
"outbox",
["available_at", "id"],
postgresql_where=sa.column("published_at").is_(None),
)
op.create_index("outbox_job_idx", "outbox", ["job_id"])
op.create_table(
"job_events",
sa.Column("id", sa.BigInteger(), sa.Identity(always=True), primary_key=True),
sa.Column("job_id", sa.Uuid(), nullable=False),
sa.Column("attachment_id", sa.Uuid(), nullable=False),
sa.Column("kind", sa.Text(), nullable=False),
sa.Column("attempt", sa.Integer(), nullable=True),
sa.Column("detail", postgresql.JSONB(), nullable=False, server_default="{}"),
sa.Column("at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
)
op.create_index("job_events_attachment_idx", "job_events", ["attachment_id", "id"])
def downgrade() -> None:
raise RuntimeError("Downgrade would delete pipeline data")
-20
View File
@@ -1,20 +0,0 @@
import sqlalchemy as sa
from alembic import op
revision = "0005_failed_registry"
down_revision = "0004_jobs"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_index(
"jobs_failed_idx",
"jobs",
[sa.column("updated_at").desc()],
postgresql_where=sa.column("status") == "failed",
)
def downgrade() -> None:
op.drop_index("jobs_failed_idx", table_name="jobs")
@@ -1,28 +0,0 @@
import sqlalchemy as sa
from alembic import op
revision = "0006_attachment_quarantine"
down_revision = "0005_failed_registry"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("attachments", sa.Column("quarantine_reason", sa.Text(), nullable=True))
op.create_check_constraint(
"attachments_quarantine_reason_check",
"attachments",
"quarantine_reason IN ('missing', 'size_mismatch', 'sha_mismatch')",
)
op.add_column(
"attachments", sa.Column("quarantined_at", sa.DateTime(timezone=True), nullable=True)
)
op.create_check_constraint(
"attachments_quarantine_check",
"attachments",
"(quarantine_reason IS NULL) = (quarantined_at IS NULL)",
)
def downgrade() -> None:
raise RuntimeError("Downgrade would delete quarantine data")
+1 -4
View File
@@ -1,14 +1,11 @@
# Topology worksheet, deliberately NOT a runnable stack (no application supplied).
# Implement compose.yaml or stack.yaml; do not submit this empty file as deployment.
# Service names below are logical roles, not imposed container names.
x-lab-number: 5
x-lab-number: 3
x-required-roles:
- api
- postgres
- load_balancer
- s3
- broker
- worker
# TODO: implement real images/builds, network wiring, volumes, healthchecks,
# credentials and bootstrap. Details: contracts/infrastructure.json and README.md.
services: {}
+8 -263
View File
@@ -2,87 +2,6 @@ x-app: &app
image: ${API_IMAGE:-mtusi-messenger:${COMPOSE_PROJECT_NAME}}
networks: [backend]
x-s3-env: &s3-env
S3_ENDPOINT: ${S3_ENDPOINT:-http://s3:9000}
S3_REGION: ${S3_REGION:-us-east-1}
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_access_key_id
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_secret_access_key
S3_CONNECT_TIMEOUT_SECONDS: ${S3_CONNECT_TIMEOUT_SECONDS:-1}
S3_READ_TIMEOUT_SECONDS: ${S3_READ_TIMEOUT_SECONDS:-2}
MAX_UPLOAD_BYTES: ${MAX_UPLOAD_BYTES:-10485760}
MAX_IMAGE_PIXELS: ${MAX_IMAGE_PIXELS:-40000000}
IMAGE_CHECK_CONCURRENCY: ${IMAGE_CHECK_CONCURRENCY:-2}
UPLOAD_CONCURRENCY: ${UPLOAD_CONCURRENCY:-3}
UPLOAD_GRACE_SECONDS: ${UPLOAD_GRACE_SECONDS:-900}
UPLOAD_SWEEP_INTERVAL_SECONDS: ${UPLOAD_SWEEP_INTERVAL_SECONDS:-30}
UPLOAD_SWEEP_BATCH: ${UPLOAD_SWEEP_BATCH:-100}
x-pipeline: &pipeline
image: ${API_IMAGE:-mtusi-messenger:${COMPOSE_PROJECT_NAME}}
pull_policy: never
restart: unless-stopped
init: true
read_only: true
tmpfs: [/tmp]
security_opt: ["no-new-privileges:true"]
cap_drop: [ALL]
stop_grace_period: 10s
networks: [backend]
healthcheck:
test: ["CMD", "python", "-c", "import os,sys,time;sys.exit(time.time()-os.stat('/tmp/heartbeat').st_mtime>30)"]
interval: 10s
timeout: 5s
retries: 3
start_period: 30s
start_interval: 2s
depends_on:
migrate:
condition: service_completed_successfully
s3-init:
condition: service_completed_successfully
broker:
condition: service_healthy
x-pipeline-env: &pipeline-env
<<: *s3-env
DATABASE_URL_FILE: /run/secrets/database_url
BROKER_URL_FILE: /run/secrets/broker_url
DB_POOL_MAX_SIZE: ${PIPELINE_DB_POOL_MAX_SIZE:-4}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
HEARTBEAT_FILE: /tmp/heartbeat
JOB_MAX_ATTEMPTS: ${JOB_MAX_ATTEMPTS:-5}
JOB_TIMEOUT_SECONDS: ${JOB_TIMEOUT_SECONDS:-60}
JOB_BACKOFF_BASE_SECONDS: ${JOB_BACKOFF_BASE_SECONDS:-2}
STAGING_RETENTION_SECONDS: ${STAGING_RETENTION_SECONDS:-86400}
x-s3-client: &s3-client
image: rustfs/rc:v0.1.36@sha256:ab024bfebee49a750ce886b4c70963ccd9ddaa03f491704a90710641d7a26699
read_only: true
tmpfs: ["/tmp:size=16m"]
environment:
S3_ENDPOINT: ${S3_ENDPOINT:-http://s3:9000}
S3_BUCKET: ${S3_BUCKET:-messenger-attachments}
RC_CONFIG_DIR: /tmp/rc
user: "65534:65534"
volumes: ["./infra/s3:/s3:ro"]
secrets:
- s3_root_user
- s3_root_password
- s3_access_key_id
- s3_secret_access_key
- s3_reconciler_access_key_id
- s3_reconciler_secret_access_key
- s3_worker_access_key_id
- s3_worker_secret_access_key
cap_drop: [ALL]
security_opt: ["no-new-privileges:true"]
restart: "no"
networks: [backend]
deploy:
resources:
limits: {cpus: "0.5", memory: 64m}
services:
lb:
image: haproxy:3.2.26-alpine@sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b
@@ -90,16 +9,15 @@ services:
API_SERVER_MAXCONN: ${API_SERVER_MAXCONN:-64}
volumes: [./infra/haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro]
ports:
- "127.0.0.1:${LB_HTTP_PORT:-18085}:8080"
- "127.0.0.1:${LB_STATS_PORT:-25085}:8404"
- "127.0.0.1:${LB_HTTP_PORT:-8080}:8080"
- "127.0.0.1:${LB_STATS_PORT:-8404}:8404"
networks: [edge, backend]
healthcheck:
test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1:8404/healthz"]
interval: 2s
timeout: 2s
retries: 3
start_period: 60s
start_interval: 1s
start_period: 10s
sysctls:
net.ipv4.ip_local_port_range: "10240 65535"
net.ipv4.tcp_tw_reuse: "1"
@@ -108,6 +26,7 @@ services:
restart: unless-stopped
read_only: true
cap_drop: [ALL]
cpuset: ${LB_CPUSET:-}
deploy:
resources:
limits: {cpus: "${LB_CPUS:-1}", memory: 128m}
@@ -116,7 +35,6 @@ services:
<<: *app
build: .
environment:
<<: *s3-env
DATABASE_URL_FILE: /run/secrets/database_url
CACHE_URL_FILE: /run/secrets/cache_url
CURSOR_SECRET_FILE: /run/secrets/cursor_secret
@@ -125,14 +43,12 @@ services:
WEB_CONCURRENCY: ${WEB_CONCURRENCY:-2}
DB_POOL_MAX_SIZE: ${DB_POOL_MAX_SIZE:-8}
REQUEST_TIMEOUT_SECONDS: ${REQUEST_TIMEOUT_SECONDS:-8}
MIN_TRANSFER_RATE_BYTES: ${MIN_TRANSFER_RATE_BYTES:-131072}
DB_ACQUIRE_TIMEOUT_SECONDS: ${DB_ACQUIRE_TIMEOUT_SECONDS:-1}
DB_COMMAND_TIMEOUT_SECONDS: ${DB_COMMAND_TIMEOUT_SECONDS:-2}
DRAIN_SECONDS: ${DRAIN_SECONDS:-2}
SHUTDOWN_TIMEOUT_SECONDS: ${SHUTDOWN_TIMEOUT_SECONDS:-6}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
MALLOC_MMAP_THRESHOLD_: ${MALLOC_MMAP_THRESHOLD_:-131072}
secrets: [database_url, cache_url, cursor_secret, s3_access_key_id, s3_secret_access_key]
secrets: [database_url, cache_url, cursor_secret]
healthcheck:
test: ["CMD", "python", "-c", "import sys,urllib.request;sys.exit(urllib.request.urlopen('http://127.0.0.1:8080/health/ready',timeout=2).status!=200)"]
interval: 5s
@@ -144,16 +60,12 @@ services:
restart: unless-stopped
read_only: true
tmpfs: [/tmp]
sysctls:
net.ipv4.tcp_rmem: "4096 65536 262144"
net.ipv4.tcp_wmem: "4096 65536 262144"
cpuset: ${API_CPUSET:-}
depends_on:
migrate:
condition: service_completed_successfully
valkey:
condition: service_started
s3-init:
condition: service_completed_successfully
deploy:
replicas: ${API_REPLICAS:-2}
resources:
@@ -162,7 +74,6 @@ services:
migrate:
<<: *app
pull_policy: never
init: true
command: ["alembic", "upgrade", "head"]
environment:
DATABASE_URL_FILE: /run/secrets/database_url
@@ -233,6 +144,7 @@ services:
- volatile-lru
secrets: [valkey.conf]
volumes: [valkeydata:/data]
cpuset: ${VALKEY_CPUSET:-}
healthcheck:
test: ["CMD-SHELL", "REDISCLI_AUTH=\"$$(sed -n 's/^requirepass //p' /run/secrets/valkey.conf)\" valkey-cli ping | grep -q PONG"]
interval: 2s
@@ -245,148 +157,6 @@ services:
resources:
limits: {cpus: "0.5", memory: 128m}
s3:
image: rustfs/rustfs:1.0.4@sha256:ed9210d6411d77ffceb70d94b964a3cb7d225983368bc25bebe01e995293dcc2
environment:
RUSTFS_ACCESS_KEY_FILE: /run/secrets/s3_root_user
RUSTFS_SECRET_KEY_FILE: /run/secrets/s3_root_password
RUSTFS_ADDRESS: ":9000"
RUSTFS_CONSOLE_ENABLE: "false"
RUSTFS_REGION: ${S3_REGION:-us-east-1}
RUSTFS_OBS_LOG_DIRECTORY: ""
RUSTFS_OBS_LOGGER_LEVEL: ${S3_LOG_LEVEL:-warn}
secrets: [s3_root_user, s3_root_password]
volumes: [s3data:/data]
cpuset: ${S3_CPUSET:-}
read_only: true
tmpfs: ["/tmp:size=64m"]
cap_drop: [ALL]
security_opt: ["no-new-privileges:true"]
healthcheck:
test: ["CMD", "curl", "-fsS", "-o", "/dev/null", "http://127.0.0.1:9000/health/ready"]
interval: 2s
timeout: 3s
retries: 30
start_period: 20s
stop_grace_period: 10s
restart: unless-stopped
networks: [backend]
deploy:
resources:
limits: {cpus: "${S3_CPUS:-1}", memory: "${S3_MEMORY:-512m}"}
reconciler:
<<: *app
pull_policy: never
profiles: [reconcile]
init: true
entrypoint: ["python", "-m", "messenger.attachments.entrypoints.reconcile"]
command: ["--dry-run"]
environment:
<<: *s3-env
DATABASE_URL_FILE: /run/secrets/database_url
S3_ACCESS_KEY_ID_FILE: /run/secrets/s3_reconciler_access_key_id
S3_SECRET_ACCESS_KEY_FILE: /run/secrets/s3_reconciler_secret_access_key
S3_READ_TIMEOUT_SECONDS: ${RECONCILE_S3_READ_TIMEOUT_SECONDS:-10}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
secrets: [database_url, s3_reconciler_access_key_id, s3_reconciler_secret_access_key]
healthcheck:
disable: true
restart: "no"
read_only: true
cpuset: ${RECONCILE_CPUSET:-}
depends_on:
postgres:
condition: service_healthy
s3:
condition: service_healthy
deploy:
resources:
limits: {cpus: "${RECONCILE_CPUS:-1}", memory: "${RECONCILE_MEMORY:-256m}"}
s3-init:
<<: *s3-client
entrypoint: ["/bin/sh", "/s3/init.sh"]
depends_on:
s3:
condition: service_healthy
s3-cli:
<<: *s3-client
profiles: [tools]
entrypoint: ["/bin/sh", "/s3/cli.sh"]
broker:
image: rabbitmq:4.1.8-management-alpine@sha256:54ef5ff505352c1beef05a97ee9bd6ab95b3559a76629ba0119b1d63d8d6e4b2
hostname: broker
user: "100:101"
restart: unless-stopped
read_only: true
tmpfs: [/tmp]
security_opt: ["no-new-privileges:true"]
cap_drop: [ALL]
ports:
- "127.0.0.1:${BROKER_MANAGEMENT_PORT:-25086}:15672"
networks: [backend, edge]
volumes:
- broker-data:/var/lib/rabbitmq
- ./infra/rabbitmq/rabbitmq.conf:/etc/rabbitmq/conf.d/20-messenger.conf:ro
- ./infra/rabbitmq/enabled_plugins:/etc/rabbitmq/enabled_plugins:ro
- ./infra/rabbitmq/definitions.json:/etc/rabbitmq/definitions.d/10-topology.json:ro
secrets:
- source: rabbitmq_users
target: /etc/rabbitmq/definitions.d/20-users.json
- rabbitmq_ops_password
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "-q", "check_port_connectivity"]
interval: 10s
timeout: 10s
retries: 5
start_period: 60s
start_interval: 2s
stop_grace_period: 10s
deploy:
resources:
limits: {cpus: "1", memory: 512m}
relay:
<<: *pipeline
command: ["python", "-m", "messenger.relay"]
environment:
<<: *pipeline-env
OUTBOX_POLL_SECONDS: ${OUTBOX_POLL_SECONDS:-1}
SWEEP_INTERVAL_SECONDS: ${SWEEP_INTERVAL_SECONDS:-5}
secrets:
- database_url
- source: broker_url_api
target: broker_url
- s3_access_key_id
- s3_secret_access_key
deploy:
resources:
limits: {cpus: "0.5", memory: 192m}
worker:
<<: *pipeline
command: ["python", "-m", "messenger.worker"]
environment:
<<: *pipeline-env
WORKER_CONCURRENCY: ${WORKER_CONCURRENCY:-2}
RENDER_TIMEOUT_SECONDS: ${RENDER_TIMEOUT_SECONDS:-30}
RENDER_MEMORY_MB: ${RENDER_MEMORY_MB:-448}
RENDER_TASKS_PER_CHILD: ${RENDER_TASKS_PER_CHILD:-100}
secrets:
- database_url
- source: broker_url_worker
target: broker_url
- source: s3_worker_access_key_id
target: s3_access_key_id
- source: s3_worker_secret_access_key
target: s3_secret_access_key
deploy:
resources:
limits: {cpus: "${WORKER_CPUS:-2}", memory: "${WORKER_MEMORY:-1g}"}
k6:
image: grafana/k6:2.3.0@sha256:2dd4756ec6e14e62778916e1eb850108e4b499304c02df56760fa0bdb0fba0b8
profiles: [bench]
@@ -394,6 +164,7 @@ services:
BASE_URL: http://lb:8080
volumes: [./infra/k6:/scripts:ro]
networks: [edge]
cpuset: ${K6_CPUSET:-}
deploy:
resources:
limits: {cpus: "${K6_CPUS:-2}", memory: "${K6_MEMORY:-512m}"}
@@ -409,36 +180,10 @@ secrets:
file: ./secrets/cache_url
cursor_secret:
file: ./secrets/cursor_secret
s3_root_user:
file: ./secrets/s3_root_user
s3_root_password:
file: ./secrets/s3_root_password
s3_access_key_id:
file: ./secrets/s3_access_key_id
s3_secret_access_key:
file: ./secrets/s3_secret_access_key
s3_worker_access_key_id:
file: ./secrets/s3_worker_access_key_id
s3_worker_secret_access_key:
file: ./secrets/s3_worker_secret_access_key
rabbitmq_users:
file: ./secrets/rabbitmq_users.json
rabbitmq_ops_password:
file: ./secrets/rabbitmq_ops_password
broker_url_api:
file: ./secrets/broker_url_api
broker_url_worker:
file: ./secrets/broker_url_worker
s3_reconciler_access_key_id:
file: ./secrets/s3_reconciler_access_key_id
s3_reconciler_secret_access_key:
file: ./secrets/s3_reconciler_secret_access_key
volumes:
pgdata:
valkeydata:
s3data:
broker-data:
networks:
edge:
-21
View File
@@ -1,21 +0,0 @@
# Состояния вложения и договор между producer/worker
Сообщение broker соответствует [jobs.schema.json](jobs.schema.json). Можно применять native headers брокера для trace context, но семантика carrier сохраняется; для проверки покажите отображение полей. `job_id` идентифицирует одну логическую работу и не меняется при redelivery; `attempt` начинается с 1 и отражает управляемую попытку обработки (broker redelivery сам по себе не создаёт новый job_id). Ключ эффекта — `(attachment_id, pipeline_version, variant)`.
| Переход | Условие | Что устойчиво сохранено |
| --- | --- | --- |
| нет → queued | Принимаем upload; только после durable handoff отвечаем 202 | Полный проверенный оригинал в staging, метаданные, намерение обработки |
| queued → processing | Worker захватил/арендовал работу | Попытка, lease/deadline или эквивалентная защита |
| processing → queued | Временная ошибка и остались попытки | Причина без секретов, время следующей попытки |
| processing → ready | Все обязательные объекты записаны и результат атомарно опубликован | Оригинал, thumbnail (на 4 ещё crop/optimized), метаданные/checksums |
| processing → failed | Ошибка постоянная или попытки исчерпаны | error_code и запись, доступная оператору |
| processing → queued | Истёк lease после смерти worker | Восстановленная работа, без дублирования эффектов |
| failed → queued | Явный операторский replay, доступен на 4 | Audit, новая попытка, прежний логический эффект защищён от дубля |
Повторная доставка уже ready-задачи — no-op с ack после проверки завершённого эффекта. Конкурентные попытки не перезаписывают более новый pipeline_version. Основной курс использует pipeline_version=1; версионирование нескольких pipeline — трек 5. Заявляйте готовность только когда все обещанные уровнем варианты доступны. Нельзя откатывать ready в processing из-за старого дубля.
**Окна отказа для защиты:** после staging до записи задания; после COMMIT до publish; после publish до confirm; после записи S3 до COMMIT результата; после COMMIT результата до ack. На 3 покажите сохранность успешно принятой работы и повторную доставку; на 4 автоматическое восстановление разрыва DB/broker (outbox/эквивалент) и зависшего processing.
Staging не удаляется до безопасной публикации результатов. Неуспешные/непривязанные исходники имеют документированный retention и cleanup; он не удаляет текущую работу. Broker принимает только ID/служебные данные, не произвольный URL для скачивания и не путь из клиентского filename. Worker берёт доверенные storage metadata из БД.
Ack подтверждает устойчивую обработку, publisher confirm — приём брокером; это разные границы. См. [RabbitMQ reliability](https://www.rabbitmq.com/docs/reliability). At-least-once + идемпотентный эффект не означает exactly-once доставку.
+2 -4
View File
@@ -34,8 +34,7 @@
"chat_id": "33333333-3333-4333-8333-333333333333",
"sender_id": "11111111-1111-4111-8111-111111111111",
"text": "Привет, Боб!",
"created_at": "2026-09-01T12:00:00Z",
"attachment_ids": []
"created_at": "2026-09-01T12:00:00Z"
},
"MessagePage": {
"items": [
@@ -44,8 +43,7 @@
"chat_id": "33333333-3333-4333-8333-333333333333",
"sender_id": "11111111-1111-4111-8111-111111111111",
"text": "Привет, Боб!",
"created_at": "2026-09-01T12:00:00Z",
"attachment_ids": []
"created_at": "2026-09-01T12:00:00Z"
}
],
"next_cursor": null
+1 -27
View File
@@ -1,5 +1,5 @@
{
"lab": 5,
"lab": 3,
"kind": "requirements, not runnable deployment",
"required_for_grade_3": [
{
@@ -17,32 +17,6 @@
"role": "load_balancer",
"public_entrypoint": true,
"backend_readiness": true
},
{
"role": "s3",
"choose_one": [
"MinIO",
"SeaweedFS",
"RustFS"
],
"private_bucket": true,
"persistent_volume": true
},
{
"role": "broker",
"choose_one": [
"RabbitMQ",
"Redis Streams",
"Valkey Streams",
"NATS JetStream",
"documented durable equivalent"
],
"persistent_volume": true,
"delivery": "at-least-once"
},
{
"role": "worker",
"separate_process": true
}
],
"optional_components": [
-63
View File
@@ -1,63 +0,0 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "urn:mtusi:messenger:attachment-job:v1",
"title": "Durable attachment processing job v1",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"job_id",
"type",
"attachment_id",
"pipeline_version",
"attempt",
"created_at"
],
"properties": {
"schema_version": {
"const": 1
},
"job_id": {
"type": "string",
"format": "uuid"
},
"type": {
"const": "attachment.process.v1"
},
"attachment_id": {
"type": "string",
"format": "uuid"
},
"pipeline_version": {
"type": "integer",
"minimum": 1
},
"attempt": {
"type": "integer",
"minimum": 1
},
"created_at": {
"type": "string",
"format": "date-time"
},
"traceparent": {
"type": "string",
"pattern": "^00-[0-9a-f]{32}-[0-9a-f]{16}-[0-9a-f]{2}$"
},
"tracestate": {
"type": "string",
"maxLength": 512
}
},
"examples": [
{
"schema_version": 1,
"job_id": "11111111-1111-4111-8111-111111111111",
"type": "attachment.process.v1",
"attachment_id": "22222222-2222-4222-8222-222222222222",
"pipeline_version": 1,
"attempt": 1,
"created_at": "2026-09-01T12:00:00Z"
}
]
}
+7 -991
View File
File diff suppressed because it is too large Load Diff
+197 -36
View File
@@ -1,48 +1,209 @@
# Индекс доказательств ЛР5
date: 2026-11-06 commit: b063c8780eabf63cdb9da3802f82440750308d6b (validation documentation worktree)
Актуальный [REPORT.md](../REPORT.md) обновлён после независимого ревью **6096aa7948b8b666aa3892397a96d044a55f02c6 (M)**. Предыдущее review — **42575281b352d35bda17ee7601bae5548a996e91 (R)**; полные динамические серии — **0e921116610d289716af0a40ec823376eb8bbcbc (D)**. Validation на `b30e2f6a84419d1ca64525ffb40df92183cccaef` (V, runtime = R) подтвердила PostgreSQL outage, delivery subset и HTTP tRNS, но выявила четыре FAIL на 40 Мп PNG при штатной памяти.
# Доказательства
В M изменён только renderer: плитки 256×256 и явное освобождение pixel core. 440 offline + 20 границ PASS, 2 Linux-only skips; 132 варианта совпали с V. Re-review не нашло major в статике/offline; **прежний major закрыт Linux/HTTP validation F** (`74c33160ac01618f70e9e1c6a121a5e45c3c7155`, runtime = M): пять граничных PNG, concurrency=2, принудительные лимиты и HTTP 8/48 PASS. 984 student tests passed / 2 Linux-only skips; Linux-эквивалент выполнен отдельно. Исходные FAIL V сохраняются и не становятся PASS. Аудит применимости V — [load-validation.md](load-validation.md), исходная инструкция адресных проверок — [RESULTS исправления памяти](review-png-memory-2026-11-07/RESULTS.md).
Финальная validation — [load-validation.md](load-validation.md) и
[final-validation/README.md](final-validation/README.md). Проверяемый checkout `b063c8780eabf63cdb9da3802f82440750308d6b`
содержит тот же код, что review `8c9ec653ed9d0722bccecd4690def220a803c969`.
Полные 135 измерений сохранены и приняты без повторного запуска; новый after
проверяет только затронутый путь. Student-test: 372 passed; stack: 12 passed.
Smoke/persistence/failover, detection и rolling прошли; сырые результаты не смешаны
с прежними SHA. Публикации не было, minor с суррогатами остаётся.
## Как читать каталог
[report-after-review.md](report-after-review.md) — исторический этап отчёта до validation:
тогда Docker/student-test были запрещены. `openspec/config.yaml` не меняется на текущем
этапе; его исторический контекст не заменяет актуальный REPORT.
| Файлы / каталог | Код и роль доказательства | Применимость |
Следующие заметки описывают именно этапы исправлений; перечисленные там динамические
проверки теперь выполнены в final-validation.
Не-ASCII в Basic после третьего ревью — [review-non-ascii-basic/README.md](review-non-ascii-basic/README.md),
код `7e19d4e33ee817bca0649f47c75bb0ff42972e52`: 144 unit-теста offline,
четыре HTTP-регрессии только собраны. Обязательный прогон с PostgreSQL и через LB
передан финальному агенту; новых нагрузочных результатов этот этап не содержит.
NUL во входных строках после повторного ревью — [review-nul-inputs/README.md](review-nul-inputs/README.md),
код `e66e2d7d685ef8cdef74b3104bd6b5550b01c0af`: 136 unit-тестов и 18 архитектурных проверок
offline, 17 HTTP-регрессий только собраны. Там же обязательные проверки финального агента;
динамические свойства и нагрузка на исправленном SHA здесь не подтверждаются.
Исправления после независимого ревью — [review-fixes/README.md](review-fixes/README.md),
код `2af24d7762a87173aa2d34c3464c64184b3ee8bf`. Здесь выполнены только offline-проверки;
ниже сохранены измерения прежнего кода. Обязательные динамические проверки нового SHA
перечислены в заметке для финального агента.
Обезличенные реальные прогоны; пользователи, сообщения и пароли тестов синтетические. Токены,
содержимое secrets и дампы не сохраняются. В первой строке текстовых логов — UTC и SHA;
`grade3-*.txt` и `grade4-*.txt` также содержат команду, окружение, код завершения и длительность.
В текстовом выводе удалены пробелы на концах строк Docker/psql; локальный путь заменён на `<repo>`.
В grade4-student-tests.txt замаскирован синтетический неверный Bearer из имени параметризованного теста.
## Сохранённый итог до исправлений ревью
На основании измерений прежнего кода ранее заявлялась «5», трек 1 «Цена масштабирования».
Автор — Костылев Артём, БПИ2303. Все источники таблицы ниже относятся к версии **до ревью**;
grade3/grade4 также предшествуют кэшированию SQL. Это не прогоны текущего HEAD.
| Что проверялось | Сохранённый источник до ревью | Код |
| --- | --- | --- |
| [Адресная validation M](final-memory-validation-2026-11-07/RESULTS.md) | F, runtime = M | Linux 448 MiB/30 с, пять 40 Мп PNG × v1/v2; HTTP boundary 8 сценариев / 8 инвариантов и малая матрица 8/48 / 8 инвариантов; 19 ID / 115 объектов; major закрыт, семь minor сохранены |
| [Исправление памяти](review-png-memory-2026-11-07/RESULTS.md) | M; реализация `1a8020524909ebc5e2e43303436c9d1ce3c1a87b` | 440 + 20 passed / 2 skips, счётчик pixel buffers, 132 идентичных варианта; 28 хешей. Linux/HTTP выполняет следующий каталог validation F |
| [Аудит обновления отчёта](report-memory-audit-2026-11-07.md) | M + только документация | Лёгкие проверки, аудит новых хешей/чисел/ссылок; семь minor последнего review, без новых динамических прогонов |
| [Финальная validation](final-validation-2026-11-07/RESULTS.md), [аудит нагрузки](load-validation.md) | V, runtime = R | 980 tests, HTTP PNG и real DB outage; 12 адресных сценариев PASS; четыре 40 Мп PNG FAIL, один major; полный track2 не повторён |
| [Первый аудит отчёта](report-audit-2026-11-07.md) | R + изменения документации, до validation V и исправления M | Реальные лёгкие проверки того этапа, ссылки, хеши, арифметика; без Docker и новых динамических тестов |
| [forward-final-2026-11-06/RESULTS.md](forward-final-2026-11-06/RESULTS.md) | D, полная динамическая квалификация после поздних исправлений ЛР4 | Основная измеренная база: 819 passed / 1 skip, smoke/persistence/failover, уровень 4 и три повтора трека 2; до исправлений R |
| [review-trns-depth-2026-11-06/RESULTS.md](review-trns-depth-2026-11-06/RESULTS.md) | R, исправление глубины PNG tRNS | 430 unit + 20 границ, 114 совместимых вариантов, fixtures и 34 хеша; offline, не Linux/HTTP-квалификация |
| [review-fixes-2026-11-06/RESULTS.md](review-fixes-2026-11-06/RESULTS.md) | Реализация `3fad49d6b913afb122d2c1348ea8d9f99e25cc27`, архив `6e97bdbe9638c5a53951337fce639a1a0db7f92e` | История первого исправления: policy сохранена в R, renderer затем дополнен. 57 тестов не доказывали все tRNS; поздние major и исправление описаны в следующем каталоге |
| [forward-port.md](forward-port.md), [source-inventory.json](forward-final-2026-11-06/source-inventory.json), [forward-audit.json](forward-final-2026-11-06/forward-audit.json) | Перенос ЛР4 `d7478726a85791c21e065d58f610735a9e687433` → `4ed889c1e4008e3f6040ae2534bef99262d3b8ce` | Решения по 253 файлам и 25 хешей на D. Сверяется snapshot, не другой живой репозиторий |
| [sqlalchemy-2026-11-06/grade3/NOTES.md](sqlalchemy-2026-11-06/grade3/NOTES.md) | `24a51744f521aa585e50a931d0a0219d2e960197` | История после перехода на SQLAlchemy, до исправлений track2/forward/review; topology/ack — дополнительное свидетельство, не текущая policy |
| [sqlalchemy-2026-11-06/grade4/NOTES.md](sqlalchemy-2026-11-06/grade4/NOTES.md) | `e9ce6d0afacadc16f1666f4bb7d54d3d7377a269` | Исторический уровень 4; `level4-retry.json` — успешный повтор; исходные неуспешные результаты не переименовываются в PASS |
| [sqlalchemy-2026-11-06/track2/NOTES.md](sqlalchemy-2026-11-06/track2/NOTES.md) | Несколько промежуточных SHA внутри логов | История диагностики и исправлений; для итоговых чисел используется qualified, затем D |
| [sqlalchemy-2026-11-06/track2/qualified/RESULTS.md](sqlalchemy-2026-11-06/track2/qualified/RESULTS.md) | `9ba3076fa8ba707de932e053fc22f455d83b3210` | Старая квалификация 39/174 до позднего forward: run1, run2, run3-retry. Прерванный run3 исключён, повторена только эта серия |
| Все корневые `*.txt` и `*.json` в `evidence/` | Историческая реализация до SQLAlchemy/Alembic, даты/старые SHA — внутри файлов и истории Git | **Не подтверждают R, D или M.** Не использовать старые порты, asyncpg-мигратор, результаты и формулировки как текущие |
| Уровни «3»/«4», публичные проверки, отказы зависимостей | [Команды](track1-sqlalchemy/revalidation/commands.txt), [сценарии](track1-sqlalchemy/revalidation/) | `3fb741e`, API идентичен `57744c5` |
| Каждая реплика: времена, ошибки, потери, реальные контейнеры | [a855a190991a](track1-sqlalchemy/revalidation/failover-each/a855a190991a/result.json), [d4dfe6128c9d](track1-sqlalchemy/revalidation/failover-each/d4dfe6128c9d/result.json); `action.txt` рядом | `3fb741e` |
| Rolling под нагрузкой, один migrate, история | [Результат](track1-sqlalchemy/revalidation/rolling/result.json), [действие](track1-sqlalchemy/revalidation/rolling/action.txt) | `5c8f605` → API `57744c5`, обвязка `3fb741e` |
| 135 измерений, методика и подтверждённое улучшение | [Методика](track1-sqlalchemy/README.md), [аудит](track1-sqlalchemy/audit.txt), [сводка](track1-sqlalchemy/plots/capacity.md) | baseline `5c8f605`, after `57744c5` |
| Исправление сохранения диагностических журналов | [Повторные 290 tests и smoke](track1-sqlalchemy/diagnostics/setup-log-fix-checks.txt) | `55b99a2`; API не менялся |
| Проверки первого отчёта, коммит `4513d90` | [check/validate/lint/student-test/OpenSpec](report-checks.txt), [ссылки, метрики и состав изменений](report-audit.txt) | `f591fa64e525679a31c9db7f942d36c789e1a12f` + только документы |
Последняя строка включает `contract-checks.txt`, `environment.txt`, `failover-upload.json`, `failover.txt`, `level3.json`, `level4.json`, `limits.txt`, `operator.txt`, `persistence.txt`, `pipeline-stand.txt`, `s3-outage.json`, `s3-security.txt`, `smoke.txt`, `stack.txt`, `student-tests.txt`, `topology.txt`, `upload-faults.txt`. Эти файлы оставлены для истории, без изменения сырья. Старые Markdown-итоги также читаются в пределах указанного выше SHA; их прежние заявления о полноте не распространяются на M. В частности, требования «нужен fix/review» в load-validation.md и RESULTS V описывают историческую остановку: fix и re-review M уже выполнены, Linux/HTTP-подтверждение завершено на F; старые документы не переписаны.
На первом этапе report (`4513d90`, до ревью) нагрузка заново не снималась: код приложения,
миграции, образы и конфигурация стенда тогда не изменялись. Ссылки и численные выводы сверены с сохранённым
сырьём; собственные тесты выполнены заново. Журналы содержат фактические коды выхода,
включая неуспехи. Этот исторический журнал не описывает нынешний этап после ревью.
## Основные первичные данные D
## Уровень «3» после SQLAlchemy, до оптимизации — 2026-11-05
- [level4-command.txt](forward-final-2026-11-06/level4-command.txt), [level4.json](forward-final-2026-11-06/level4.json): 17 сценариев, 8 инвариантов; 33 ID, 28 ready, 5 ожидаемых failed, 116 объектов.
- [track-run1.txt](forward-final-2026-11-06/track-run1.txt), [track-run1.json](forward-final-2026-11-06/track-run1.json); [track-run2.txt](forward-final-2026-11-06/track-run2.txt), [track-run2.json](forward-final-2026-11-06/track-run2.json); [track-run3.txt](forward-final-2026-11-06/track-run3.txt), [track-run3.json](forward-final-2026-11-06/track-run3.json): каждый 13 сценариев, 8 инвариантов, 13 ID, 58 объектов. [summary.json](forward-final-2026-11-06/summary.json): всего 39/24/39/174.
- [final-smoke.txt](forward-final-2026-11-06/final-smoke.txt), [persistence.txt](forward-final-2026-11-06/persistence.txt), [failover.txt](forward-final-2026-11-06/failover.txt): публичные проверки. [persistence-containers.txt](forward-final-2026-11-06/persistence-containers.txt), [failover-containers.txt](forward-final-2026-11-06/failover-containers.txt), [restore.txt](forward-final-2026-11-06/restore.txt): фактические операции со своим стендом.
- [student-test.txt](forward-final-2026-11-06/student-test.txt): 819 passed / 1 skipped до последних двух исправлений. Предыдущие offline-результаты R: [offline-tests.txt](review-trns-depth-2026-11-06/offline-tests.txt), [boundaries.txt](review-trns-depth-2026-11-06/boundaries.txt). Текущие M: [offline-tests.txt](review-png-memory-2026-11-07/offline-tests.txt), [boundaries.txt](review-png-memory-2026-11-07/boundaries.txt).
- [conditions-before.json](forward-final-2026-11-06/conditions-before.json), [conditions-track-start.json](forward-final-2026-11-06/conditions-track-start.json), остальные `conditions-*.json`: моментальные срезы общего Docker, не непрерывный контроль фоновой нагрузки.
- [down.txt](forward-final-2026-11-06/down.txt), [stopped-containers.txt](forward-final-2026-11-06/stopped-containers.txt), [retained-volumes.txt](forward-final-2026-11-06/retained-volumes.txt): остановка стенда D. Более поздняя остановка V — [down.txt](final-validation-2026-11-07/down.txt), [stopped-containers.txt](final-validation-2026-11-07/stopped-containers.txt), [retained-volumes.txt](final-validation-2026-11-07/retained-volumes.txt): четыре volume сохранены. Исправление M и отчёт Docker не запускали. Последняя остановка F — [down.txt](final-memory-validation-2026-11-07/down.txt), [пустой project](final-memory-validation-2026-11-07/stopped-containers.txt), [четыре volumes](final-memory-validation-2026-11-07/retained-volumes.txt).
Код всех итоговых прогонов: **`0a1f48bd73e5780c732356880c23fdb476bcff98`**.
Compose-проект `mtusi2026lab3-grade3-final-20261105`, `LB_HTTP_PORT=18083`, `LB_STATS_PORT=23083`,
`API_REPLICAS=2`; BASE_URL автоматически определяется по порту LB. Обе реплики:
api-1 `8ffe0c70f4ea`, api-2 `1b4a73f61b07`. После прогонов контейнеры остановлены,
volumes сохранены. До конца grade4 код приложения после этого SHA не менялся;
на этапе grade4 усилена обвязка `scripts/probe.py`, на track5 добавлен кэш компиляции SQL.
## Не считать успехом или измерением производительности
| Файл | Команда | Результат |
| --- | --- | --- |
| `grade3-start.txt` | `make up`, `docker compose ps -a`, чтение `alembic_version` | Новый проект без прежних volumes, сборка успешна, 2 healthy API, one-shot Alembic вышел с 0, head `0002_idempotency_keys` |
| `checks.txt` | `make check`, `make validate`, `make lint`, `uv lock --check`, `openspec validate --all --strict`, diff против upstream | Все успешно; нормативные файлы не менялись |
| `smoke.txt` | `make test` | Публичный smoke PASS |
| `stack.txt`, `environment.txt`, `grade3-stack.txt` | `make stack-evidence` | Порты только у LB, реальные контейнеры и hostname, 2 UP, общие PostgreSQL/Valkey, uid, версии библиотек |
| `grade3-lb-check.txt` (актуальный `lb-check.json` — повтор grade4) | `make lb-check` | Один токен 20 + 20 GET; 20 × 8 POST, 160 ответов 201, по одной копии, 20 конфликтов другого body; pass |
| `retry.json`, `grade3-retry.txt` | `make retry-check` | Потерянный ответ при живой/убитой реплике, повтор через соседа 201, одна копия; pass |
| `failover-current-1.txt`, `failover-current-2.txt` | `make test-failover ACTION_SCRIPT=scripts/stop-one.sh` и `docker compose ps -a` | По очереди api-1 и api-2 Exited (0); 129/128 записей, 0/0 ошибок, восстановление 2.762/2.817 с |
| `grade3-restore-1.txt`, `grade3-restore-2.txt` | `make restore` | Остановленные реплики возвращены в ротацию |
| `persistence.txt`, `grade3-persistence.txt` | `make persistence-evidence` → публичный `make test-persistence ACTION_SCRIPT=scripts/restart.sh` | PASS, IDs прежние, StartedAt всех сервисов обновились, токен и история сохранились |
| `faults.txt` | `make test-faults` | 12 passed; после PostgreSQL 40 регистраций через 2 реплики без рестарта API |
| `student-tests.txt` | `make student-test PYTEST_ARGS=-vv` | 271 passed, 12 deselected; стендовые 12 запущены отдельно |
| `schema.txt`, `migrations.txt`, `grade3-storage.txt` | `make storage-evidence` | Схема Alembic, повтор без изменения 70 users / 24 sessions / 6 chats / 12 members / 284 messages / 284 keys / 1 version |
| `kdf.txt`, `secrets.txt` | `make storage-evidence` | Argon2id, хеши токенов 32 байта; секреты/.env вне Git |
| `grade3-final-review.txt` | Финальные проверки после обновления отчёта и архива | check/validate/lint/OpenSpec, diff нормативных и исполняемых файлов, отсутствие контейнеров этапа |
| `grade3-stop.txt` | `make down`, `docker compose ps -a`, volumes проектов | Контейнеров нет, volumes обоих прогонов сохранены |
- `crash_publish_confirm` и raw `window: after publish, before confirm` — историческое имя/описание; реальная точка находится **после Basic.Ack и до COMMIT outbox**. Потерю confirm данный сценарий не проверял.
- [grade3/limits.txt](sqlalchemy-2026-11-06/grade3/limits.txt) имеет **exit_code=1**, два ложных FAIL из-за порядка secrets. Это не PASS.
- [clock-observation.json](forward-final-2026-11-06/clock-observation.json): UTC/elapsed run2 = 1161.195/186.458 с, run3 = 571.513/180.259 с. Причина неизвестна; длительности исключены из RPS/SLO/сравнения разброса. Функциональные результаты пригодны.
- `no_orphans` проверяет выборку ID; на D сохранены 15 исторических/ожидаемых сообщений DLQ, после malformed-сценария V их 16; в F это число не выросло. Это не глобально пустые S3/DLQ.
- `forward-final-2026-11-06/baseline.txt` — 0 тестов; `baseline-regressions.txt` — ожидаемые 29 FAIL до исправления. `review-fixes-2026-11-06/regressions-fixture-error.txt` — неуспешная промежуточная fixture; `review-trns-depth-2026-11-06/regressions-before.txt` — воспроизведённый дефект. Их сохраняют как диагностику, не суммируют с PASS.
- Исторический [findings.json](review-trns-depth-2026-11-06/findings.json) содержит пять minor: окно confirm, limits, legacy reconcile, отдельный replay failed v2 и часы. Дополнительная документационная неточность `201` в `openspec/config.yaml` отражена в REPORT; конфиг этой фазе менять запрещено. Седьмой minor из review M — JobMessage принимает четыре значения вне JSON Schema; источник и ограничения — [аудит отчёта](report-memory-audit-2026-11-07.md). Исторические findings.json с пятью/шестью minor не перезаписываются.
Исходный JSON уровня «3» сохранён в `grade3-lb-check.txt` и `grade3-retry.txt` вместе с датой и полным SHA.
`lb-check.json` обновлён повторным прогоном grade4; `retry.json` остаётся результатом grade3.
Нагрузочных RPS/p95 для M здесь не заявлено. Финальный агент выполнил только отсутствующие/затронутые проверки и сохранил пригодные серии. После re-review M повторены только затронутые PNG/ресурсные проверки: четыре 40 Мп tRNS и grayscale16 без tRNS с импортами worker, HTTP subset 8/48, concurrency=2, v1/v2, принудительные лимиты и cleanup после sweep. Выполненная процедура и результаты — [validation F](final-memory-validation-2026-11-07/RESULTS.md). PostgreSQL outage, delivery subset и полный track2 повторно не нужны без нового основания.
## Неуспешные промежуточные прогоны
Сохранять только обезличенные команды, exit codes, SHA, условия и агрегаты. Секреты, `.env`, credentials, session tokens/cookies, реальные сообщения, дампы и signed URLs в evidence не входят. Существующие сырые измерения неизменны; новые свидетельства записаны отдельно. Validation завершена, публикация не выполнялась и требует отдельной фазы.
Они сохранены отдельно, не выдаются за итоговые PASS.
| Файл | Код / состояние | Наблюдение и исправление |
| --- | --- | --- |
| `grade3-faults.txt` | `7fc911c332aefe33285ea9e7811c37abd9bb20a6` | 11 passed / 1 failed: POST users → 503 после PostgreSQL; включён `pool_pre_ping` |
| `grade3-pool-regression-before.txt` | `18de595` + новый тест (приложение прежнее) | Первый POST после `pg_terminate_backend` → 503 вместо 201 |
| `grade3-pool-regression-after.txt` | `18de595` + исправления до коммита `0a1f48b` | 2 passed: закрытое соединение и SIGTERM; затем полный набор проверен на `0a1f48b` |
| `grade3-student-tests-interrupted.txt` | `7fc911c` | 46 passed, затем прерывание: gunicorn теста использовал порт LB 18083; теперь свободный 23000–23999 и дедлайн |
| `grade3-student-tests-startup.txt.gz` | `18de595` + исправления до `0a1f48b` | Полный обезличенный вывод: 70 passed / 201 errors, SQLSTATE 57P03 при старте fixture БД; временный отказ теперь повторяется до дедлайна |
| `grade3-whitespace-before.txt` | `0a1f48b` + evidence/OpenSpec | `git diff --check` нашёл пробелы в конце строк Docker/psql; после нормализации итоговая проверка прошла |
| `grade3-initial-stop.txt` | `18de595` + WIP | Остановка исходного проекта без удаления его volumes перед итоговым прогоном |
`clean-clone.txt`, `hooks.txt`, `failover-fe75c6c37e5a.txt`, `failover-ff81f93e8b0d.txt`
и `rolling-from-8d28d45/` — история до SQLAlchemy. Прежние версии перезаписанных доказательств
доступны через `git show 72a1e57:evidence/<имя>`. Новые измерения трека приведены ниже;
исходный SQLAlchemy change завершён и архивирован 2026-11-06.
## Уровень «4» после SQLAlchemy, до оптимизации — 2026-11-05
Код прогонов: **`252e3e3ffeea5004cfd9dc342e686af1417a77a2`**.
Проект `mtusi2026lab3-grade4-20261105`, порты `18083/23083`, две реплики по 1 CPU/512 MiB,
два воркера и пул 8 на воркер; HAProxy maxconn 64. Локальный HTTP явно обходит системный прокси
через `NO_PROXY=localhost,127.0.0.1,::1`. Команды и исходные результаты сохранены ниже.
JSON содержит поля date/commit; первая строка CSV — служебный комментарий с UTC/SHA,
который следует пропустить при чтении таблицы. Эти метаданные добавлены после прогонов;
результаты не изменялись. Из текстового вывода убраны пробелы на концах строк.
| Файл | Команда | Результат |
| --- | --- | --- |
| `grade4-start.txt` | `make up`, версии пакетов, `docker compose ps -a`, Alembic head | Новый проект с пустыми volumes; SQLAlchemy 2.0.45 / Alembic 1.16.5; migrate Exited (0), head `0002_idempotency_keys` |
| `grade4-baseline.txt`, `lb-check.json`, `limits.txt` | `make test`, `make limits`, `make lb-check` | Smoke PASS; порты только LB; 160 ответов 201 через обе реплики, 20/20 одинаковых тел/единственных копий/конфликтов другого body |
| `grade4-drain.txt` | `make student-test PYTEST_ARGS='-v -s student-tests/api/test_drain.py student-tests/unit/test_timeouts.py'` | 3 passed; ready/live 503/200; порт закрыт через 1.05 с, выход через 2.44 с; принятый запрос → 201, зависший → 503 с заголовками контракта |
| `grade4-detection.txt`, `detection/run{1,2,3}/*.json` | `make detection` | Все 12 PASS, диапазоны и пределы — в REPORT |
| `grade4-failover-each.txt`, `failover-each/` | `make failover-each RATE=100` | Каждая реплика Exited (0) и возвращена; по 6001 запросу, 0 ошибок/dropped; 645/642 сообщения зонда полностью сохранены |
| `grade4-runtime-limits.txt` | `docker ps -a` и `docker inspect` во время k6 | Фактические CPU/RAM, uid, image ID, порты и read_only всех сервисов, включая k6 и завершённый migrate |
| `grade4-limits-negative.txt`, `limits-violations.txt` | `scripts/limits.py` с временным override из лога, `API_REPLICAS=1` только для config | Ожидаемый exit 1, все 5 нарушений найдены; override никогда не применялся к контейнерам |
| `grade4-rolling.txt`, `rolling/` | `make rolling-update RATE=100` | Новые реплики готовы до остановки прежних; один migrate, минимум 2 UP, все checks PASS |
| `grade4-rolling-from-7fc911c.txt`, `rolling-from-7fc911c/` | `make rolling-update FROM_REV=7fc911c RATE=100` | Переход между SQLAlchemy-версиями; старый/новый image ID каждой реплики в action.txt |
| `grade4-migration-repeat.txt` | Повторный `make migrate`, Alembic head и счётчики всех таблиц до/после | Результат записан отдельно; миграция выполняется при остановленной нагрузке |
| `grade4-persistence-scoped.txt` | Публичный persistence и docker inspect до/после | IDs контейнеров прежние, StartedAt всех lb/api/postgres/valkey/migrate обновлены |
| `grade4-public.txt` | Публичные smoke/persistence/failover, restore и реальные контейнеры | Итоговые проверки после rolling update |
| `grade4-evidence-audit.txt` | Сверка result.json с сырыми CSV, проверка количества миграторов, поиск похожих на credentials строк | Все 12 detection и 4 нагрузки PASS; численные поля совпадают; один migrate на rolling |
| `grade4-checks.txt`, `grade4-student-tests.txt` | check/validate/lint/lock/OpenSpec, полный student-test | Итоговые проверки на коде stage4 |
| `grade4-stop.txt` | `make down`, список контейнеров и volumes проекта | Контейнеры остановлены, volumes сохранены |
Промежуточные неуспехи не заменяют итоговые результаты:
| Файл | Наблюдение |
| --- | --- |
| `grade4-evidence-audit-scan-failed.txt`, `grade4-evidence-audit-fixture-token.txt` | Численные сверки PASS. Первый поиск нашёл собственный regex; после уточнения — синтетический неверный Bearer в имени теста. Имя параметра замаскировано, итоговый поиск прошёл |
| `grade4-development.txt` | Первоначальные E501/ARG005, исправления форматирования, 14 новых регрессионных тестов; уточнение library ID Context7 |
| `grade4-baseline-proxy-failed.txt` | Smoke: RemoteDisconnected из-за системного прокси; прямой curl отвечал 200; повтор с NO_PROXY прошёл |
| `grade4-detection-proxy-interrupted.txt`, `grade4-detection-proxy-interrupted/` | Прогон без NO_PROXY: статистика LB недоступна, SIGTERM честно fail; остановлен во время kill, реплики восстановлены; выполнен полный новый прогон |
| `grade4-public-events-unavailable.txt` | Дополнительная ретроспективная проверка через docker events не подтвердила рестарт: в оставшемся буфере не было событий start/die нужного окна. Persistence повторён с непосредственным снимком StartedAt до/после |
| `grade4-limits-negative-config-failed.txt`, `grade4-limits-compose-rejection.txt` | Compose отклонил сочетание container_name и replicas=2 до проверки политики; отрицательная fixture повторена с replicas=1 (конфигурация не применялась) |
## Трек «5» на SQLAlchemy до исправлений ревью — 2026-11-05–06 (МСК)
Производительность `57744c5` измерена в `track1-sqlalchemy/`; исторический `track1/`
не входит в новые медианы и графики. API baseline `5c8f605`, after `57744c5`;
код API не менялся в последующих коммитах обвязки. Графики — `3fb741e`.
Гипотеза: повторная компиляция CTE расходует CPU API; статическое кэширование сохраняет SQL.
| Файл | Команда | Наблюдение |
| --- | --- | --- |
| `track1-sqlalchemy/README.md`, `host.txt`, `runtime-resources.json` | `make track1-host`; снимок лимитов | Методика, версии, собственный project, CPU pinning, ограничения и происхождение образов |
| `track1-sqlalchemy/series-a/` | `make track1-a` | 54 точки: 1×4/2×2/4×1 при 4 CPU/2 GiB; пики 2687/2532/2422 RPS |
| `track1-sqlalchemy/series-b/` | `make track1-b` | 36 точек: 1×1/2×1/8×1; вместе с 4×1 из A — сублинейный рост до 3420 RPS |
| `track1-sqlalchemy/compare/` | `make track1-compare` | 18 чередующихся точек: при 2400 p95 14.28→4.63 мс; при 5400 throughput +8.7%; min–max не пересекаются |
| `track1-sqlalchemy/control-pool/` | первая часть `make track1-control` | 15 точек: пул 16 добавляет Lock и снижает RPS на 13.7%; увеличение maxconn не помогает |
| `track1-sqlalchemy/control-cpus/` | CPU-рецепт `make track1-control` после исправления, `corrected-cpus-run.txt` | 12 чередующихся точек на одинаковых 4 CPU/2 GiB; 8×0.5 медленнее 4×1 |
| `track1-sqlalchemy/plots/`, `plots-run.txt` | `make plots` | 12 просмотренных PNG, сводка capacity.csv/md; диапазоны повторов, без старых RPS |
| `track1-sqlalchemy/audit.txt` | полный скрипт и команда внутри | 135 основных точек, 8 271 783 ответа; метрики сверены с k6, image IDs и реальными CPU/RAM/cpuset; внешних помех 0 |
| `track1-sqlalchemy/explain-history.txt` | SQLAlchemy EXPLAIN ANALYZE, скрипт внутри | Первая страница и keyset используют messages_history_idx; ограничение custom plan отмечено |
| `track1-sqlalchemy/revalidation/` | команды и exit codes в `commands.txt` | 290 tests, 12 faults, публичные сценарии, 12/12 detection, оба failover и rolling PASS на `3fb741e`; история сохранена |
| `track1-sqlalchemy/openspec-close.txt` | синхронизация specs, архивация, strict validation | Оба change завершены: 11 и 17 задач; перед архивом 16/16, после 14/14; активных changes нет |
| `track1-sqlalchemy/final-checks.txt` | итоговые команды перед коммитом | Проверки окончательных документов и состава evidence |
Стек остановлен только в своём project, volumes сохранены (`track1-sqlalchemy/stop.txt`).
Промежуточные неуспехи и исключённые результаты сохранены:
- `diagnostics/staged-whitespace-failed.txt`: проверка staged-файлов нашла CRLF
в CSV и пустые строки в конце журналов; нормализованы, равенство CSV-полей проверено.
- `series-b/compose-up-failed.txt`, `compare/compose-up-failed.txt`: три ошибки LB unhealthy
при подготовке, до k6; retry восстановил готовность. В `55b99a2` исправлено сохранение
будущих журналов, повторные 290 tests и smoke прошли (`diagnostics/setup-log-fix-checks.txt`).
- `diagnostics/final-audit-log-headers-failed.txt`: аудит обнаружил игнорируемые .log;
журналы сохранены с provenance, финальный `final-audit.txt` прошёл.
- `diagnostics/ordered-control-cpus/`: 12 завершённых замеров с группировкой конфигураций;
исключены из основных графиков и полностью повторены с чередованием.
- `diagnostics/order-regression.txt`: тест воспроизвёл неверный порядок (1 failed, 1 passed);
`order-fix-checks.txt`: после исправления check/lint, 290 passed и smoke.
- `diagnostics/audit-image-tag-failed.txt`: неверное сравнение старого OCI index с текущим
тегом; исправленный аудит использует build ID соответствующего журнала сборки.
- `diagnostics/explain-driver-uuid-failed.txt`: диагностический скрипт не преобразовал UUID
драйвера в стандартный UUID для literal; повтор соответствует существующему адаптеру и проходит.
- `diagnostics/pilot/`: пилот по одному повтору, в выводы о разбросе не входит.
- При пуле 16 в перегрузке были 0.009–0.016% HTTP-ошибок; они остаются в summary и отчёте.
Предупреждения Docker CLI, первоначальные lint/OpenSpec ошибки и их исправления описаны
в `track1-sqlalchemy/README.md`; данные не выданы за проверки без ошибок.
## Трек 1 «Цена масштабирования» — исторические результаты до SQLAlchemy
`track1/` — серии A и B, сравнение до/после, контрольные прогоны, графики и проверки после улучшения; состав и поля CSV —
`track1/README.md`.
| Файл | Что доказывает | Команда | Коммит (код) |
| --- | --- | --- | --- |
| `track1/host.txt` | снимок хоста, версии и соседние контейнеры перед сериями | `make track1-host` | `67f6d68` |
| `track1/series-a/` | 1×4, 2×2, 4×1 при 4 CPU / 2 GiB API | `make track1-a` | код `107fbbb` (до улучшения) |
| `track1/series-b/` | 1×1, 2×1, 8×1 по 1 CPU (4×1 — из серии A) | `make track1-b` | код `107fbbb` |
| `track1/compare/` | улучшение: «до» (`edc091d`) и «после» на 8×1 | `make track1-compare` | `edc091d` против `22f0e62` |
| `track1/control-pool/`, `track1/control-cpus/` | контроль: пул и `maxconn` на 8×1; 8×0,5 против 4×1 на одних CPU | `make track1-control` | код `b1c80ee` (HEAD `f5fea5c`–`a67bc65`) |
| `track1/plots/` | графики и сводка потолков `capacity.md` | `make plots` | из всех `summary.csv` |
| `track1/after-checks.txt` | уровни «3» и «4» на коде с улучшением | команды внутри файла | `dde113b` |
| `track1/host-others.csv`, `track1/aborted/` | нагрузка соседних стендов во время замеров, прерванная первая попытка серии A и отброшенная первая попытка контроля размещения | см. `track1/README.md` | — |
+34
View File
@@ -0,0 +1,34 @@
date: 2026-11-05T18:56:09.780488+00:00 commit: 0a1f48bd73e5780c732356880c23fdb476bcff98
environment: COMPOSE_PROJECT_NAME=mtusi2026lab3-grade3-final-20261105 LB_HTTP_PORT=18083 LB_STATS_PORT=23083 API_REPLICAS=2 NO_PROXY=localhost,127.0.0.1,::1
$ sh -c 'set -e; make check; make validate; make lint; uv lock --check; openspec validate --all --strict; git diff --exit-code upstream/main -- contracts tests README.md COURSE.md lab.json MAINTAINERS.md'
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
.venv/bin/python scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
.venv/bin/ruff check src student-tests alembic scripts/probe.py scripts/limits.py scripts/scaling.py tools
All checks passed!
.venv/bin/ruff format --check src student-tests alembic scripts/probe.py scripts/limits.py scripts/scaling.py tools
107 files already formatted
.venv/bin/ty check src student-tests alembic
All checks passed!
Resolved 64 packages in 14ms
- Validating...
✓ change/2026-11-05-adopt-sqlalchemy-alembic
✓ change/fix-lab3-postgres-pool-recovery
✓ spec/lab1-event-publication
✓ spec/lab1-messenger-chats
✓ spec/lab1-messenger-history
✓ spec/lab1-messenger-identity
✓ spec/lab1-service-operability
✓ spec/lab2-auth-sessions
✓ spec/lab2-dependency-failures
✓ spec/lab2-history-index
✓ spec/lab2-persistent-storage
✓ spec/lab2-session-cache
✓ spec/lab3-failover-operations
✓ spec/lab3-idempotent-messages
✓ spec/lab3-load-balancing
✓ spec/lab3-scaling-cost
Totals: 16 passed, 0 failed (16 items)
exit_code: 0 elapsed_seconds: 1.458
+360
View File
@@ -0,0 +1,360 @@
date: 2026-10-30T02:55:09Z commit: e049c26 dir: mtusi2026lab3-clean (свой compose-проект и volumes)
# шаги — команды блока «Запуск из чистого clone» REPORT.md дословно; строки с пометкой «наблюдение» — проверки между шагами
$ git clone <repo> mtusi2026lab3-clean && cd mtusi2026lab3-clean && git checkout solution/bpi2303-kostylev
HEAD e049c26 Reproduce a lost answer on the stand and enrich the run records
$ docker volume ls -q --filter label=com.docker.compose.project=mtusi2026lab3-clean | wc -l | tr -d ' ' # наблюдение, не шаг инструкции
0
$ make secrets
sh scripts/secrets.sh
создан secrets/postgres_password
создан secrets/database_url
создан secrets/cursor_secret
создан secrets/valkey_password
создан secrets/valkey.conf
создан secrets/cache_url
создан .env из .env.example
exit=0
$ sed LB_HTTP_PORT=18083 LB_STATS_PORT=23083 .env # отступление: 8080 на этой машине занят стендами других лаб
$ git status --short --ignored secrets .env # наблюдение, не шаг инструкции
!! .env
!! secrets/
$ docker compose --env-file .env config --quiet
exit=0
$ make up
sh scripts/secrets.sh
docker compose up -d --build --wait
#1 [internal] load local bake definitions
#1 reading from stdin 606B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.57kB done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d
#3 DONE 0.0s
#4 [internal] load metadata for ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142
#4 DONE 0.0s
#5 [internal] load .dockerignore
#5 transferring context: 437B done
#5 DONE 0.0s
#6 [builder 1/7] FROM docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d
#6 resolve docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d done
#6 DONE 0.0s
#7 FROM ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142
#7 resolve ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 done
#7 DONE 0.0s
#8 [internal] load build context
#8 transferring context: 282.44kB 0.0s done
#8 DONE 0.0s
#9 [builder 3/7] WORKDIR /build
#9 CACHED
#10 [builder 2/7] COPY --from=ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 /uv /usr/local/bin/uv
#10 CACHED
#11 [builder 4/7] COPY pyproject.toml uv.lock README.md ./
#11 CACHED
#12 [builder 5/7] RUN uv sync --locked --no-dev --no-install-project --no-editable
#12 CACHED
#13 [builder 6/7] COPY src ./src
#13 DONE 0.0s
#14 [builder 7/7] RUN uv sync --locked --no-dev --no-editable
#14 0.177 Resolved 59 packages in 0.70ms
#14 0.179 Building mtusi-messenger @ file:///build
#14 1.370 Built mtusi-messenger @ file:///build
#14 1.373 Prepared 1 package in 1.19s
#14 1.377 Installed 1 package in 3ms
#14 1.462 Bytecode compiled 889 files in 85ms
#14 1.462 + mtusi-messenger==1.3.0 (from file:///build)
#14 DONE 1.5s
#15 [runtime 2/4] RUN groupadd --system --gid 10001 messenger && useradd --system --uid 10001 --gid messenger --no-create-home --shell /usr/sbin/nologin messenger
#15 CACHED
#16 [runtime 3/4] COPY --from=builder --chown=root:root /opt/venv /opt/venv
#16 DONE 0.1s
#17 [runtime 4/4] WORKDIR /app
#17 DONE 0.0s
#18 exporting to image
#18 exporting layers
#18 exporting layers 1.4s done
#18 exporting manifest sha256:41b1d8e9a345c4df365f8ff02ecbc631fbe4ddba70f99aa9e5eb551238e974d6
#18 exporting manifest sha256:41b1d8e9a345c4df365f8ff02ecbc631fbe4ddba70f99aa9e5eb551238e974d6 done
#18 exporting config sha256:5d3385c825a95590c9cd0253aa72316de78fd11dc85ab9b61af766943ef9847d done
#18 exporting attestation manifest sha256:7729b5721c86110d489c782e5df22a34b492328eb4b92fa30f2d9dea23c810d3 done
#18 exporting manifest list sha256:0b27d26576cb62d154e45c8e6af3d80e9663cd275eb234a96a6a1ebc0c494c3a done
#18 naming to docker.io/library/mtusi-messenger:mtusi2026lab3-clean done
#18 unpacking to docker.io/library/mtusi-messenger:mtusi2026lab3-clean
#18 unpacking to docker.io/library/mtusi-messenger:mtusi2026lab3-clean 0.2s done
#18 DONE 1.7s
#19 resolving provenance for metadata file
#19 DONE 0.0s
api Built
Network mtusi2026lab3-clean_edge Creating
Network mtusi2026lab3-clean_edge Created
Network mtusi2026lab3-clean_backend Creating
Network mtusi2026lab3-clean_backend Created
Volume "mtusi2026lab3-clean_pgdata" Creating
Volume "mtusi2026lab3-clean_pgdata" Created
Volume "mtusi2026lab3-clean_valkeydata" Creating
Volume "mtusi2026lab3-clean_valkeydata" Created
Container mtusi2026lab3-clean-lb-1 Creating
Container mtusi2026lab3-clean-valkey-1 Creating
Container mtusi2026lab3-clean-postgres-1 Creating
Container mtusi2026lab3-clean-valkey-1 Created
Container mtusi2026lab3-clean-lb-1 Created
Container mtusi2026lab3-clean-postgres-1 Created
Container mtusi2026lab3-clean-migrate-1 Creating
Container mtusi2026lab3-clean-migrate-1 Created
Container mtusi2026lab3-clean-api-1 Creating
Container mtusi2026lab3-clean-api-2 Creating
Container mtusi2026lab3-clean-api-2 Created
Container mtusi2026lab3-clean-api-1 Created
Container mtusi2026lab3-clean-lb-1 Starting
Container mtusi2026lab3-clean-postgres-1 Starting
Container mtusi2026lab3-clean-valkey-1 Starting
Container mtusi2026lab3-clean-valkey-1 Started
Container mtusi2026lab3-clean-postgres-1 Started
Container mtusi2026lab3-clean-postgres-1 Waiting
Container mtusi2026lab3-clean-lb-1 Started
Container mtusi2026lab3-clean-postgres-1 Healthy
Container mtusi2026lab3-clean-migrate-1 Starting
Container mtusi2026lab3-clean-migrate-1 Started
Container mtusi2026lab3-clean-migrate-1 Waiting
Container mtusi2026lab3-clean-migrate-1 Exited
Container mtusi2026lab3-clean-api-1 Starting
Container mtusi2026lab3-clean-api-1 Started
Container mtusi2026lab3-clean-api-2 Starting
Container mtusi2026lab3-clean-api-2 Started
Container mtusi2026lab3-clean-lb-1 Waiting
Container mtusi2026lab3-clean-api-1 Waiting
Container mtusi2026lab3-clean-api-2 Waiting
Container mtusi2026lab3-clean-migrate-1 Waiting
Container mtusi2026lab3-clean-postgres-1 Waiting
Container mtusi2026lab3-clean-valkey-1 Waiting
Container mtusi2026lab3-clean-postgres-1 Healthy
Container mtusi2026lab3-clean-migrate-1 Exited
Container mtusi2026lab3-clean-valkey-1 Healthy
Container mtusi2026lab3-clean-api-1 Healthy
Container mtusi2026lab3-clean-api-2 Healthy
Container mtusi2026lab3-clean-lb-1 Healthy
scripts/lb-wait.sh
exit=0
make up (build, migrate, wait healthy, all replicas UP in LB): 10 s
$ docker compose logs --no-log-prefix migrate # наблюдение, не шаг инструкции
{"version":"0001_init","event":"migration.applied","level":"info","ts":"2026-10-30T02:55:17.402396Z"}
{"version":"0002_idempotency_keys","event":"migration.applied","level":"info","ts":"2026-10-30T02:55:17.402481Z"}
{"reason":"applied","event":"migration.completed","level":"info","ts":"2026-10-30T02:55:17.402535Z"}
$ docker compose ps -a --format '{{.Service}} {{.State}} {{.Status}} {{.Ports}}' # наблюдение, не шаг инструкции
api running Up 3 seconds (healthy)
api running Up 3 seconds (healthy)
lb running Up 6 seconds (healthy) 127.0.0.1:18083->8080/tcp, 127.0.0.1:23083->8404/tcp
migrate exited Exited (0) 4 seconds ago
postgres running Up 6 seconds (healthy)
valkey running Up 6 seconds (healthy)
$ curl -s "http://$(docker compose port lb 8404)/stats;csv" | awk -F, '$1 == "api" && $2 != "BACKEND" {print $2, $18}' # наблюдение, не шаг инструкции
api1 UP
api2 UP
api3 MAINT (resolution)
api4 MAINT (resolution)
api5 MAINT (resolution)
api6 MAINT (resolution)
api7 MAINT (resolution)
api8 MAINT (resolution)
$ make test
python3 tests/smoke.py --base-url "http://127.0.0.1:18083"
PASS: lab3 public API scenario; infrastructure and grading evidence remain separate.
exit=0
$ make lb-check
python3 scripts/probe.py check --rounds 20 --out evidence/lb-check.json
{
"instances_one_token": {
"8c9722d76b31": 20,
"58d535b73f25": 20
},
"same_key": {
"rounds": 20,
"parallel": 8,
"statuses": {
"201": 160
},
"instances": {
"8c9722d76b31": 80,
"58d535b73f25": 80
},
"identical_bodies_rounds": 20,
"single_copy_rounds": 20,
"conflict_rounds": 20
},
"pass": true
}
exit=0
$ make retry-check
python3 scripts/probe.py retry --out evidence/retry.json
{
"variants": [
{
"variant": "timeout",
"lost_attempt": {
"status": 0,
"client_timeout_s": 0.5,
"chat_lock_held_s": 1.5,
"replica": "8c9722d76b31",
"replica_killed": false,
"replica_answered": 201
},
"retry": {
"after_s": 1.546,
"status": 201,
"attempts": 1,
"replica": "58d535b73f25"
},
"copies_in_history": 1,
"same_message": true,
"written_by": "lost attempt",
"pass": true
},
{
"variant": "kill",
"lost_attempt": {
"status": 0,
"client_timeout_s": 0.5,
"chat_lock_held_s": 1.5,
"replica": "58d535b73f25",
"replica_killed": true,
"replica_answered": null
},
"retry": {
"after_s": 1.533,
"status": 201,
"attempts": 1,
"replica": "8c9722d76b31"
},
"copies_in_history": 1,
"same_message": true,
"written_by": "lost attempt",
"pass": true
}
],
"pass": true
}
exit=0
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18083"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 135, "failed_attempts": 0, "first_recovery_seconds": 2.911}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
exit=0
$ docker compose ps -a --format '{{.Service}} {{.Name}} {{.State}} {{.Status}}' api # наблюдение, не шаг инструкции
api mtusi2026lab3-clean-api-1 running Up 27 seconds (healthy)
api mtusi2026lab3-clean-api-2 exited Exited (0) 12 seconds ago
$ make restore
scripts/restore.sh
Container mtusi2026lab3-clean-api-1 Running
Container mtusi2026lab3-clean-api-2 Starting
Container mtusi2026lab3-clean-api-2 Started
Container mtusi2026lab3-clean-api-1 Waiting
Container mtusi2026lab3-clean-api-2 Waiting
Container mtusi2026lab3-clean-api-1 Healthy
Container mtusi2026lab3-clean-api-2 Healthy
exit=0
$ curl -s -o /dev/null "http://$(docker compose port lb 8080)/health/live"
exit=0
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh && make restore
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18083"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 135, "failed_attempts": 0, "first_recovery_seconds": 2.572}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
scripts/restore.sh
Container mtusi2026lab3-clean-api-2 Running
Container mtusi2026lab3-clean-api-1 Starting
Container mtusi2026lab3-clean-api-1 Started
Container mtusi2026lab3-clean-api-1 Waiting
Container mtusi2026lab3-clean-api-2 Waiting
Container mtusi2026lab3-clean-api-1 Healthy
Container mtusi2026lab3-clean-api-2 Healthy
exit=0
$ docker compose ps -a --format '{{.Service}} {{.Name}} {{.State}} {{.Status}}' api # наблюдение, не шаг инструкции
api mtusi2026lab3-clean-api-1 running Up 1 second (healthy)
api mtusi2026lab3-clean-api-2 running Up 19 seconds (healthy)
$ make test-persistence ACTION_SCRIPT=scripts/restart.sh
python3 tests/resilience.py --mode persistence --action-script "scripts/restart.sh" --base-url "http://127.0.0.1:18083"
Executing explicit persistence hook; hook output suppressed to avoid leaking secrets.
PASS: persistence preserved confirmed data and existing session. Hook scope needs operator evidence.
exit=0
$ docker compose ps -a --format '{{.Service}} {{.State}} {{.RunningFor}} {{.Status}}' # наблюдение, не шаг инструкции
api running 58 seconds ago Up 3 seconds (healthy)
api running 58 seconds ago Up 3 seconds (healthy)
lb running 58 seconds ago Up 5 seconds (healthy)
migrate exited 58 seconds ago Exited (0) 4 seconds ago
postgres running 58 seconds ago Up 5 seconds (healthy)
valkey running 58 seconds ago Up 5 seconds (healthy)
$ make migrate
docker compose run --rm migrate
Container mtusi2026lab3-clean-postgres-1 Running
{"reason":"up to date","event":"migration.completed","level":"info","ts":"2026-10-30T02:56:15.101805Z"}
exit=0
$ make storage-evidence
sh scripts/evidence-storage.sh
evidence/{schema,migrations,kdf,secrets}.txt обновлены
exit=0
$ docker volume ls --format '{{.Name}}' --filter label=com.docker.compose.project=mtusi2026lab3-clean # наблюдение, не шаг инструкции
mtusi2026lab3-clean_pgdata
mtusi2026lab3-clean_valkeydata
$ make down
docker compose --profile bench down
Container mtusi2026lab3-clean-api-1 Stopping
Container mtusi2026lab3-clean-api-2 Stopping
Container mtusi2026lab3-clean-lb-1 Stopping
Container mtusi2026lab3-clean-lb-1 Stopped
Container mtusi2026lab3-clean-lb-1 Removing
Container mtusi2026lab3-clean-lb-1 Removed
Container mtusi2026lab3-clean-api-2 Stopped
Container mtusi2026lab3-clean-api-2 Removing
Container mtusi2026lab3-clean-api-1 Stopped
Container mtusi2026lab3-clean-api-1 Removing
Container mtusi2026lab3-clean-api-2 Removed
Container mtusi2026lab3-clean-api-1 Removed
Container mtusi2026lab3-clean-valkey-1 Stopping
Container mtusi2026lab3-clean-migrate-1 Stopping
Container mtusi2026lab3-clean-migrate-1 Stopped
Container mtusi2026lab3-clean-migrate-1 Removing
Container mtusi2026lab3-clean-migrate-1 Removed
Container mtusi2026lab3-clean-postgres-1 Stopping
Container mtusi2026lab3-clean-postgres-1 Stopped
Container mtusi2026lab3-clean-postgres-1 Removing
Container mtusi2026lab3-clean-postgres-1 Removed
Container mtusi2026lab3-clean-valkey-1 Stopped
Container mtusi2026lab3-clean-valkey-1 Removing
Container mtusi2026lab3-clean-valkey-1 Removed
Network mtusi2026lab3-clean_edge Removing
Network mtusi2026lab3-clean_backend Removing
Network mtusi2026lab3-clean_edge Removed
Network mtusi2026lab3-clean_backend Removed
exit=0
$ docker volume ls --format '{{.Name}}' --filter label=com.docker.compose.project=mtusi2026lab3-clean # наблюдение, не шаг инструкции
mtusi2026lab3-clean_pgdata
mtusi2026lab3-clean_valkeydata
$ make reset-data
Удаляются volumes с учебными данными стенда
docker compose --profile bench down -v --remove-orphans
Volume mtusi2026lab3-clean_pgdata Removing
Volume mtusi2026lab3-clean_valkeydata Removing
Volume mtusi2026lab3-clean_pgdata Removed
Volume mtusi2026lab3-clean_valkeydata Removed
exit=0
$ docker volume ls -q --filter label=com.docker.compose.project=mtusi2026lab3-clean | wc -l | tr -d ' ' # наблюдение, не шаг инструкции
0
-14
View File
@@ -1,14 +0,0 @@
date: 2026-10-29T00:36:33Z commit: 0b85edf
$ make check
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
$ make validate PYTHON=<venv с requirements-dev.txt>
<venv>/bin/python scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
$ make lint
.venv/bin/ruff check src student-tests scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/make_images.py scripts/pipeline_suite.py scripts/pipeline_evidence.py
All checks passed!
.venv/bin/ruff format --check src student-tests scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/make_images.py scripts/pipeline_suite.py scripts/pipeline_evidence.py
159 files already formatted
.venv/bin/ty check src student-tests
All checks passed!
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"kill",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 0.113,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 0.934,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "scripts/restore.sh",
"restore_seconds": 2.57,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"pause",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 0.057,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 3.215,
"declared_max_seconds": 3.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "docker unpause 7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170 && scripts/lb-wait.sh",
"restore_seconds": 2.425,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"stop",
"f225c6cd77bb40f6e598bd65a1b50fa8244700978ee4086700860e0fb076857a"
],
"action_exit": 0,
"action_seconds": 0.167,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 0,
"detection_seconds": 1.196,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 503,
"x_instance_id_present": false,
"error_code": "unavailable"
},
"restore": "docker start f225c6cd77bb40f6e598bd65a1b50fa8244700978ee4086700860e0fb076857a && scripts/lb-wait.sh",
"restore_seconds": 2.475,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+35
View File
@@ -0,0 +1,35 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"stop",
"-t",
"7",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 2.535,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 1.069,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "scripts/restore.sh",
"restore_seconds": 1.969,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"excluded_before_action_end": true,
"restored": true
},
"pass": true
}
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"kill",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 0.162,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 0.901,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "scripts/restore.sh",
"restore_seconds": 3.086,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"pause",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 0.063,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 2.817,
"declared_max_seconds": 3.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "docker unpause 7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170 && scripts/lb-wait.sh",
"restore_seconds": 2.378,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"stop",
"f225c6cd77bb40f6e598bd65a1b50fa8244700978ee4086700860e0fb076857a"
],
"action_exit": 0,
"action_seconds": 0.17,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 0,
"detection_seconds": 1.263,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 503,
"x_instance_id_present": false,
"error_code": "unavailable"
},
"restore": "docker start f225c6cd77bb40f6e598bd65a1b50fa8244700978ee4086700860e0fb076857a && scripts/lb-wait.sh",
"restore_seconds": 2.455,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+35
View File
@@ -0,0 +1,35 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"stop",
"-t",
"7",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 2.662,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 1.205,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "scripts/restore.sh",
"restore_seconds": 1.953,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"excluded_before_action_end": true,
"restored": true
},
"pass": true
}
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"kill",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 0.163,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 0.886,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "scripts/restore.sh",
"restore_seconds": 3.095,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"pause",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 0.063,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 2.778,
"declared_max_seconds": 3.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "docker unpause 7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170 && scripts/lb-wait.sh",
"restore_seconds": 2.416,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+32
View File
@@ -0,0 +1,32 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"stop",
"f225c6cd77bb40f6e598bd65a1b50fa8244700978ee4086700860e0fb076857a"
],
"action_exit": 0,
"action_seconds": 0.163,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 0,
"detection_seconds": 1.239,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 503,
"x_instance_id_present": false,
"error_code": "unavailable"
},
"restore": "docker start f225c6cd77bb40f6e598bd65a1b50fa8244700978ee4086700860e0fb076857a && scripts/lb-wait.sh",
"restore_seconds": 2.481,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"restored": true
},
"pass": true
}
+35
View File
@@ -0,0 +1,35 @@
{"date": "2026-11-05T19:17:01.673107+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"action": [
"docker",
"stop",
"-t",
"7",
"7cfd91ddf780f4c432d3f69358c626eed4986c5841c85eab201e9330976e1170"
],
"action_exit": 0,
"action_seconds": 2.665,
"lb_up_before": 2,
"lb_samples_missing": 0,
"lb_up_counted_as_detected": 1,
"detection_seconds": 1.266,
"declared_max_seconds": 1.5,
"measurement_slack_seconds": 0.25,
"lb_answer_after_detection": {
"path": "/health/live",
"status": 200,
"x_instance_id_present": true,
"error_code": null
},
"restore": "scripts/restore.sh",
"restore_seconds": 1.984,
"lb_up_after_restore": 2,
"checks": {
"action_ok": true,
"lb_observed": true,
"detected_in_time": true,
"lb_answer_ok": true,
"excluded_before_action_end": true,
"restored": true
},
"pass": true
}
+4 -27
View File
@@ -1,29 +1,25 @@
date: 2026-10-29T00:36:34Z commit: 0b85edf
date: 2026-11-05T18:59:00Z commit: 0a1f48bd73e5780c732356880c23fdb476bcff98
host: Apple M4 Max, 16 CPU, 48 GiB RAM, macOS 15.1
docker: 28.3.2, VM: 16 CPU, 8217190400 bytes
compose: 2.38.2-desktop.1
images:
image: ${API_IMAGE:-mtusi-messenger:lab5}
image: ${API_IMAGE:-mtusi-messenger:lab5}
image: rustfs/rc:v0.1.36@sha256:ab024bfebee49a750ce886b4c70963ccd9ddaa03f491704a90710641d7a26699
image: ${API_IMAGE:-mtusi-messenger:${COMPOSE_PROJECT_NAME}}
image: haproxy:3.2.26-alpine@sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b
image: postgres:17.11-alpine@sha256:266d3c257594e195b309ae9d1805e54c788f1d7b375f56bf6c367aac6c917e7b
image: valkey/valkey:8.1.11-alpine@sha256:2602ace2be6569bc2bbb06f873776516ab9089821525be34f4305c67eb9db02a
image: rustfs/rustfs:1.0.4@sha256:ed9210d6411d77ffceb70d94b964a3cb7d225983368bc25bebe01e995293dcc2
image: rabbitmq:4.1.8-management-alpine@sha256:54ef5ff505352c1beef05a97ee9bd6ab95b3559a76629ba0119b1d63d8d6e4b2
image: grafana/k6:2.3.0@sha256:2dd4756ec6e14e62778916e1eb850108e4b499304c02df56760fa0bdb0fba0b8
FROM python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d AS builder
COPY --from=ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 /uv /usr/local/bin/uv
FROM python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d AS runtime
python in image: Python 3.14.7
haproxy: HAProxy version 3.2.26-3e0f11a 2026/10/14 - https://haproxy.org/
rustfs: rustfs 1.0.4
rabbitmq: 4.1.8
packages (uv.lock):
fastapi 0.141.1
uvicorn 0.54.0
uvicorn-worker 0.4.0
gunicorn 26.2.0
sqlalchemy 2.0.45
alembic 1.16.5
asyncpg 0.32.0
argon2-cffi 25.1.0
pydantic 2.13.5
@@ -31,28 +27,9 @@ packages (uv.lock):
dishka 1.10.1
structlog 26.1.0
redis 8.1.0
aiobotocore 3.10.0
botocore 1.43.117
aiohttp 3.14.5
pillow 13.0.0
faststream 0.7.8
aio-pika 10.2.1
aiormq 7.2.4
dishka-faststream 0.7.0
process users (service uid command):
api 10001 python
broker 100 beam.smp
broker 100 epmd
broker 100 erl_child_setup
broker 100 inet_gethost
broker 100 sh
lb 99 haproxy
postgres 70 postgres
relay 10001 docker-init
relay 10001 python
s3 10001 rustfs
valkey 999 tini
valkey 999 valkey-server
worker 10001 docker-init
worker 10001 python
worker limits: 2 CPU, memory.max 1024 MiB; render: WORKER_CONCURRENCY=2 processes, RLIMIT_AS 448 MiB, timeout 30 s, 100 tasks per process
+23
View File
@@ -0,0 +1,23 @@
date: 2026-11-05T18:58:02.224418+00:00 commit: 0a1f48bd73e5780c732356880c23fdb476bcff98
environment: COMPOSE_PROJECT_NAME=mtusi2026lab3-grade3-final-20261105 LB_HTTP_PORT=18083 LB_STATS_PORT=23083 API_REPLICAS=2 NO_PROXY=localhost,127.0.0.1,::1
$ sh -c 'docker compose ps -a; make test-failover ACTION_SCRIPT=scripts/stop-one.sh; result=$?; docker compose ps -a; exit "$result"'
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi2026lab3-grade3-final-20261105-api-1 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "python -m gunicorn …" api 2 minutes ago Up 3 seconds (healthy)
mtusi2026lab3-grade3-final-20261105-api-2 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "python -m gunicorn …" api 2 minutes ago Up About a minute (healthy)
mtusi2026lab3-grade3-final-20261105-lb-1 haproxy:3.2.26-alpine@sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b "docker-entrypoint.s…" lb 2 minutes ago Up About a minute (healthy) 127.0.0.1:18083->8080/tcp, 127.0.0.1:23083->8404/tcp
mtusi2026lab3-grade3-final-20261105-migrate-1 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "alembic upgrade head" migrate 2 minutes ago Exited (0) About a minute ago
mtusi2026lab3-grade3-final-20261105-postgres-1 postgres:17.11-alpine@sha256:266d3c257594e195b309ae9d1805e54c788f1d7b375f56bf6c367aac6c917e7b "docker-entrypoint.s…" postgres 2 minutes ago Up About a minute (healthy)
mtusi2026lab3-grade3-final-20261105-valkey-1 valkey/valkey:8.1.11-alpine@sha256:2602ace2be6569bc2bbb06f873776516ab9089821525be34f4305c67eb9db02a "tini -- docker-entr…" valkey 2 minutes ago Up About a minute (healthy)
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18083"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 129, "failed_attempts": 0, "first_recovery_seconds": 2.762}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi2026lab3-grade3-final-20261105-api-1 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "python -m gunicorn …" api 2 minutes ago Exited (0) 12 seconds ago
mtusi2026lab3-grade3-final-20261105-api-2 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "python -m gunicorn …" api 2 minutes ago Up 2 minutes (healthy)
mtusi2026lab3-grade3-final-20261105-lb-1 haproxy:3.2.26-alpine@sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b "docker-entrypoint.s…" lb 2 minutes ago Up 2 minutes (healthy) 127.0.0.1:18083->8080/tcp, 127.0.0.1:23083->8404/tcp
mtusi2026lab3-grade3-final-20261105-migrate-1 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "alembic upgrade head" migrate 2 minutes ago Exited (0) 2 minutes ago
mtusi2026lab3-grade3-final-20261105-postgres-1 postgres:17.11-alpine@sha256:266d3c257594e195b309ae9d1805e54c788f1d7b375f56bf6c367aac6c917e7b "docker-entrypoint.s…" postgres 2 minutes ago Up 2 minutes (healthy)
mtusi2026lab3-grade3-final-20261105-valkey-1 valkey/valkey:8.1.11-alpine@sha256:2602ace2be6569bc2bbb06f873776516ab9089821525be34f4305c67eb9db02a "tini -- docker-entr…" valkey 2 minutes ago Up 2 minutes (healthy)
exit_code: 0 elapsed_seconds: 15.820
+23
View File
@@ -0,0 +1,23 @@
date: 2026-11-05T18:58:20.145733+00:00 commit: 0a1f48bd73e5780c732356880c23fdb476bcff98
environment: COMPOSE_PROJECT_NAME=mtusi2026lab3-grade3-final-20261105 LB_HTTP_PORT=18083 LB_STATS_PORT=23083 API_REPLICAS=2 NO_PROXY=localhost,127.0.0.1,::1
$ sh -c 'curl -fsS -o /dev/null "http://$(docker compose port lb 8080)/health/live"; docker compose ps -a; make test-failover ACTION_SCRIPT=scripts/stop-one.sh; result=$?; docker compose ps -a; exit "$result"'
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi2026lab3-grade3-final-20261105-api-1 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "python -m gunicorn …" api 2 minutes ago Up 2 seconds (healthy)
mtusi2026lab3-grade3-final-20261105-api-2 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "python -m gunicorn …" api 2 minutes ago Up 2 minutes (healthy)
mtusi2026lab3-grade3-final-20261105-lb-1 haproxy:3.2.26-alpine@sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b "docker-entrypoint.s…" lb 2 minutes ago Up 2 minutes (healthy) 127.0.0.1:18083->8080/tcp, 127.0.0.1:23083->8404/tcp
mtusi2026lab3-grade3-final-20261105-migrate-1 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "alembic upgrade head" migrate 2 minutes ago Exited (0) 2 minutes ago
mtusi2026lab3-grade3-final-20261105-postgres-1 postgres:17.11-alpine@sha256:266d3c257594e195b309ae9d1805e54c788f1d7b375f56bf6c367aac6c917e7b "docker-entrypoint.s…" postgres 2 minutes ago Up 2 minutes (healthy)
mtusi2026lab3-grade3-final-20261105-valkey-1 valkey/valkey:8.1.11-alpine@sha256:2602ace2be6569bc2bbb06f873776516ab9089821525be34f4305c67eb9db02a "tini -- docker-entr…" valkey 2 minutes ago Up 2 minutes (healthy)
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18083"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 128, "failed_attempts": 0, "first_recovery_seconds": 2.817}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
mtusi2026lab3-grade3-final-20261105-api-1 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "python -m gunicorn …" api 3 minutes ago Up 17 seconds (healthy)
mtusi2026lab3-grade3-final-20261105-api-2 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "python -m gunicorn …" api 3 minutes ago Exited (0) 12 seconds ago
mtusi2026lab3-grade3-final-20261105-lb-1 haproxy:3.2.26-alpine@sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b "docker-entrypoint.s…" lb 3 minutes ago Up 2 minutes (healthy) 127.0.0.1:18083->8080/tcp, 127.0.0.1:23083->8404/tcp
mtusi2026lab3-grade3-final-20261105-migrate-1 mtusi-messenger:mtusi2026lab3-grade3-final-20261105 "alembic upgrade head" migrate 3 minutes ago Exited (0) 2 minutes ago
mtusi2026lab3-grade3-final-20261105-postgres-1 postgres:17.11-alpine@sha256:266d3c257594e195b309ae9d1805e54c788f1d7b375f56bf6c367aac6c917e7b "docker-entrypoint.s…" postgres 3 minutes ago Up 2 minutes (healthy)
mtusi2026lab3-grade3-final-20261105-valkey-1 valkey/valkey:8.1.11-alpine@sha256:2602ace2be6569bc2bbb06f873776516ab9089821525be34f4305c67eb9db02a "tini -- docker-entr…" valkey 3 minutes ago Up 2 minutes (healthy)
exit_code: 0 elapsed_seconds: 15.802
@@ -0,0 +1,15 @@
date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
7cfd91ddf780 api Up About a minute (healthy)
5bbf259ff136 api Exited (0) Less than a second ago
19:19:56 database bytes,messages: 12678835,8785
mtusi2026lab3-grade4-20261105-api-1 cpu=25.25% mem=209.7MiB / 512MiB
mtusi2026lab3-grade4-20261105-lb-1 cpu=2.71% mem=30.9MiB / 128MiB
mtusi2026lab3-grade4-20261105-postgres-1 cpu=6.45% mem=51.05MiB / 1GiB
mtusi2026lab3-grade4-20261105-valkey-1 cpu=0.43% mem=12.96MiB / 128MiB
Container mtusi2026lab3-grade4-20261105-api-1 Running
Container mtusi2026lab3-grade4-20261105-api-2 Starting
Container mtusi2026lab3-grade4-20261105-api-2 Started
Container mtusi2026lab3-grade4-20261105-api-2 Waiting
Container mtusi2026lab3-grade4-20261105-api-1 Waiting
Container mtusi2026lab3-grade4-20261105-api-2 Healthy
Container mtusi2026lab3-grade4-20261105-api-1 Healthy
@@ -0,0 +1,3 @@
date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
time="2026-11-05T19:19:28Z" level=info msg=LOAD-START source=console
measure: 6001 requests, failed ratio 0
@@ -0,0 +1,309 @@
{"date": "2026-11-05T19:18:13.470607+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"options": {
"summaryTrendStats": [
"avg",
"min",
"med",
"p(90)",
"p(95)",
"p(99)",
"max"
],
"summaryTimeUnit": "",
"noColor": false
},
"state": {
"isStdOutTTY": false,
"isStdErrTTY": false,
"testRunDurationMs": 70512.503422
},
"metrics": {
"dropped_iterations": {
"type": "counter",
"contains": "default",
"values": {
"count": 0,
"rate": 0
}
},
"data_sent": {
"values": {
"count": 2054247,
"rate": 29133.08846384076
},
"type": "counter",
"contains": "data"
},
"http_req_tls_handshaking": {
"contains": "time",
"values": {
"med": 0,
"p(90)": 0,
"p(95)": 0,
"p(99)": 0,
"max": 0,
"avg": 0,
"min": 0
},
"type": "trend"
},
"http_req_duration": {
"type": "trend",
"contains": "time",
"values": {
"p(99)": 5.295193749999997,
"max": 53.791625,
"avg": 3.101954274815648,
"min": 0.861194,
"med": 2.9916875000000003,
"p(90)": 3.6507209,
"p(95)": 3.9084498499999993
}
},
"checks": {
"contains": "default",
"values": {
"rate": 1,
"passes": 7002,
"fails": 0
},
"type": "rate"
},
"http_req_duration{scenario:measure}": {
"type": "trend",
"contains": "time",
"values": {
"min": 0.861194,
"med": 2.979916,
"p(90)": 3.638291,
"p(95)": 3.8775,
"p(99)": 4.531834,
"max": 53.791625,
"avg": 3.0515494415930533
}
},
"http_req_sending": {
"type": "trend",
"contains": "time",
"values": {
"med": 0.011667,
"p(90)": 0.0155,
"p(95)": 0.018935449999999986,
"p(99)": 0.033915669999999995,
"max": 0.254083,
"avg": 0.01292155005672162,
"min": 0.002584
}
},
"http_req_failed": {
"type": "rate",
"contains": "default",
"values": {
"rate": 0,
"passes": 0,
"fails": 7052
}
},
"http_reqs{scenario:measure}": {
"type": "counter",
"contains": "default",
"values": {
"count": 6001,
"rate": 85.10547362196873
}
},
"http_req_duration{expected_response:true}": {
"type": "trend",
"contains": "time",
"values": {
"avg": 3.101954274815648,
"min": 0.861194,
"med": 2.9916875000000003,
"p(90)": 3.6507209,
"p(95)": 3.9084498499999993,
"p(99)": 5.295193749999997,
"max": 53.791625
}
},
"http_req_receiving": {
"contains": "time",
"values": {
"p(95)": 0.04025,
"p(99)": 0.06647825,
"max": 0.282375,
"avg": 0.023369897192285907,
"min": 0.008,
"med": 0.020459,
"p(90)": 0.033583
},
"type": "trend"
},
"iteration_duration": {
"type": "trend",
"contains": "time",
"values": {
"p(90)": 3.7281128,
"p(95)": 3.97822085,
"p(99)": 4.60467816,
"max": 53.948708,
"avg": 3.1402372599257378,
"min": 1.758833,
"med": 3.0767085
}
},
"vus": {
"type": "gauge",
"contains": "default",
"values": {
"value": 1,
"min": 0,
"max": 1
}
},
"iterations": {
"type": "counter",
"contains": "default",
"values": {
"count": 7002,
"rate": 99.30153746059408
}
},
"dropped_iterations{scenario:measure}": {
"type": "counter",
"contains": "default",
"values": {
"count": 0,
"rate": 0
}
},
"data_received": {
"type": "counter",
"contains": "data",
"values": {
"count": 17075371,
"rate": 242160.89588832358
}
},
"http_req_duration{scenario:measure,name:post_message}": {
"values": {
"max": 53.791625,
"avg": 2.996232064174453,
"min": 0.861194,
"med": 2.948,
"p(90)": 3.5552918,
"p(95)": 3.7865877,
"p(99)": 4.3190254999999995
},
"type": "trend",
"contains": "time"
},
"http_reqs": {
"values": {
"count": 7052,
"rate": 100.0106315584275
},
"type": "counter",
"contains": "default"
},
"vus_max": {
"contains": "default",
"values": {
"value": 20,
"min": 20,
"max": 20
},
"type": "gauge"
},
"http_req_waiting": {
"type": "trend",
"contains": "time",
"values": {
"p(95)": 3.8635248499999997,
"p(99)": 5.2155688399999915,
"max": 53.760833,
"avg": 3.065662827566643,
"min": 0.829445,
"med": 2.9556665,
"p(90)": 3.6086503
}
},
"http_req_blocked": {
"type": "trend",
"contains": "time",
"values": {
"min": 0.000625,
"med": 0.001667,
"p(90)": 0.002541,
"p(95)": 0.003041,
"p(99)": 0.006434919999999987,
"max": 0.540333,
"avg": 0.002490143788996061
}
},
"http_req_connecting": {
"contains": "time",
"values": {
"p(99)": 0,
"max": 0.235959,
"avg": 0.00042762110039705047,
"min": 0,
"med": 0,
"p(90)": 0,
"p(95)": 0
},
"type": "trend"
},
"http_req_duration{scenario:measure,name:get_history}": {
"type": "trend",
"contains": "time",
"values": {
"p(95)": 4.139021,
"p(99)": 5.169410699999997,
"max": 51.836375,
"avg": 3.2761305311973,
"min": 1.79625,
"med": 3.1579794999999997,
"p(90)": 3.8563755
}
},
"http_req_failed{scenario:measure}": {
"type": "rate",
"contains": "default",
"values": {
"rate": 0,
"passes": 0,
"fails": 6001
}
},
"iterations{scenario:measure}": {
"type": "counter",
"contains": "default",
"values": {
"count": 6001,
"rate": 85.10547362196873
}
}
},
"root_group": {
"groups": [],
"checks": [
{
"path": "::message 201",
"id": "6445aff337e1ade0bc10402fbac79882",
"passes": 5622,
"fails": 0,
"name": "message 201"
},
{
"fails": 0,
"name": "history 200",
"path": "::history 200",
"id": "fe27b3b8edd292c7f2170a6357b087d0",
"passes": 1380
}
],
"name": "",
"path": "",
"id": "d41d8cd98f00b204e9800998ecf8427e"
}
}
@@ -0,0 +1,657 @@
# date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
t_rel_s,up
-25.748,2
-25.645,2
-25.541,2
-25.439,2
-25.328,2
-25.218,2
-25.105,2
-25.002,2
-24.89,2
-24.778,2
-24.667,2
-24.556,2
-24.453,2
-24.341,2
-24.236,2
-24.132,2
-24.025,2
-23.912,2
-23.801,2
-23.691,2
-23.584,2
-23.472,2
-23.36,2
-23.249,2
-23.147,2
-23.035,2
-22.922,2
-22.819,2
-22.712,2
-22.6,2
-22.488,2
-22.381,2
-22.27,2
-22.164,2
-22.056,2
-21.947,2
-21.834,2
-21.724,2
-21.621,2
-21.516,2
-21.404,2
-21.3,2
-21.188,2
-21.083,2
-20.976,2
-20.868,2
-20.756,2
-20.652,2
-20.549,2
-20.442,2
-20.333,2
-20.222,2
-20.119,2
-20.005,2
-19.892,2
-19.784,2
-19.675,2
-19.561,2
-19.449,2
-19.337,2
-19.224,2
-19.121,2
-19.015,2
-18.908,2
-18.798,2
-18.689,2
-18.583,2
-18.476,2
-18.362,2
-18.254,2
-18.141,2
-18.032,2
-17.918,2
-17.806,2
-17.701,2
-17.596,2
-17.492,2
-17.388,2
-17.282,2
-17.179,2
-17.073,2
-16.961,2
-16.853,2
-16.742,2
-16.631,2
-16.517,2
-16.413,2
-16.3,2
-16.191,2
-16.085,2
-15.973,2
-15.86,2
-15.752,2
-15.639,2
-15.533,2
-15.425,2
-15.316,2
-15.21,2
-15.107,2
-15.0,2
-14.893,2
-14.783,2
-14.671,2
-14.558,2
-14.448,2
-14.336,2
-14.224,2
-14.114,2
-14.008,2
-13.901,2
-13.789,2
-13.677,2
-13.564,2
-13.458,2
-13.345,2
-13.242,2
-13.129,2
-13.018,2
-12.909,2
-12.8,2
-12.688,2
-12.575,2
-12.473,2
-12.361,2
-12.252,2
-12.139,2
-12.027,2
-11.917,2
-11.808,2
-11.701,2
-11.592,2
-11.482,2
-11.374,2
-11.268,2
-11.16,2
-11.054,2
-10.943,2
-10.838,2
-10.731,2
-10.627,2
-10.514,2
-10.408,2
-10.298,2
-10.184,2
-10.075,2
-9.969,2
-9.861,2
-9.756,2
-9.644,2
-9.532,2
-9.425,2
-9.314,2
-9.201,2
-9.088,2
-8.984,2
-8.871,2
-8.759,2
-8.654,2
-8.541,2
-8.428,2
-8.317,2
-8.208,2
-8.095,2
-7.982,2
-7.875,2
-7.768,2
-7.659,2
-7.547,2
-7.436,2
-7.333,2
-7.222,2
-7.112,2
-7.007,2
-6.895,2
-6.782,2
-6.67,2
-6.559,2
-6.452,2
-6.345,2
-6.234,2
-6.125,2
-6.017,2
-5.909,2
-5.799,2
-5.687,2
-5.575,2
-5.465,2
-5.354,2
-5.248,2
-5.142,2
-5.04,2
-4.934,2
-4.829,2
-4.717,2
-4.609,2
-4.5,2
-4.392,2
-4.284,2
-4.18,2
-4.068,2
-3.96,2
-3.853,2
-3.743,2
-3.63,2
-3.519,2
-3.413,2
-3.301,2
-3.198,2
-3.087,2
-2.975,2
-2.864,2
-2.758,2
-2.649,2
-2.544,2
-2.434,2
-2.326,2
-2.214,2
-2.1,2
-1.986,2
-1.875,2
-1.771,2
-1.659,2
-1.55,2
-1.437,2
-1.326,2
-1.217,2
-1.108,2
-0.999,2
-0.892,2
-0.779,2
-0.666,2
-0.555,2
-0.445,2
-0.335,2
-0.227,2
-0.117,2
-0.011,2
0.1,2
0.206,2
0.309,2
0.416,2
0.52,2
0.632,2
0.743,2
0.85,2
0.958,2
1.069,2
1.176,2
1.288,2
1.391,2
1.498,1
1.608,1
1.718,1
1.826,1
1.933,1
2.045,1
2.151,1
2.258,1
2.368,1
2.477,1
2.591,1
2.697,1
2.8,1
2.902,1
3.009,1
3.112,1
3.215,1
3.326,1
3.433,1
3.542,1
3.652,1
3.762,1
3.873,1
3.982,1
4.09,1
4.202,1
4.315,1
4.421,1
4.533,1
4.641,1
4.746,1
4.858,1
4.966,1
5.077,1
5.182,1
5.291,1
5.405,1
5.519,1
5.628,1
5.733,1
5.839,1
5.952,1
6.057,1
6.166,1
6.271,1
6.376,1
6.479,1
6.583,1
6.691,1
6.8,1
6.912,1
7.024,1
7.136,1
7.241,1
7.351,1
7.457,1
7.566,1
7.677,1
7.79,1
7.896,1
8.008,1
8.12,1
8.224,1
8.338,1
8.451,1
8.564,1
8.677,1
8.79,1
8.894,1
8.999,1
9.109,1
9.213,1
9.323,1
9.428,1
9.541,1
9.654,1
9.766,1
9.879,1
9.982,1
10.094,1
10.202,1
10.315,1
10.427,1
10.538,1
10.64,1
10.753,1
10.858,1
10.964,1
11.074,1
11.178,1
11.287,1
11.399,1
11.507,1
11.617,1
11.723,1
11.83,1
11.942,1
12.054,1
12.166,1
12.273,1
12.382,1
12.49,1
12.598,1
12.71,1
12.818,1
12.93,1
13.038,1
13.145,1
13.257,1
13.366,1
13.477,1
13.588,1
13.693,1
13.797,1
13.908,1
14.02,1
14.127,1
14.237,1
14.34,1
14.452,1
14.562,1
14.666,1
14.769,1
14.882,1
14.99,2
15.101,2
15.212,2
15.314,2
15.426,2
15.534,2
15.638,2
15.743,2
15.849,2
15.961,2
16.072,2
16.184,2
16.291,2
16.396,2
16.507,2
16.619,2
16.731,2
16.837,2
16.951,2
17.062,2
17.165,2
17.274,2
17.385,2
17.495,2
17.604,2
17.716,2
17.828,2
17.94,2
18.052,2
18.161,2
18.268,2
18.372,2
18.482,2
18.593,2
18.699,2
18.81,2
18.923,2
19.03,2
19.14,2
19.25,2
19.362,2
19.473,2
19.585,2
19.695,2
19.798,2
19.903,2
20.014,2
20.127,2
20.238,2
20.351,2
20.457,2
20.562,2
20.668,2
20.778,2
20.888,2
20.997,2
21.106,2
21.218,2
21.322,2
21.436,2
21.541,2
21.654,2
21.768,2
21.882,2
21.985,2
22.097,2
22.21,2
22.322,2
22.433,2
22.54,2
22.653,2
22.76,2
22.867,2
22.978,2
23.09,2
23.199,2
23.309,2
23.414,2
23.521,2
23.623,2
23.734,2
23.845,2
23.949,2
24.058,2
24.165,2
24.268,2
24.37,2
24.48,2
24.582,2
24.691,2
24.802,2
24.908,2
25.022,2
25.136,2
25.25,2
25.354,2
25.466,2
25.578,2
25.691,2
25.803,2
25.907,2
26.015,2
26.125,2
26.232,2
26.335,2
26.443,2
26.555,2
26.668,2
26.773,2
26.878,2
26.99,2
27.103,2
27.217,2
27.321,2
27.433,2
27.54,2
27.651,2
27.764,2
27.876,2
27.988,2
28.099,2
28.211,2
28.322,2
28.435,2
28.537,2
28.649,2
28.76,2
28.863,2
28.972,2
29.084,2
29.196,2
29.308,2
29.417,2
29.528,2
29.637,2
29.742,2
29.848,2
29.956,2
30.069,2
30.177,2
30.29,2
30.392,2
30.498,2
30.607,2
30.714,2
30.823,2
30.926,2
31.035,2
31.139,2
31.242,2
31.351,2
31.461,2
31.567,2
31.678,2
31.79,2
31.898,2
32.012,2
32.123,2
32.231,2
32.344,2
32.457,2
32.569,2
32.681,2
32.788,2
32.898,2
33.01,2
33.122,2
33.231,2
33.346,2
33.457,2
33.565,2
33.673,2
33.783,2
33.896,2
34.006,2
34.12,2
34.223,2
34.33,2
34.443,2
34.555,2
34.665,2
34.773,2
34.881,2
34.986,2
35.096,2
35.209,2
35.313,2
35.422,2
35.534,2
35.638,2
35.75,2
35.856,2
35.965,2
36.068,2
36.181,2
36.29,2
36.398,2
36.507,2
36.615,2
36.728,2
36.84,2
36.953,2
37.065,2
37.177,2
37.289,2
37.4,2
37.504,2
37.614,2
37.722,2
37.834,2
37.94,2
38.047,2
38.154,2
38.267,2
38.371,2
38.483,2
38.593,2
38.705,2
38.814,2
38.918,2
39.023,2
39.135,2
39.237,2
39.341,2
39.446,2
39.553,2
39.656,2
39.765,2
39.873,2
39.979,2
40.081,2
40.192,2
40.299,2
40.403,2
40.507,2
40.614,2
40.726,2
40.83,2
40.941,2
41.047,2
41.158,2
41.261,2
41.37,2
41.481,2
41.589,2
41.697,2
41.808,2
41.916,2
42.018,2
42.123,2
42.236,2
42.339,2
42.448,2
42.559,2
42.664,2
42.777,2
42.889,2
42.998,2
43.105,2
43.217,2
43.328,2
43.441,2
43.554,2
43.667,2
43.774,2
43.88,2
43.99,2
44.102,2
44.206,2
44.315,2
44.426,2
44.53,2
44.636,2
44.741,2
44.846,2
44.958,2
45.068,2
45.181,2
45.285,2
45.394,2
45.497,2
1 # date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
2 t_rel_s,up
3 -25.748,2
4 -25.645,2
5 -25.541,2
6 -25.439,2
7 -25.328,2
8 -25.218,2
9 -25.105,2
10 -25.002,2
11 -24.89,2
12 -24.778,2
13 -24.667,2
14 -24.556,2
15 -24.453,2
16 -24.341,2
17 -24.236,2
18 -24.132,2
19 -24.025,2
20 -23.912,2
21 -23.801,2
22 -23.691,2
23 -23.584,2
24 -23.472,2
25 -23.36,2
26 -23.249,2
27 -23.147,2
28 -23.035,2
29 -22.922,2
30 -22.819,2
31 -22.712,2
32 -22.6,2
33 -22.488,2
34 -22.381,2
35 -22.27,2
36 -22.164,2
37 -22.056,2
38 -21.947,2
39 -21.834,2
40 -21.724,2
41 -21.621,2
42 -21.516,2
43 -21.404,2
44 -21.3,2
45 -21.188,2
46 -21.083,2
47 -20.976,2
48 -20.868,2
49 -20.756,2
50 -20.652,2
51 -20.549,2
52 -20.442,2
53 -20.333,2
54 -20.222,2
55 -20.119,2
56 -20.005,2
57 -19.892,2
58 -19.784,2
59 -19.675,2
60 -19.561,2
61 -19.449,2
62 -19.337,2
63 -19.224,2
64 -19.121,2
65 -19.015,2
66 -18.908,2
67 -18.798,2
68 -18.689,2
69 -18.583,2
70 -18.476,2
71 -18.362,2
72 -18.254,2
73 -18.141,2
74 -18.032,2
75 -17.918,2
76 -17.806,2
77 -17.701,2
78 -17.596,2
79 -17.492,2
80 -17.388,2
81 -17.282,2
82 -17.179,2
83 -17.073,2
84 -16.961,2
85 -16.853,2
86 -16.742,2
87 -16.631,2
88 -16.517,2
89 -16.413,2
90 -16.3,2
91 -16.191,2
92 -16.085,2
93 -15.973,2
94 -15.86,2
95 -15.752,2
96 -15.639,2
97 -15.533,2
98 -15.425,2
99 -15.316,2
100 -15.21,2
101 -15.107,2
102 -15.0,2
103 -14.893,2
104 -14.783,2
105 -14.671,2
106 -14.558,2
107 -14.448,2
108 -14.336,2
109 -14.224,2
110 -14.114,2
111 -14.008,2
112 -13.901,2
113 -13.789,2
114 -13.677,2
115 -13.564,2
116 -13.458,2
117 -13.345,2
118 -13.242,2
119 -13.129,2
120 -13.018,2
121 -12.909,2
122 -12.8,2
123 -12.688,2
124 -12.575,2
125 -12.473,2
126 -12.361,2
127 -12.252,2
128 -12.139,2
129 -12.027,2
130 -11.917,2
131 -11.808,2
132 -11.701,2
133 -11.592,2
134 -11.482,2
135 -11.374,2
136 -11.268,2
137 -11.16,2
138 -11.054,2
139 -10.943,2
140 -10.838,2
141 -10.731,2
142 -10.627,2
143 -10.514,2
144 -10.408,2
145 -10.298,2
146 -10.184,2
147 -10.075,2
148 -9.969,2
149 -9.861,2
150 -9.756,2
151 -9.644,2
152 -9.532,2
153 -9.425,2
154 -9.314,2
155 -9.201,2
156 -9.088,2
157 -8.984,2
158 -8.871,2
159 -8.759,2
160 -8.654,2
161 -8.541,2
162 -8.428,2
163 -8.317,2
164 -8.208,2
165 -8.095,2
166 -7.982,2
167 -7.875,2
168 -7.768,2
169 -7.659,2
170 -7.547,2
171 -7.436,2
172 -7.333,2
173 -7.222,2
174 -7.112,2
175 -7.007,2
176 -6.895,2
177 -6.782,2
178 -6.67,2
179 -6.559,2
180 -6.452,2
181 -6.345,2
182 -6.234,2
183 -6.125,2
184 -6.017,2
185 -5.909,2
186 -5.799,2
187 -5.687,2
188 -5.575,2
189 -5.465,2
190 -5.354,2
191 -5.248,2
192 -5.142,2
193 -5.04,2
194 -4.934,2
195 -4.829,2
196 -4.717,2
197 -4.609,2
198 -4.5,2
199 -4.392,2
200 -4.284,2
201 -4.18,2
202 -4.068,2
203 -3.96,2
204 -3.853,2
205 -3.743,2
206 -3.63,2
207 -3.519,2
208 -3.413,2
209 -3.301,2
210 -3.198,2
211 -3.087,2
212 -2.975,2
213 -2.864,2
214 -2.758,2
215 -2.649,2
216 -2.544,2
217 -2.434,2
218 -2.326,2
219 -2.214,2
220 -2.1,2
221 -1.986,2
222 -1.875,2
223 -1.771,2
224 -1.659,2
225 -1.55,2
226 -1.437,2
227 -1.326,2
228 -1.217,2
229 -1.108,2
230 -0.999,2
231 -0.892,2
232 -0.779,2
233 -0.666,2
234 -0.555,2
235 -0.445,2
236 -0.335,2
237 -0.227,2
238 -0.117,2
239 -0.011,2
240 0.1,2
241 0.206,2
242 0.309,2
243 0.416,2
244 0.52,2
245 0.632,2
246 0.743,2
247 0.85,2
248 0.958,2
249 1.069,2
250 1.176,2
251 1.288,2
252 1.391,2
253 1.498,1
254 1.608,1
255 1.718,1
256 1.826,1
257 1.933,1
258 2.045,1
259 2.151,1
260 2.258,1
261 2.368,1
262 2.477,1
263 2.591,1
264 2.697,1
265 2.8,1
266 2.902,1
267 3.009,1
268 3.112,1
269 3.215,1
270 3.326,1
271 3.433,1
272 3.542,1
273 3.652,1
274 3.762,1
275 3.873,1
276 3.982,1
277 4.09,1
278 4.202,1
279 4.315,1
280 4.421,1
281 4.533,1
282 4.641,1
283 4.746,1
284 4.858,1
285 4.966,1
286 5.077,1
287 5.182,1
288 5.291,1
289 5.405,1
290 5.519,1
291 5.628,1
292 5.733,1
293 5.839,1
294 5.952,1
295 6.057,1
296 6.166,1
297 6.271,1
298 6.376,1
299 6.479,1
300 6.583,1
301 6.691,1
302 6.8,1
303 6.912,1
304 7.024,1
305 7.136,1
306 7.241,1
307 7.351,1
308 7.457,1
309 7.566,1
310 7.677,1
311 7.79,1
312 7.896,1
313 8.008,1
314 8.12,1
315 8.224,1
316 8.338,1
317 8.451,1
318 8.564,1
319 8.677,1
320 8.79,1
321 8.894,1
322 8.999,1
323 9.109,1
324 9.213,1
325 9.323,1
326 9.428,1
327 9.541,1
328 9.654,1
329 9.766,1
330 9.879,1
331 9.982,1
332 10.094,1
333 10.202,1
334 10.315,1
335 10.427,1
336 10.538,1
337 10.64,1
338 10.753,1
339 10.858,1
340 10.964,1
341 11.074,1
342 11.178,1
343 11.287,1
344 11.399,1
345 11.507,1
346 11.617,1
347 11.723,1
348 11.83,1
349 11.942,1
350 12.054,1
351 12.166,1
352 12.273,1
353 12.382,1
354 12.49,1
355 12.598,1
356 12.71,1
357 12.818,1
358 12.93,1
359 13.038,1
360 13.145,1
361 13.257,1
362 13.366,1
363 13.477,1
364 13.588,1
365 13.693,1
366 13.797,1
367 13.908,1
368 14.02,1
369 14.127,1
370 14.237,1
371 14.34,1
372 14.452,1
373 14.562,1
374 14.666,1
375 14.769,1
376 14.882,1
377 14.99,2
378 15.101,2
379 15.212,2
380 15.314,2
381 15.426,2
382 15.534,2
383 15.638,2
384 15.743,2
385 15.849,2
386 15.961,2
387 16.072,2
388 16.184,2
389 16.291,2
390 16.396,2
391 16.507,2
392 16.619,2
393 16.731,2
394 16.837,2
395 16.951,2
396 17.062,2
397 17.165,2
398 17.274,2
399 17.385,2
400 17.495,2
401 17.604,2
402 17.716,2
403 17.828,2
404 17.94,2
405 18.052,2
406 18.161,2
407 18.268,2
408 18.372,2
409 18.482,2
410 18.593,2
411 18.699,2
412 18.81,2
413 18.923,2
414 19.03,2
415 19.14,2
416 19.25,2
417 19.362,2
418 19.473,2
419 19.585,2
420 19.695,2
421 19.798,2
422 19.903,2
423 20.014,2
424 20.127,2
425 20.238,2
426 20.351,2
427 20.457,2
428 20.562,2
429 20.668,2
430 20.778,2
431 20.888,2
432 20.997,2
433 21.106,2
434 21.218,2
435 21.322,2
436 21.436,2
437 21.541,2
438 21.654,2
439 21.768,2
440 21.882,2
441 21.985,2
442 22.097,2
443 22.21,2
444 22.322,2
445 22.433,2
446 22.54,2
447 22.653,2
448 22.76,2
449 22.867,2
450 22.978,2
451 23.09,2
452 23.199,2
453 23.309,2
454 23.414,2
455 23.521,2
456 23.623,2
457 23.734,2
458 23.845,2
459 23.949,2
460 24.058,2
461 24.165,2
462 24.268,2
463 24.37,2
464 24.48,2
465 24.582,2
466 24.691,2
467 24.802,2
468 24.908,2
469 25.022,2
470 25.136,2
471 25.25,2
472 25.354,2
473 25.466,2
474 25.578,2
475 25.691,2
476 25.803,2
477 25.907,2
478 26.015,2
479 26.125,2
480 26.232,2
481 26.335,2
482 26.443,2
483 26.555,2
484 26.668,2
485 26.773,2
486 26.878,2
487 26.99,2
488 27.103,2
489 27.217,2
490 27.321,2
491 27.433,2
492 27.54,2
493 27.651,2
494 27.764,2
495 27.876,2
496 27.988,2
497 28.099,2
498 28.211,2
499 28.322,2
500 28.435,2
501 28.537,2
502 28.649,2
503 28.76,2
504 28.863,2
505 28.972,2
506 29.084,2
507 29.196,2
508 29.308,2
509 29.417,2
510 29.528,2
511 29.637,2
512 29.742,2
513 29.848,2
514 29.956,2
515 30.069,2
516 30.177,2
517 30.29,2
518 30.392,2
519 30.498,2
520 30.607,2
521 30.714,2
522 30.823,2
523 30.926,2
524 31.035,2
525 31.139,2
526 31.242,2
527 31.351,2
528 31.461,2
529 31.567,2
530 31.678,2
531 31.79,2
532 31.898,2
533 32.012,2
534 32.123,2
535 32.231,2
536 32.344,2
537 32.457,2
538 32.569,2
539 32.681,2
540 32.788,2
541 32.898,2
542 33.01,2
543 33.122,2
544 33.231,2
545 33.346,2
546 33.457,2
547 33.565,2
548 33.673,2
549 33.783,2
550 33.896,2
551 34.006,2
552 34.12,2
553 34.223,2
554 34.33,2
555 34.443,2
556 34.555,2
557 34.665,2
558 34.773,2
559 34.881,2
560 34.986,2
561 35.096,2
562 35.209,2
563 35.313,2
564 35.422,2
565 35.534,2
566 35.638,2
567 35.75,2
568 35.856,2
569 35.965,2
570 36.068,2
571 36.181,2
572 36.29,2
573 36.398,2
574 36.507,2
575 36.615,2
576 36.728,2
577 36.84,2
578 36.953,2
579 37.065,2
580 37.177,2
581 37.289,2
582 37.4,2
583 37.504,2
584 37.614,2
585 37.722,2
586 37.834,2
587 37.94,2
588 38.047,2
589 38.154,2
590 38.267,2
591 38.371,2
592 38.483,2
593 38.593,2
594 38.705,2
595 38.814,2
596 38.918,2
597 39.023,2
598 39.135,2
599 39.237,2
600 39.341,2
601 39.446,2
602 39.553,2
603 39.656,2
604 39.765,2
605 39.873,2
606 39.979,2
607 40.081,2
608 40.192,2
609 40.299,2
610 40.403,2
611 40.507,2
612 40.614,2
613 40.726,2
614 40.83,2
615 40.941,2
616 41.047,2
617 41.158,2
618 41.261,2
619 41.37,2
620 41.481,2
621 41.589,2
622 41.697,2
623 41.808,2
624 41.916,2
625 42.018,2
626 42.123,2
627 42.236,2
628 42.339,2
629 42.448,2
630 42.559,2
631 42.664,2
632 42.777,2
633 42.889,2
634 42.998,2
635 43.105,2
636 43.217,2
637 43.328,2
638 43.441,2
639 43.554,2
640 43.667,2
641 43.774,2
642 43.88,2
643 43.99,2
644 44.102,2
645 44.206,2
646 44.315,2
647 44.426,2
648 44.53,2
649 44.636,2
650 44.741,2
651 44.846,2
652 44.958,2
653 45.068,2
654 45.181,2
655 45.285,2
656 45.394,2
657 45.497,2
@@ -0,0 +1,644 @@
# date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
t_rel_s,status,latency_ms,instance
-25.748,201,3.4,7cfd91ddf780
-25.644,201,5.5,5bbf259ff136
-25.536,201,4.8,7cfd91ddf780
-25.431,201,3.4,7cfd91ddf780
-25.325,201,3.2,7cfd91ddf780
-25.217,201,4.9,5bbf259ff136
-25.107,201,3.4,7cfd91ddf780
-24.997,201,3.0,7cfd91ddf780
-24.884,201,3.5,7cfd91ddf780
-24.778,201,5.8,7cfd91ddf780
-24.664,201,5.3,7cfd91ddf780
-24.558,201,6.9,7cfd91ddf780
-24.442,201,4.8,5bbf259ff136
-24.331,201,5.8,5bbf259ff136
-24.215,201,5.9,5bbf259ff136
-24.1,201,5.1,7cfd91ddf780
-23.99,201,5.2,7cfd91ddf780
-23.875,201,5.0,7cfd91ddf780
-23.766,201,4.3,7cfd91ddf780
-23.657,201,5.1,7cfd91ddf780
-23.541,201,5.3,5bbf259ff136
-23.426,201,6.7,5bbf259ff136
-23.318,201,5.7,5bbf259ff136
-23.208,201,5.1,5bbf259ff136
-23.092,201,6.1,7cfd91ddf780
-22.978,201,4.6,7cfd91ddf780
-22.864,201,4.3,5bbf259ff136
-22.759,201,4.6,7cfd91ddf780
-22.644,201,5.3,5bbf259ff136
-22.531,201,5.7,5bbf259ff136
-22.417,201,4.9,5bbf259ff136
-22.308,201,4.6,5bbf259ff136
-22.197,201,5.7,5bbf259ff136
-22.082,201,5.0,7cfd91ddf780
-21.967,201,5.8,7cfd91ddf780
-21.852,201,5.3,5bbf259ff136
-21.74,201,4.5,5bbf259ff136
-21.625,201,6.9,7cfd91ddf780
-21.515,201,6.3,5bbf259ff136
-21.402,201,5.0,7cfd91ddf780
-21.287,201,4.5,7cfd91ddf780
-21.179,201,4.6,7cfd91ddf780
-21.067,201,5.3,7cfd91ddf780
-20.95,201,6.0,5bbf259ff136
-20.835,201,3.9,5bbf259ff136
-20.724,201,3.8,7cfd91ddf780
-20.617,201,4.8,5bbf259ff136
-20.506,201,6.4,5bbf259ff136
-20.392,201,5.9,7cfd91ddf780
-20.284,201,5.1,7cfd91ddf780
-20.175,201,7.0,5bbf259ff136
-20.068,201,4.9,5bbf259ff136
-19.958,201,5.2,5bbf259ff136
-19.843,201,5.2,7cfd91ddf780
-19.728,201,4.6,7cfd91ddf780
-19.622,201,6.1,7cfd91ddf780
-19.514,201,5.2,7cfd91ddf780
-19.405,201,5.7,7cfd91ddf780
-19.295,201,4.9,5bbf259ff136
-19.182,201,4.7,7cfd91ddf780
-19.076,201,6.0,5bbf259ff136
-18.967,201,5.8,5bbf259ff136
-18.86,201,5.6,5bbf259ff136
-18.748,201,6.3,5bbf259ff136
-18.642,201,6.0,5bbf259ff136
-18.533,201,5.0,5bbf259ff136
-18.426,201,5.5,7cfd91ddf780
-18.315,201,5.3,5bbf259ff136
-18.208,201,6.1,7cfd91ddf780
-18.092,201,4.6,5bbf259ff136
-17.979,201,5.3,5bbf259ff136
-17.867,201,5.5,5bbf259ff136
-17.752,201,3.8,7cfd91ddf780
-17.643,201,5.2,7cfd91ddf780
-17.531,201,5.1,7cfd91ddf780
-17.415,201,7.0,5bbf259ff136
-17.3,201,5.9,5bbf259ff136
-17.187,201,5.5,5bbf259ff136
-17.076,201,7.6,7cfd91ddf780
-16.967,201,5.9,5bbf259ff136
-16.86,201,4.9,5bbf259ff136
-16.747,201,6.0,5bbf259ff136
-16.631,201,6.8,7cfd91ddf780
-16.516,201,5.4,7cfd91ddf780
-16.407,201,4.9,7cfd91ddf780
-16.297,201,4.4,7cfd91ddf780
-16.188,201,4.9,7cfd91ddf780
-16.073,201,5.0,5bbf259ff136
-15.964,201,5.2,5bbf259ff136
-15.849,201,4.9,5bbf259ff136
-15.74,201,3.9,5bbf259ff136
-15.636,201,3.4,7cfd91ddf780
-15.523,201,5.0,5bbf259ff136
-15.416,201,4.3,7cfd91ddf780
-15.302,201,5.4,5bbf259ff136
-15.192,201,5.1,5bbf259ff136
-15.077,201,5.5,5bbf259ff136
-14.969,201,6.1,7cfd91ddf780
-14.859,201,5.1,7cfd91ddf780
-14.745,201,5.3,7cfd91ddf780
-14.63,201,6.8,5bbf259ff136
-14.514,201,5.5,5bbf259ff136
-14.398,201,6.2,7cfd91ddf780
-14.282,201,6.5,5bbf259ff136
-14.168,201,7.5,5bbf259ff136
-14.059,201,6.1,5bbf259ff136
-13.952,201,5.3,5bbf259ff136
-13.838,201,5.4,5bbf259ff136
-13.727,201,6.0,5bbf259ff136
-13.611,201,4.8,7cfd91ddf780
-13.504,201,6.4,5bbf259ff136
-13.394,201,6.9,7cfd91ddf780
-13.283,201,5.9,7cfd91ddf780
-13.172,201,6.3,7cfd91ddf780
-13.059,201,4.9,7cfd91ddf780
-12.948,201,4.7,7cfd91ddf780
-12.835,201,7.7,7cfd91ddf780
-12.722,201,6.1,5bbf259ff136
-12.607,201,5.3,5bbf259ff136
-12.5,201,5.3,5bbf259ff136
-12.395,201,6.2,7cfd91ddf780
-12.288,201,4.9,7cfd91ddf780
-12.18,201,5.3,7cfd91ddf780
-12.074,201,6.5,7cfd91ddf780
-11.967,201,5.8,5bbf259ff136
-11.859,201,4.6,5bbf259ff136
-11.747,201,5.1,5bbf259ff136
-11.637,201,4.7,5bbf259ff136
-11.525,201,5.4,5bbf259ff136
-11.41,201,5.2,7cfd91ddf780
-11.296,201,5.5,7cfd91ddf780
-11.189,201,5.4,7cfd91ddf780
-11.083,201,5.3,7cfd91ddf780
-10.968,201,4.5,7cfd91ddf780
-10.858,201,5.1,7cfd91ddf780
-10.75,201,5.1,7cfd91ddf780
-10.643,201,6.3,7cfd91ddf780
-10.534,201,3.8,7cfd91ddf780
-10.427,201,4.0,5bbf259ff136
-10.32,201,5.9,5bbf259ff136
-10.209,201,5.5,5bbf259ff136
-10.095,201,5.4,5bbf259ff136
-9.98,201,4.9,7cfd91ddf780
-9.868,201,5.4,7cfd91ddf780
-9.761,201,6.5,7cfd91ddf780
-9.644,201,4.8,5bbf259ff136
-9.531,201,6.3,5bbf259ff136
-9.417,201,5.1,5bbf259ff136
-9.303,201,4.8,7cfd91ddf780
-9.192,201,5.6,7cfd91ddf780
-9.084,201,4.8,7cfd91ddf780
-8.975,201,4.7,5bbf259ff136
-8.863,201,4.7,7cfd91ddf780
-8.75,201,4.8,7cfd91ddf780
-8.643,201,4.7,7cfd91ddf780
-8.536,201,5.2,5bbf259ff136
-8.424,201,5.2,5bbf259ff136
-8.318,201,7.7,5bbf259ff136
-8.205,201,5.1,5bbf259ff136
-8.098,201,5.7,5bbf259ff136
-7.983,201,6.8,7cfd91ddf780
-7.869,201,4.6,7cfd91ddf780
-7.76,201,5.0,7cfd91ddf780
-7.645,201,5.5,7cfd91ddf780
-7.531,201,5.0,5bbf259ff136
-7.425,201,3.3,7cfd91ddf780
-7.315,201,5.0,7cfd91ddf780
-7.202,201,5.6,5bbf259ff136
-7.092,201,5.2,5bbf259ff136
-6.985,201,4.9,7cfd91ddf780
-6.877,201,6.5,7cfd91ddf780
-6.76,201,5.0,5bbf259ff136
-6.65,201,4.6,5bbf259ff136
-6.536,201,5.2,5bbf259ff136
-6.423,201,5.4,7cfd91ddf780
-6.307,201,4.0,7cfd91ddf780
-6.203,201,4.3,7cfd91ddf780
-6.093,201,4.3,7cfd91ddf780
-5.985,201,7.2,5bbf259ff136
-5.868,201,4.4,7cfd91ddf780
-5.761,201,4.2,7cfd91ddf780
-5.657,201,4.9,5bbf259ff136
-5.546,201,5.7,5bbf259ff136
-5.434,201,4.5,7cfd91ddf780
-5.325,201,3.8,5bbf259ff136
-5.216,201,3.9,5bbf259ff136
-5.112,201,5.1,5bbf259ff136
-5.0,201,6.2,5bbf259ff136
-4.893,201,5.2,5bbf259ff136
-4.784,201,6.3,5bbf259ff136
-4.675,201,5.5,7cfd91ddf780
-4.567,201,4.6,7cfd91ddf780
-4.453,201,4.8,5bbf259ff136
-4.338,201,5.0,5bbf259ff136
-4.229,201,6.4,5bbf259ff136
-4.115,201,6.0,5bbf259ff136
-4.007,201,5.4,5bbf259ff136
-3.896,201,6.5,5bbf259ff136
-3.782,201,5.5,7cfd91ddf780
-3.669,201,5.5,7cfd91ddf780
-3.562,201,5.4,7cfd91ddf780
-3.452,201,5.5,7cfd91ddf780
-3.341,201,5.9,7cfd91ddf780
-3.234,201,6.5,7cfd91ddf780
-3.117,201,5.1,7cfd91ddf780
-3.007,201,6.6,7cfd91ddf780
-2.892,201,6.3,5bbf259ff136
-2.775,201,5.6,5bbf259ff136
-2.669,201,6.0,5bbf259ff136
-2.559,201,6.6,5bbf259ff136
-2.442,201,6.4,7cfd91ddf780
-2.326,201,3.6,7cfd91ddf780
-2.214,201,5.1,5bbf259ff136
-2.099,201,4.8,5bbf259ff136
-1.986,201,6.5,5bbf259ff136
-1.877,201,7.0,5bbf259ff136
-1.768,201,4.7,5bbf259ff136
-1.658,201,5.8,5bbf259ff136
-1.548,201,5.2,5bbf259ff136
-1.433,201,5.4,7cfd91ddf780
-1.326,201,7.0,5bbf259ff136
-1.217,201,5.3,5bbf259ff136
-1.108,201,5.1,5bbf259ff136
-0.995,201,4.4,5bbf259ff136
-0.884,201,5.8,7cfd91ddf780
-0.776,201,5.7,5bbf259ff136
-0.662,201,5.5,7cfd91ddf780
-0.554,201,6.5,7cfd91ddf780
-0.442,201,4.7,7cfd91ddf780
-0.331,201,5.1,7cfd91ddf780
-0.223,201,3.6,5bbf259ff136
-0.111,201,3.9,7cfd91ddf780
-0.004,201,5.0,5bbf259ff136
0.106,201,4.8,7cfd91ddf780
0.217,201,5.2,5bbf259ff136
0.323,201,6.3,5bbf259ff136
0.44,201,5.2,7cfd91ddf780
0.547,201,5.5,7cfd91ddf780
0.657,201,3.2,7cfd91ddf780
0.771,201,2.8,7cfd91ddf780
0.879,201,3.9,7cfd91ddf780
0.993,201,3.9,7cfd91ddf780
1.107,201,4.7,5bbf259ff136
1.222,201,4.9,5bbf259ff136
1.333,201,3.9,5bbf259ff136
1.439,201,6.0,5bbf259ff136
1.551,201,6.0,7cfd91ddf780
1.66,201,6.1,7cfd91ddf780
1.775,201,5.1,7cfd91ddf780
1.887,201,7.0,7cfd91ddf780
2.004,201,5.9,7cfd91ddf780
2.12,201,5.4,7cfd91ddf780
2.233,201,4.1,7cfd91ddf780
2.338,201,3.8,7cfd91ddf780
2.449,201,6.8,7cfd91ddf780
2.557,201,4.3,7cfd91ddf780
2.661,201,6.9,7cfd91ddf780
2.768,201,4.5,7cfd91ddf780
2.876,201,4.2,7cfd91ddf780
2.983,201,4.6,7cfd91ddf780
3.098,201,4.4,7cfd91ddf780
3.203,201,5.2,7cfd91ddf780
3.31,201,5.2,7cfd91ddf780
3.425,201,5.5,7cfd91ddf780
3.537,201,5.9,7cfd91ddf780
3.646,201,8.7,7cfd91ddf780
3.758,201,5.7,7cfd91ddf780
3.867,201,7.2,7cfd91ddf780
3.982,201,6.7,7cfd91ddf780
4.099,201,5.7,7cfd91ddf780
4.206,201,4.6,7cfd91ddf780
4.316,201,13.6,7cfd91ddf780
4.439,201,4.2,7cfd91ddf780
4.554,201,5.9,7cfd91ddf780
4.665,201,7.6,7cfd91ddf780
4.781,201,4.8,7cfd91ddf780
4.896,201,5.9,7cfd91ddf780
5.008,201,5.5,7cfd91ddf780
5.123,201,4.3,7cfd91ddf780
5.233,201,4.4,7cfd91ddf780
5.338,201,5.1,7cfd91ddf780
5.449,201,5.8,7cfd91ddf780
5.557,201,5.6,7cfd91ddf780
5.673,201,6.9,7cfd91ddf780
5.781,201,6.3,7cfd91ddf780
5.887,201,6.8,7cfd91ddf780
6.004,201,5.0,7cfd91ddf780
6.118,201,5.1,7cfd91ddf780
6.232,201,5.5,7cfd91ddf780
6.347,201,5.2,7cfd91ddf780
6.454,201,5.8,7cfd91ddf780
6.564,201,5.1,7cfd91ddf780
6.674,201,4.4,7cfd91ddf780
6.783,201,5.7,7cfd91ddf780
6.892,201,3.0,7cfd91ddf780
7.004,201,6.7,7cfd91ddf780
7.119,201,6.2,7cfd91ddf780
7.235,201,5.3,7cfd91ddf780
7.351,201,6.6,7cfd91ddf780
7.458,201,6.4,7cfd91ddf780
7.568,201,5.2,7cfd91ddf780
7.677,201,7.3,7cfd91ddf780
7.793,201,5.2,7cfd91ddf780
7.899,201,3.3,7cfd91ddf780
8.012,201,2.8,7cfd91ddf780
8.124,201,3.9,7cfd91ddf780
8.232,201,4.9,7cfd91ddf780
8.337,201,6.0,7cfd91ddf780
8.449,201,5.9,7cfd91ddf780
8.563,201,5.8,7cfd91ddf780
8.674,201,7.4,7cfd91ddf780
8.782,201,4.3,7cfd91ddf780
8.894,201,6.0,7cfd91ddf780
9.01,201,4.9,7cfd91ddf780
9.122,201,5.6,7cfd91ddf780
9.233,201,6.0,7cfd91ddf780
9.339,201,5.2,7cfd91ddf780
9.449,201,6.0,7cfd91ddf780
9.556,201,6.4,7cfd91ddf780
9.673,201,5.0,7cfd91ddf780
9.788,201,5.0,7cfd91ddf780
9.899,201,4.8,7cfd91ddf780
10.009,201,5.2,7cfd91ddf780
10.12,201,5.8,7cfd91ddf780
10.233,201,4.7,7cfd91ddf780
10.348,201,4.6,7cfd91ddf780
10.453,201,4.3,7cfd91ddf780
10.566,201,5.3,7cfd91ddf780
10.681,201,5.9,7cfd91ddf780
10.789,201,7.5,7cfd91ddf780
10.899,201,6.3,7cfd91ddf780
11.012,201,6.7,7cfd91ddf780
11.122,201,4.8,7cfd91ddf780
11.228,201,5.5,7cfd91ddf780
11.343,201,4.9,7cfd91ddf780
11.458,201,5.8,7cfd91ddf780
11.573,201,5.4,7cfd91ddf780
11.688,201,6.7,7cfd91ddf780
11.805,201,5.5,7cfd91ddf780
11.916,201,6.0,7cfd91ddf780
12.023,201,5.8,7cfd91ddf780
12.137,201,6.0,7cfd91ddf780
12.244,201,4.7,7cfd91ddf780
12.355,201,6.5,7cfd91ddf780
12.466,201,5.1,7cfd91ddf780
12.578,201,5.1,7cfd91ddf780
12.69,201,5.4,7cfd91ddf780
12.8,201,5.2,7cfd91ddf780
12.915,201,4.0,7cfd91ddf780
13.026,201,4.9,7cfd91ddf780
13.138,201,3.7,7cfd91ddf780
13.242,201,4.3,7cfd91ddf780
13.351,201,3.3,7cfd91ddf780
13.455,201,3.9,7cfd91ddf780
13.569,201,3.5,7cfd91ddf780
13.676,201,3.9,7cfd91ddf780
13.782,201,3.1,7cfd91ddf780
13.89,201,3.8,7cfd91ddf780
13.999,201,4.0,7cfd91ddf780
14.103,201,4.8,7cfd91ddf780
14.21,201,4.8,7cfd91ddf780
14.318,201,5.0,7cfd91ddf780
14.43,201,5.3,7cfd91ddf780
14.543,201,5.1,7cfd91ddf780
14.65,201,5.2,7cfd91ddf780
14.756,201,4.8,7cfd91ddf780
14.863,201,5.7,7cfd91ddf780
14.979,201,5.9,7cfd91ddf780
15.093,201,5.5,7cfd91ddf780
15.203,201,5.1,7cfd91ddf780
15.309,201,4.9,7cfd91ddf780
15.416,201,8.0,7cfd91ddf780
15.534,201,6.0,7cfd91ddf780
15.642,201,3.5,7cfd91ddf780
15.756,201,4.7,5bbf259ff136
15.871,201,6.0,5bbf259ff136
15.987,201,4.7,7cfd91ddf780
16.097,201,4.9,7cfd91ddf780
16.212,201,5.2,7cfd91ddf780
16.324,201,6.2,7cfd91ddf780
16.44,201,5.5,5bbf259ff136
16.556,201,4.9,7cfd91ddf780
16.666,201,5.2,7cfd91ddf780
16.777,201,6.3,7cfd91ddf780
16.893,201,5.8,7cfd91ddf780
17.009,201,5.4,5bbf259ff136
17.123,201,5.5,5bbf259ff136
17.229,201,5.4,5bbf259ff136
17.343,201,5.1,5bbf259ff136
17.457,201,5.4,7cfd91ddf780
17.564,201,5.2,5bbf259ff136
17.679,201,7.2,7cfd91ddf780
17.79,201,6.8,7cfd91ddf780
17.899,201,5.0,7cfd91ddf780
18.013,201,5.1,7cfd91ddf780
18.126,201,8.5,5bbf259ff136
18.241,201,3.9,5bbf259ff136
18.352,201,4.5,5bbf259ff136
18.465,201,5.5,7cfd91ddf780
18.577,201,5.9,7cfd91ddf780
18.691,201,4.8,7cfd91ddf780
18.806,201,14.2,5bbf259ff136
18.92,201,5.3,5bbf259ff136
19.03,201,4.6,5bbf259ff136
19.141,201,5.5,5bbf259ff136
19.249,201,5.7,5bbf259ff136
19.357,201,4.8,5bbf259ff136
19.468,201,5.1,5bbf259ff136
19.582,201,6.6,5bbf259ff136
19.695,201,5.3,7cfd91ddf780
19.802,201,5.1,5bbf259ff136
19.912,201,4.9,5bbf259ff136
20.021,201,5.2,5bbf259ff136
20.136,201,5.4,7cfd91ddf780
20.251,201,4.2,7cfd91ddf780
20.361,201,5.0,7cfd91ddf780
20.473,201,5.2,7cfd91ddf780
20.582,201,5.0,7cfd91ddf780
20.692,201,5.9,7cfd91ddf780
20.805,201,4.7,7cfd91ddf780
20.912,201,4.5,7cfd91ddf780
21.023,201,4.7,7cfd91ddf780
21.137,201,5.2,5bbf259ff136
21.249,201,4.4,5bbf259ff136
21.362,201,5.1,5bbf259ff136
21.472,201,5.0,5bbf259ff136
21.583,201,5.6,5bbf259ff136
21.691,201,6.1,5bbf259ff136
21.8,201,7.2,5bbf259ff136
21.917,201,6.8,7cfd91ddf780
22.03,201,5.0,7cfd91ddf780
22.137,201,6.5,7cfd91ddf780
22.253,201,7.0,7cfd91ddf780
22.364,201,5.3,7cfd91ddf780
22.474,201,5.3,7cfd91ddf780
22.582,201,5.7,7cfd91ddf780
22.691,201,5.6,7cfd91ddf780
22.801,201,4.9,7cfd91ddf780
22.916,201,4.4,5bbf259ff136
23.02,201,4.9,7cfd91ddf780
23.135,201,5.3,5bbf259ff136
23.242,201,6.3,7cfd91ddf780
23.358,201,4.7,5bbf259ff136
23.467,201,4.5,5bbf259ff136
23.572,201,4.1,7cfd91ddf780
23.682,201,5.6,7cfd91ddf780
23.79,201,4.8,7cfd91ddf780
23.905,201,5.1,5bbf259ff136
24.015,201,5.0,5bbf259ff136
24.122,201,4.7,7cfd91ddf780
24.233,201,4.4,7cfd91ddf780
24.342,201,5.8,7cfd91ddf780
24.458,201,6.0,5bbf259ff136
24.569,201,5.4,5bbf259ff136
24.684,201,6.7,7cfd91ddf780
24.801,201,5.3,7cfd91ddf780
24.916,201,5.2,5bbf259ff136
25.023,201,6.6,7cfd91ddf780
25.14,201,5.7,5bbf259ff136
25.251,201,4.5,5bbf259ff136
25.357,201,5.0,5bbf259ff136
25.472,201,5.1,5bbf259ff136
25.582,201,4.8,5bbf259ff136
25.691,201,6.0,5bbf259ff136
25.807,201,5.2,7cfd91ddf780
25.915,201,4.9,7cfd91ddf780
26.03,201,5.1,7cfd91ddf780
26.14,201,5.1,7cfd91ddf780
26.247,201,5.6,7cfd91ddf780
26.354,201,5.3,5bbf259ff136
26.467,201,6.8,7cfd91ddf780
26.583,201,5.2,7cfd91ddf780
26.694,201,5.2,7cfd91ddf780
26.806,201,6.2,5bbf259ff136
26.914,201,6.5,5bbf259ff136
27.031,201,5.9,5bbf259ff136
27.144,201,5.4,7cfd91ddf780
27.255,201,5.1,7cfd91ddf780
27.362,201,6.0,5bbf259ff136
27.478,201,6.3,7cfd91ddf780
27.59,201,6.3,7cfd91ddf780
27.707,201,5.4,5bbf259ff136
27.814,201,7.4,5bbf259ff136
27.929,201,3.9,5bbf259ff136
28.04,201,4.0,5bbf259ff136
28.153,201,5.2,5bbf259ff136
28.269,201,4.8,7cfd91ddf780
28.381,201,5.6,7cfd91ddf780
28.487,201,5.1,7cfd91ddf780
28.597,201,5.1,7cfd91ddf780
28.708,201,3.9,7cfd91ddf780
28.816,201,5.1,7cfd91ddf780
28.931,201,4.9,7cfd91ddf780
29.046,201,5.9,5bbf259ff136
29.162,201,4.9,5bbf259ff136
29.276,201,3.8,7cfd91ddf780
29.381,201,5.3,5bbf259ff136
29.489,201,4.7,5bbf259ff136
29.6,201,4.9,5bbf259ff136
29.715,201,5.9,7cfd91ddf780
29.825,201,5.5,7cfd91ddf780
29.94,201,5.9,7cfd91ddf780
30.047,201,7.0,7cfd91ddf780
30.155,201,5.5,7cfd91ddf780
30.271,201,6.1,7cfd91ddf780
30.38,201,4.9,7cfd91ddf780
30.49,201,5.3,7cfd91ddf780
30.597,201,7.2,7cfd91ddf780
30.714,201,6.6,5bbf259ff136
30.822,201,6.9,7cfd91ddf780
30.933,201,6.4,7cfd91ddf780
31.049,201,3.8,5bbf259ff136
31.163,201,4.9,5bbf259ff136
31.273,201,5.4,5bbf259ff136
31.383,201,5.1,5bbf259ff136
31.49,201,5.0,5bbf259ff136
31.605,201,5.9,7cfd91ddf780
31.717,201,6.5,7cfd91ddf780
31.825,201,5.7,7cfd91ddf780
31.938,201,5.0,7cfd91ddf780
32.053,201,4.8,7cfd91ddf780
32.164,201,4.9,7cfd91ddf780
32.271,201,4.6,7cfd91ddf780
32.378,201,4.7,7cfd91ddf780
32.49,201,5.0,7cfd91ddf780
32.605,201,4.2,7cfd91ddf780
32.717,201,4.1,5bbf259ff136
32.828,201,5.4,5bbf259ff136
32.937,201,6.0,5bbf259ff136
33.048,201,6.2,5bbf259ff136
33.163,201,5.3,5bbf259ff136
33.276,201,6.7,7cfd91ddf780
33.39,201,6.6,7cfd91ddf780
33.501,201,5.0,7cfd91ddf780
33.616,201,5.3,5bbf259ff136
33.722,201,4.9,7cfd91ddf780
33.831,201,3.9,7cfd91ddf780
33.945,201,5.9,5bbf259ff136
34.06,201,5.4,5bbf259ff136
34.168,201,5.5,5bbf259ff136
34.28,201,5.6,5bbf259ff136
34.392,201,3.1,5bbf259ff136
34.505,201,5.5,7cfd91ddf780
34.621,201,4.7,7cfd91ddf780
34.736,201,5.2,5bbf259ff136
34.851,201,6.5,5bbf259ff136
34.965,201,5.7,7cfd91ddf780
35.072,201,4.9,5bbf259ff136
35.182,201,6.6,5bbf259ff136
35.291,201,4.8,5bbf259ff136
35.403,201,6.7,5bbf259ff136
35.518,201,6.8,7cfd91ddf780
35.627,201,4.7,7cfd91ddf780
35.732,201,6.7,5bbf259ff136
35.848,201,5.8,7cfd91ddf780
35.962,201,6.7,7cfd91ddf780
36.073,201,5.2,7cfd91ddf780
36.182,201,6.8,7cfd91ddf780
36.29,201,6.2,7cfd91ddf780
36.405,201,5.0,5bbf259ff136
36.52,201,5.0,5bbf259ff136
36.634,201,5.0,5bbf259ff136
36.747,201,4.6,7cfd91ddf780
36.861,201,5.0,7cfd91ddf780
36.974,201,5.6,5bbf259ff136
37.088,201,4.8,5bbf259ff136
37.203,201,5.6,5bbf259ff136
37.314,201,5.5,7cfd91ddf780
37.43,201,5.7,7cfd91ddf780
37.545,201,4.8,7cfd91ddf780
37.659,201,4.3,5bbf259ff136
37.765,201,5.4,5bbf259ff136
37.873,201,5.3,7cfd91ddf780
37.981,201,6.1,7cfd91ddf780
38.097,201,6.0,5bbf259ff136
38.214,201,6.3,7cfd91ddf780
38.327,201,5.5,7cfd91ddf780
38.44,201,6.0,7cfd91ddf780
38.548,201,5.4,7cfd91ddf780
38.656,201,6.9,7cfd91ddf780
38.765,201,3.5,5bbf259ff136
38.868,201,3.5,5bbf259ff136
38.981,201,3.7,5bbf259ff136
39.088,201,3.1,5bbf259ff136
39.2,201,3.6,5bbf259ff136
39.308,201,4.0,5bbf259ff136
39.414,201,4.3,7cfd91ddf780
39.523,201,3.5,7cfd91ddf780
39.635,201,4.3,5bbf259ff136
39.744,201,4.2,5bbf259ff136
39.859,201,4.0,5bbf259ff136
39.965,201,3.4,5bbf259ff136
40.076,201,7.7,5bbf259ff136
40.192,201,6.5,5bbf259ff136
40.306,201,5.0,7cfd91ddf780
40.414,201,3.5,5bbf259ff136
40.518,201,2.6,5bbf259ff136
40.623,201,2.6,7cfd91ddf780
40.733,201,4.6,7cfd91ddf780
40.845,201,4.6,5bbf259ff136
40.958,201,4.2,5bbf259ff136
41.064,201,4.2,7cfd91ddf780
41.171,201,2.8,7cfd91ddf780
41.281,201,4.9,7cfd91ddf780
41.388,201,4.3,7cfd91ddf780
41.497,201,5.0,7cfd91ddf780
41.606,201,5.1,7cfd91ddf780
41.721,201,2.9,7cfd91ddf780
41.829,201,4.8,7cfd91ddf780
41.942,201,3.0,7cfd91ddf780
42.053,201,5.6,7cfd91ddf780
42.164,201,6.5,7cfd91ddf780
42.281,201,5.6,5bbf259ff136
42.389,201,4.5,5bbf259ff136
42.503,201,4.6,5bbf259ff136
42.618,201,5.8,7cfd91ddf780
42.734,201,5.2,7cfd91ddf780
42.849,201,5.9,5bbf259ff136
42.959,201,5.0,5bbf259ff136
43.066,201,4.9,5bbf259ff136
43.181,201,6.2,5bbf259ff136
43.289,201,5.7,5bbf259ff136
43.4,201,6.0,5bbf259ff136
43.516,201,7.1,7cfd91ddf780
43.631,201,6.1,7cfd91ddf780
43.746,201,6.7,5bbf259ff136
43.862,201,5.9,5bbf259ff136
43.969,201,5.5,5bbf259ff136
44.084,201,5.4,7cfd91ddf780
44.2,201,4.6,7cfd91ddf780
44.305,201,5.8,7cfd91ddf780
44.421,201,6.1,7cfd91ddf780
44.53,201,5.2,7cfd91ddf780
44.637,201,3.7,7cfd91ddf780
44.75,201,3.3,7cfd91ddf780
44.855,201,3.8,5bbf259ff136
44.961,201,4.7,5bbf259ff136
45.072,201,6.1,7cfd91ddf780
45.184,201,7.3,5bbf259ff136
45.296,201,7.5,7cfd91ddf780
45.409,201,5.2,5bbf259ff136
45.521,201,5.5,7cfd91ddf780
1 # date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
2 t_rel_s,status,latency_ms,instance
3 -25.748,201,3.4,7cfd91ddf780
4 -25.644,201,5.5,5bbf259ff136
5 -25.536,201,4.8,7cfd91ddf780
6 -25.431,201,3.4,7cfd91ddf780
7 -25.325,201,3.2,7cfd91ddf780
8 -25.217,201,4.9,5bbf259ff136
9 -25.107,201,3.4,7cfd91ddf780
10 -24.997,201,3.0,7cfd91ddf780
11 -24.884,201,3.5,7cfd91ddf780
12 -24.778,201,5.8,7cfd91ddf780
13 -24.664,201,5.3,7cfd91ddf780
14 -24.558,201,6.9,7cfd91ddf780
15 -24.442,201,4.8,5bbf259ff136
16 -24.331,201,5.8,5bbf259ff136
17 -24.215,201,5.9,5bbf259ff136
18 -24.1,201,5.1,7cfd91ddf780
19 -23.99,201,5.2,7cfd91ddf780
20 -23.875,201,5.0,7cfd91ddf780
21 -23.766,201,4.3,7cfd91ddf780
22 -23.657,201,5.1,7cfd91ddf780
23 -23.541,201,5.3,5bbf259ff136
24 -23.426,201,6.7,5bbf259ff136
25 -23.318,201,5.7,5bbf259ff136
26 -23.208,201,5.1,5bbf259ff136
27 -23.092,201,6.1,7cfd91ddf780
28 -22.978,201,4.6,7cfd91ddf780
29 -22.864,201,4.3,5bbf259ff136
30 -22.759,201,4.6,7cfd91ddf780
31 -22.644,201,5.3,5bbf259ff136
32 -22.531,201,5.7,5bbf259ff136
33 -22.417,201,4.9,5bbf259ff136
34 -22.308,201,4.6,5bbf259ff136
35 -22.197,201,5.7,5bbf259ff136
36 -22.082,201,5.0,7cfd91ddf780
37 -21.967,201,5.8,7cfd91ddf780
38 -21.852,201,5.3,5bbf259ff136
39 -21.74,201,4.5,5bbf259ff136
40 -21.625,201,6.9,7cfd91ddf780
41 -21.515,201,6.3,5bbf259ff136
42 -21.402,201,5.0,7cfd91ddf780
43 -21.287,201,4.5,7cfd91ddf780
44 -21.179,201,4.6,7cfd91ddf780
45 -21.067,201,5.3,7cfd91ddf780
46 -20.95,201,6.0,5bbf259ff136
47 -20.835,201,3.9,5bbf259ff136
48 -20.724,201,3.8,7cfd91ddf780
49 -20.617,201,4.8,5bbf259ff136
50 -20.506,201,6.4,5bbf259ff136
51 -20.392,201,5.9,7cfd91ddf780
52 -20.284,201,5.1,7cfd91ddf780
53 -20.175,201,7.0,5bbf259ff136
54 -20.068,201,4.9,5bbf259ff136
55 -19.958,201,5.2,5bbf259ff136
56 -19.843,201,5.2,7cfd91ddf780
57 -19.728,201,4.6,7cfd91ddf780
58 -19.622,201,6.1,7cfd91ddf780
59 -19.514,201,5.2,7cfd91ddf780
60 -19.405,201,5.7,7cfd91ddf780
61 -19.295,201,4.9,5bbf259ff136
62 -19.182,201,4.7,7cfd91ddf780
63 -19.076,201,6.0,5bbf259ff136
64 -18.967,201,5.8,5bbf259ff136
65 -18.86,201,5.6,5bbf259ff136
66 -18.748,201,6.3,5bbf259ff136
67 -18.642,201,6.0,5bbf259ff136
68 -18.533,201,5.0,5bbf259ff136
69 -18.426,201,5.5,7cfd91ddf780
70 -18.315,201,5.3,5bbf259ff136
71 -18.208,201,6.1,7cfd91ddf780
72 -18.092,201,4.6,5bbf259ff136
73 -17.979,201,5.3,5bbf259ff136
74 -17.867,201,5.5,5bbf259ff136
75 -17.752,201,3.8,7cfd91ddf780
76 -17.643,201,5.2,7cfd91ddf780
77 -17.531,201,5.1,7cfd91ddf780
78 -17.415,201,7.0,5bbf259ff136
79 -17.3,201,5.9,5bbf259ff136
80 -17.187,201,5.5,5bbf259ff136
81 -17.076,201,7.6,7cfd91ddf780
82 -16.967,201,5.9,5bbf259ff136
83 -16.86,201,4.9,5bbf259ff136
84 -16.747,201,6.0,5bbf259ff136
85 -16.631,201,6.8,7cfd91ddf780
86 -16.516,201,5.4,7cfd91ddf780
87 -16.407,201,4.9,7cfd91ddf780
88 -16.297,201,4.4,7cfd91ddf780
89 -16.188,201,4.9,7cfd91ddf780
90 -16.073,201,5.0,5bbf259ff136
91 -15.964,201,5.2,5bbf259ff136
92 -15.849,201,4.9,5bbf259ff136
93 -15.74,201,3.9,5bbf259ff136
94 -15.636,201,3.4,7cfd91ddf780
95 -15.523,201,5.0,5bbf259ff136
96 -15.416,201,4.3,7cfd91ddf780
97 -15.302,201,5.4,5bbf259ff136
98 -15.192,201,5.1,5bbf259ff136
99 -15.077,201,5.5,5bbf259ff136
100 -14.969,201,6.1,7cfd91ddf780
101 -14.859,201,5.1,7cfd91ddf780
102 -14.745,201,5.3,7cfd91ddf780
103 -14.63,201,6.8,5bbf259ff136
104 -14.514,201,5.5,5bbf259ff136
105 -14.398,201,6.2,7cfd91ddf780
106 -14.282,201,6.5,5bbf259ff136
107 -14.168,201,7.5,5bbf259ff136
108 -14.059,201,6.1,5bbf259ff136
109 -13.952,201,5.3,5bbf259ff136
110 -13.838,201,5.4,5bbf259ff136
111 -13.727,201,6.0,5bbf259ff136
112 -13.611,201,4.8,7cfd91ddf780
113 -13.504,201,6.4,5bbf259ff136
114 -13.394,201,6.9,7cfd91ddf780
115 -13.283,201,5.9,7cfd91ddf780
116 -13.172,201,6.3,7cfd91ddf780
117 -13.059,201,4.9,7cfd91ddf780
118 -12.948,201,4.7,7cfd91ddf780
119 -12.835,201,7.7,7cfd91ddf780
120 -12.722,201,6.1,5bbf259ff136
121 -12.607,201,5.3,5bbf259ff136
122 -12.5,201,5.3,5bbf259ff136
123 -12.395,201,6.2,7cfd91ddf780
124 -12.288,201,4.9,7cfd91ddf780
125 -12.18,201,5.3,7cfd91ddf780
126 -12.074,201,6.5,7cfd91ddf780
127 -11.967,201,5.8,5bbf259ff136
128 -11.859,201,4.6,5bbf259ff136
129 -11.747,201,5.1,5bbf259ff136
130 -11.637,201,4.7,5bbf259ff136
131 -11.525,201,5.4,5bbf259ff136
132 -11.41,201,5.2,7cfd91ddf780
133 -11.296,201,5.5,7cfd91ddf780
134 -11.189,201,5.4,7cfd91ddf780
135 -11.083,201,5.3,7cfd91ddf780
136 -10.968,201,4.5,7cfd91ddf780
137 -10.858,201,5.1,7cfd91ddf780
138 -10.75,201,5.1,7cfd91ddf780
139 -10.643,201,6.3,7cfd91ddf780
140 -10.534,201,3.8,7cfd91ddf780
141 -10.427,201,4.0,5bbf259ff136
142 -10.32,201,5.9,5bbf259ff136
143 -10.209,201,5.5,5bbf259ff136
144 -10.095,201,5.4,5bbf259ff136
145 -9.98,201,4.9,7cfd91ddf780
146 -9.868,201,5.4,7cfd91ddf780
147 -9.761,201,6.5,7cfd91ddf780
148 -9.644,201,4.8,5bbf259ff136
149 -9.531,201,6.3,5bbf259ff136
150 -9.417,201,5.1,5bbf259ff136
151 -9.303,201,4.8,7cfd91ddf780
152 -9.192,201,5.6,7cfd91ddf780
153 -9.084,201,4.8,7cfd91ddf780
154 -8.975,201,4.7,5bbf259ff136
155 -8.863,201,4.7,7cfd91ddf780
156 -8.75,201,4.8,7cfd91ddf780
157 -8.643,201,4.7,7cfd91ddf780
158 -8.536,201,5.2,5bbf259ff136
159 -8.424,201,5.2,5bbf259ff136
160 -8.318,201,7.7,5bbf259ff136
161 -8.205,201,5.1,5bbf259ff136
162 -8.098,201,5.7,5bbf259ff136
163 -7.983,201,6.8,7cfd91ddf780
164 -7.869,201,4.6,7cfd91ddf780
165 -7.76,201,5.0,7cfd91ddf780
166 -7.645,201,5.5,7cfd91ddf780
167 -7.531,201,5.0,5bbf259ff136
168 -7.425,201,3.3,7cfd91ddf780
169 -7.315,201,5.0,7cfd91ddf780
170 -7.202,201,5.6,5bbf259ff136
171 -7.092,201,5.2,5bbf259ff136
172 -6.985,201,4.9,7cfd91ddf780
173 -6.877,201,6.5,7cfd91ddf780
174 -6.76,201,5.0,5bbf259ff136
175 -6.65,201,4.6,5bbf259ff136
176 -6.536,201,5.2,5bbf259ff136
177 -6.423,201,5.4,7cfd91ddf780
178 -6.307,201,4.0,7cfd91ddf780
179 -6.203,201,4.3,7cfd91ddf780
180 -6.093,201,4.3,7cfd91ddf780
181 -5.985,201,7.2,5bbf259ff136
182 -5.868,201,4.4,7cfd91ddf780
183 -5.761,201,4.2,7cfd91ddf780
184 -5.657,201,4.9,5bbf259ff136
185 -5.546,201,5.7,5bbf259ff136
186 -5.434,201,4.5,7cfd91ddf780
187 -5.325,201,3.8,5bbf259ff136
188 -5.216,201,3.9,5bbf259ff136
189 -5.112,201,5.1,5bbf259ff136
190 -5.0,201,6.2,5bbf259ff136
191 -4.893,201,5.2,5bbf259ff136
192 -4.784,201,6.3,5bbf259ff136
193 -4.675,201,5.5,7cfd91ddf780
194 -4.567,201,4.6,7cfd91ddf780
195 -4.453,201,4.8,5bbf259ff136
196 -4.338,201,5.0,5bbf259ff136
197 -4.229,201,6.4,5bbf259ff136
198 -4.115,201,6.0,5bbf259ff136
199 -4.007,201,5.4,5bbf259ff136
200 -3.896,201,6.5,5bbf259ff136
201 -3.782,201,5.5,7cfd91ddf780
202 -3.669,201,5.5,7cfd91ddf780
203 -3.562,201,5.4,7cfd91ddf780
204 -3.452,201,5.5,7cfd91ddf780
205 -3.341,201,5.9,7cfd91ddf780
206 -3.234,201,6.5,7cfd91ddf780
207 -3.117,201,5.1,7cfd91ddf780
208 -3.007,201,6.6,7cfd91ddf780
209 -2.892,201,6.3,5bbf259ff136
210 -2.775,201,5.6,5bbf259ff136
211 -2.669,201,6.0,5bbf259ff136
212 -2.559,201,6.6,5bbf259ff136
213 -2.442,201,6.4,7cfd91ddf780
214 -2.326,201,3.6,7cfd91ddf780
215 -2.214,201,5.1,5bbf259ff136
216 -2.099,201,4.8,5bbf259ff136
217 -1.986,201,6.5,5bbf259ff136
218 -1.877,201,7.0,5bbf259ff136
219 -1.768,201,4.7,5bbf259ff136
220 -1.658,201,5.8,5bbf259ff136
221 -1.548,201,5.2,5bbf259ff136
222 -1.433,201,5.4,7cfd91ddf780
223 -1.326,201,7.0,5bbf259ff136
224 -1.217,201,5.3,5bbf259ff136
225 -1.108,201,5.1,5bbf259ff136
226 -0.995,201,4.4,5bbf259ff136
227 -0.884,201,5.8,7cfd91ddf780
228 -0.776,201,5.7,5bbf259ff136
229 -0.662,201,5.5,7cfd91ddf780
230 -0.554,201,6.5,7cfd91ddf780
231 -0.442,201,4.7,7cfd91ddf780
232 -0.331,201,5.1,7cfd91ddf780
233 -0.223,201,3.6,5bbf259ff136
234 -0.111,201,3.9,7cfd91ddf780
235 -0.004,201,5.0,5bbf259ff136
236 0.106,201,4.8,7cfd91ddf780
237 0.217,201,5.2,5bbf259ff136
238 0.323,201,6.3,5bbf259ff136
239 0.44,201,5.2,7cfd91ddf780
240 0.547,201,5.5,7cfd91ddf780
241 0.657,201,3.2,7cfd91ddf780
242 0.771,201,2.8,7cfd91ddf780
243 0.879,201,3.9,7cfd91ddf780
244 0.993,201,3.9,7cfd91ddf780
245 1.107,201,4.7,5bbf259ff136
246 1.222,201,4.9,5bbf259ff136
247 1.333,201,3.9,5bbf259ff136
248 1.439,201,6.0,5bbf259ff136
249 1.551,201,6.0,7cfd91ddf780
250 1.66,201,6.1,7cfd91ddf780
251 1.775,201,5.1,7cfd91ddf780
252 1.887,201,7.0,7cfd91ddf780
253 2.004,201,5.9,7cfd91ddf780
254 2.12,201,5.4,7cfd91ddf780
255 2.233,201,4.1,7cfd91ddf780
256 2.338,201,3.8,7cfd91ddf780
257 2.449,201,6.8,7cfd91ddf780
258 2.557,201,4.3,7cfd91ddf780
259 2.661,201,6.9,7cfd91ddf780
260 2.768,201,4.5,7cfd91ddf780
261 2.876,201,4.2,7cfd91ddf780
262 2.983,201,4.6,7cfd91ddf780
263 3.098,201,4.4,7cfd91ddf780
264 3.203,201,5.2,7cfd91ddf780
265 3.31,201,5.2,7cfd91ddf780
266 3.425,201,5.5,7cfd91ddf780
267 3.537,201,5.9,7cfd91ddf780
268 3.646,201,8.7,7cfd91ddf780
269 3.758,201,5.7,7cfd91ddf780
270 3.867,201,7.2,7cfd91ddf780
271 3.982,201,6.7,7cfd91ddf780
272 4.099,201,5.7,7cfd91ddf780
273 4.206,201,4.6,7cfd91ddf780
274 4.316,201,13.6,7cfd91ddf780
275 4.439,201,4.2,7cfd91ddf780
276 4.554,201,5.9,7cfd91ddf780
277 4.665,201,7.6,7cfd91ddf780
278 4.781,201,4.8,7cfd91ddf780
279 4.896,201,5.9,7cfd91ddf780
280 5.008,201,5.5,7cfd91ddf780
281 5.123,201,4.3,7cfd91ddf780
282 5.233,201,4.4,7cfd91ddf780
283 5.338,201,5.1,7cfd91ddf780
284 5.449,201,5.8,7cfd91ddf780
285 5.557,201,5.6,7cfd91ddf780
286 5.673,201,6.9,7cfd91ddf780
287 5.781,201,6.3,7cfd91ddf780
288 5.887,201,6.8,7cfd91ddf780
289 6.004,201,5.0,7cfd91ddf780
290 6.118,201,5.1,7cfd91ddf780
291 6.232,201,5.5,7cfd91ddf780
292 6.347,201,5.2,7cfd91ddf780
293 6.454,201,5.8,7cfd91ddf780
294 6.564,201,5.1,7cfd91ddf780
295 6.674,201,4.4,7cfd91ddf780
296 6.783,201,5.7,7cfd91ddf780
297 6.892,201,3.0,7cfd91ddf780
298 7.004,201,6.7,7cfd91ddf780
299 7.119,201,6.2,7cfd91ddf780
300 7.235,201,5.3,7cfd91ddf780
301 7.351,201,6.6,7cfd91ddf780
302 7.458,201,6.4,7cfd91ddf780
303 7.568,201,5.2,7cfd91ddf780
304 7.677,201,7.3,7cfd91ddf780
305 7.793,201,5.2,7cfd91ddf780
306 7.899,201,3.3,7cfd91ddf780
307 8.012,201,2.8,7cfd91ddf780
308 8.124,201,3.9,7cfd91ddf780
309 8.232,201,4.9,7cfd91ddf780
310 8.337,201,6.0,7cfd91ddf780
311 8.449,201,5.9,7cfd91ddf780
312 8.563,201,5.8,7cfd91ddf780
313 8.674,201,7.4,7cfd91ddf780
314 8.782,201,4.3,7cfd91ddf780
315 8.894,201,6.0,7cfd91ddf780
316 9.01,201,4.9,7cfd91ddf780
317 9.122,201,5.6,7cfd91ddf780
318 9.233,201,6.0,7cfd91ddf780
319 9.339,201,5.2,7cfd91ddf780
320 9.449,201,6.0,7cfd91ddf780
321 9.556,201,6.4,7cfd91ddf780
322 9.673,201,5.0,7cfd91ddf780
323 9.788,201,5.0,7cfd91ddf780
324 9.899,201,4.8,7cfd91ddf780
325 10.009,201,5.2,7cfd91ddf780
326 10.12,201,5.8,7cfd91ddf780
327 10.233,201,4.7,7cfd91ddf780
328 10.348,201,4.6,7cfd91ddf780
329 10.453,201,4.3,7cfd91ddf780
330 10.566,201,5.3,7cfd91ddf780
331 10.681,201,5.9,7cfd91ddf780
332 10.789,201,7.5,7cfd91ddf780
333 10.899,201,6.3,7cfd91ddf780
334 11.012,201,6.7,7cfd91ddf780
335 11.122,201,4.8,7cfd91ddf780
336 11.228,201,5.5,7cfd91ddf780
337 11.343,201,4.9,7cfd91ddf780
338 11.458,201,5.8,7cfd91ddf780
339 11.573,201,5.4,7cfd91ddf780
340 11.688,201,6.7,7cfd91ddf780
341 11.805,201,5.5,7cfd91ddf780
342 11.916,201,6.0,7cfd91ddf780
343 12.023,201,5.8,7cfd91ddf780
344 12.137,201,6.0,7cfd91ddf780
345 12.244,201,4.7,7cfd91ddf780
346 12.355,201,6.5,7cfd91ddf780
347 12.466,201,5.1,7cfd91ddf780
348 12.578,201,5.1,7cfd91ddf780
349 12.69,201,5.4,7cfd91ddf780
350 12.8,201,5.2,7cfd91ddf780
351 12.915,201,4.0,7cfd91ddf780
352 13.026,201,4.9,7cfd91ddf780
353 13.138,201,3.7,7cfd91ddf780
354 13.242,201,4.3,7cfd91ddf780
355 13.351,201,3.3,7cfd91ddf780
356 13.455,201,3.9,7cfd91ddf780
357 13.569,201,3.5,7cfd91ddf780
358 13.676,201,3.9,7cfd91ddf780
359 13.782,201,3.1,7cfd91ddf780
360 13.89,201,3.8,7cfd91ddf780
361 13.999,201,4.0,7cfd91ddf780
362 14.103,201,4.8,7cfd91ddf780
363 14.21,201,4.8,7cfd91ddf780
364 14.318,201,5.0,7cfd91ddf780
365 14.43,201,5.3,7cfd91ddf780
366 14.543,201,5.1,7cfd91ddf780
367 14.65,201,5.2,7cfd91ddf780
368 14.756,201,4.8,7cfd91ddf780
369 14.863,201,5.7,7cfd91ddf780
370 14.979,201,5.9,7cfd91ddf780
371 15.093,201,5.5,7cfd91ddf780
372 15.203,201,5.1,7cfd91ddf780
373 15.309,201,4.9,7cfd91ddf780
374 15.416,201,8.0,7cfd91ddf780
375 15.534,201,6.0,7cfd91ddf780
376 15.642,201,3.5,7cfd91ddf780
377 15.756,201,4.7,5bbf259ff136
378 15.871,201,6.0,5bbf259ff136
379 15.987,201,4.7,7cfd91ddf780
380 16.097,201,4.9,7cfd91ddf780
381 16.212,201,5.2,7cfd91ddf780
382 16.324,201,6.2,7cfd91ddf780
383 16.44,201,5.5,5bbf259ff136
384 16.556,201,4.9,7cfd91ddf780
385 16.666,201,5.2,7cfd91ddf780
386 16.777,201,6.3,7cfd91ddf780
387 16.893,201,5.8,7cfd91ddf780
388 17.009,201,5.4,5bbf259ff136
389 17.123,201,5.5,5bbf259ff136
390 17.229,201,5.4,5bbf259ff136
391 17.343,201,5.1,5bbf259ff136
392 17.457,201,5.4,7cfd91ddf780
393 17.564,201,5.2,5bbf259ff136
394 17.679,201,7.2,7cfd91ddf780
395 17.79,201,6.8,7cfd91ddf780
396 17.899,201,5.0,7cfd91ddf780
397 18.013,201,5.1,7cfd91ddf780
398 18.126,201,8.5,5bbf259ff136
399 18.241,201,3.9,5bbf259ff136
400 18.352,201,4.5,5bbf259ff136
401 18.465,201,5.5,7cfd91ddf780
402 18.577,201,5.9,7cfd91ddf780
403 18.691,201,4.8,7cfd91ddf780
404 18.806,201,14.2,5bbf259ff136
405 18.92,201,5.3,5bbf259ff136
406 19.03,201,4.6,5bbf259ff136
407 19.141,201,5.5,5bbf259ff136
408 19.249,201,5.7,5bbf259ff136
409 19.357,201,4.8,5bbf259ff136
410 19.468,201,5.1,5bbf259ff136
411 19.582,201,6.6,5bbf259ff136
412 19.695,201,5.3,7cfd91ddf780
413 19.802,201,5.1,5bbf259ff136
414 19.912,201,4.9,5bbf259ff136
415 20.021,201,5.2,5bbf259ff136
416 20.136,201,5.4,7cfd91ddf780
417 20.251,201,4.2,7cfd91ddf780
418 20.361,201,5.0,7cfd91ddf780
419 20.473,201,5.2,7cfd91ddf780
420 20.582,201,5.0,7cfd91ddf780
421 20.692,201,5.9,7cfd91ddf780
422 20.805,201,4.7,7cfd91ddf780
423 20.912,201,4.5,7cfd91ddf780
424 21.023,201,4.7,7cfd91ddf780
425 21.137,201,5.2,5bbf259ff136
426 21.249,201,4.4,5bbf259ff136
427 21.362,201,5.1,5bbf259ff136
428 21.472,201,5.0,5bbf259ff136
429 21.583,201,5.6,5bbf259ff136
430 21.691,201,6.1,5bbf259ff136
431 21.8,201,7.2,5bbf259ff136
432 21.917,201,6.8,7cfd91ddf780
433 22.03,201,5.0,7cfd91ddf780
434 22.137,201,6.5,7cfd91ddf780
435 22.253,201,7.0,7cfd91ddf780
436 22.364,201,5.3,7cfd91ddf780
437 22.474,201,5.3,7cfd91ddf780
438 22.582,201,5.7,7cfd91ddf780
439 22.691,201,5.6,7cfd91ddf780
440 22.801,201,4.9,7cfd91ddf780
441 22.916,201,4.4,5bbf259ff136
442 23.02,201,4.9,7cfd91ddf780
443 23.135,201,5.3,5bbf259ff136
444 23.242,201,6.3,7cfd91ddf780
445 23.358,201,4.7,5bbf259ff136
446 23.467,201,4.5,5bbf259ff136
447 23.572,201,4.1,7cfd91ddf780
448 23.682,201,5.6,7cfd91ddf780
449 23.79,201,4.8,7cfd91ddf780
450 23.905,201,5.1,5bbf259ff136
451 24.015,201,5.0,5bbf259ff136
452 24.122,201,4.7,7cfd91ddf780
453 24.233,201,4.4,7cfd91ddf780
454 24.342,201,5.8,7cfd91ddf780
455 24.458,201,6.0,5bbf259ff136
456 24.569,201,5.4,5bbf259ff136
457 24.684,201,6.7,7cfd91ddf780
458 24.801,201,5.3,7cfd91ddf780
459 24.916,201,5.2,5bbf259ff136
460 25.023,201,6.6,7cfd91ddf780
461 25.14,201,5.7,5bbf259ff136
462 25.251,201,4.5,5bbf259ff136
463 25.357,201,5.0,5bbf259ff136
464 25.472,201,5.1,5bbf259ff136
465 25.582,201,4.8,5bbf259ff136
466 25.691,201,6.0,5bbf259ff136
467 25.807,201,5.2,7cfd91ddf780
468 25.915,201,4.9,7cfd91ddf780
469 26.03,201,5.1,7cfd91ddf780
470 26.14,201,5.1,7cfd91ddf780
471 26.247,201,5.6,7cfd91ddf780
472 26.354,201,5.3,5bbf259ff136
473 26.467,201,6.8,7cfd91ddf780
474 26.583,201,5.2,7cfd91ddf780
475 26.694,201,5.2,7cfd91ddf780
476 26.806,201,6.2,5bbf259ff136
477 26.914,201,6.5,5bbf259ff136
478 27.031,201,5.9,5bbf259ff136
479 27.144,201,5.4,7cfd91ddf780
480 27.255,201,5.1,7cfd91ddf780
481 27.362,201,6.0,5bbf259ff136
482 27.478,201,6.3,7cfd91ddf780
483 27.59,201,6.3,7cfd91ddf780
484 27.707,201,5.4,5bbf259ff136
485 27.814,201,7.4,5bbf259ff136
486 27.929,201,3.9,5bbf259ff136
487 28.04,201,4.0,5bbf259ff136
488 28.153,201,5.2,5bbf259ff136
489 28.269,201,4.8,7cfd91ddf780
490 28.381,201,5.6,7cfd91ddf780
491 28.487,201,5.1,7cfd91ddf780
492 28.597,201,5.1,7cfd91ddf780
493 28.708,201,3.9,7cfd91ddf780
494 28.816,201,5.1,7cfd91ddf780
495 28.931,201,4.9,7cfd91ddf780
496 29.046,201,5.9,5bbf259ff136
497 29.162,201,4.9,5bbf259ff136
498 29.276,201,3.8,7cfd91ddf780
499 29.381,201,5.3,5bbf259ff136
500 29.489,201,4.7,5bbf259ff136
501 29.6,201,4.9,5bbf259ff136
502 29.715,201,5.9,7cfd91ddf780
503 29.825,201,5.5,7cfd91ddf780
504 29.94,201,5.9,7cfd91ddf780
505 30.047,201,7.0,7cfd91ddf780
506 30.155,201,5.5,7cfd91ddf780
507 30.271,201,6.1,7cfd91ddf780
508 30.38,201,4.9,7cfd91ddf780
509 30.49,201,5.3,7cfd91ddf780
510 30.597,201,7.2,7cfd91ddf780
511 30.714,201,6.6,5bbf259ff136
512 30.822,201,6.9,7cfd91ddf780
513 30.933,201,6.4,7cfd91ddf780
514 31.049,201,3.8,5bbf259ff136
515 31.163,201,4.9,5bbf259ff136
516 31.273,201,5.4,5bbf259ff136
517 31.383,201,5.1,5bbf259ff136
518 31.49,201,5.0,5bbf259ff136
519 31.605,201,5.9,7cfd91ddf780
520 31.717,201,6.5,7cfd91ddf780
521 31.825,201,5.7,7cfd91ddf780
522 31.938,201,5.0,7cfd91ddf780
523 32.053,201,4.8,7cfd91ddf780
524 32.164,201,4.9,7cfd91ddf780
525 32.271,201,4.6,7cfd91ddf780
526 32.378,201,4.7,7cfd91ddf780
527 32.49,201,5.0,7cfd91ddf780
528 32.605,201,4.2,7cfd91ddf780
529 32.717,201,4.1,5bbf259ff136
530 32.828,201,5.4,5bbf259ff136
531 32.937,201,6.0,5bbf259ff136
532 33.048,201,6.2,5bbf259ff136
533 33.163,201,5.3,5bbf259ff136
534 33.276,201,6.7,7cfd91ddf780
535 33.39,201,6.6,7cfd91ddf780
536 33.501,201,5.0,7cfd91ddf780
537 33.616,201,5.3,5bbf259ff136
538 33.722,201,4.9,7cfd91ddf780
539 33.831,201,3.9,7cfd91ddf780
540 33.945,201,5.9,5bbf259ff136
541 34.06,201,5.4,5bbf259ff136
542 34.168,201,5.5,5bbf259ff136
543 34.28,201,5.6,5bbf259ff136
544 34.392,201,3.1,5bbf259ff136
545 34.505,201,5.5,7cfd91ddf780
546 34.621,201,4.7,7cfd91ddf780
547 34.736,201,5.2,5bbf259ff136
548 34.851,201,6.5,5bbf259ff136
549 34.965,201,5.7,7cfd91ddf780
550 35.072,201,4.9,5bbf259ff136
551 35.182,201,6.6,5bbf259ff136
552 35.291,201,4.8,5bbf259ff136
553 35.403,201,6.7,5bbf259ff136
554 35.518,201,6.8,7cfd91ddf780
555 35.627,201,4.7,7cfd91ddf780
556 35.732,201,6.7,5bbf259ff136
557 35.848,201,5.8,7cfd91ddf780
558 35.962,201,6.7,7cfd91ddf780
559 36.073,201,5.2,7cfd91ddf780
560 36.182,201,6.8,7cfd91ddf780
561 36.29,201,6.2,7cfd91ddf780
562 36.405,201,5.0,5bbf259ff136
563 36.52,201,5.0,5bbf259ff136
564 36.634,201,5.0,5bbf259ff136
565 36.747,201,4.6,7cfd91ddf780
566 36.861,201,5.0,7cfd91ddf780
567 36.974,201,5.6,5bbf259ff136
568 37.088,201,4.8,5bbf259ff136
569 37.203,201,5.6,5bbf259ff136
570 37.314,201,5.5,7cfd91ddf780
571 37.43,201,5.7,7cfd91ddf780
572 37.545,201,4.8,7cfd91ddf780
573 37.659,201,4.3,5bbf259ff136
574 37.765,201,5.4,5bbf259ff136
575 37.873,201,5.3,7cfd91ddf780
576 37.981,201,6.1,7cfd91ddf780
577 38.097,201,6.0,5bbf259ff136
578 38.214,201,6.3,7cfd91ddf780
579 38.327,201,5.5,7cfd91ddf780
580 38.44,201,6.0,7cfd91ddf780
581 38.548,201,5.4,7cfd91ddf780
582 38.656,201,6.9,7cfd91ddf780
583 38.765,201,3.5,5bbf259ff136
584 38.868,201,3.5,5bbf259ff136
585 38.981,201,3.7,5bbf259ff136
586 39.088,201,3.1,5bbf259ff136
587 39.2,201,3.6,5bbf259ff136
588 39.308,201,4.0,5bbf259ff136
589 39.414,201,4.3,7cfd91ddf780
590 39.523,201,3.5,7cfd91ddf780
591 39.635,201,4.3,5bbf259ff136
592 39.744,201,4.2,5bbf259ff136
593 39.859,201,4.0,5bbf259ff136
594 39.965,201,3.4,5bbf259ff136
595 40.076,201,7.7,5bbf259ff136
596 40.192,201,6.5,5bbf259ff136
597 40.306,201,5.0,7cfd91ddf780
598 40.414,201,3.5,5bbf259ff136
599 40.518,201,2.6,5bbf259ff136
600 40.623,201,2.6,7cfd91ddf780
601 40.733,201,4.6,7cfd91ddf780
602 40.845,201,4.6,5bbf259ff136
603 40.958,201,4.2,5bbf259ff136
604 41.064,201,4.2,7cfd91ddf780
605 41.171,201,2.8,7cfd91ddf780
606 41.281,201,4.9,7cfd91ddf780
607 41.388,201,4.3,7cfd91ddf780
608 41.497,201,5.0,7cfd91ddf780
609 41.606,201,5.1,7cfd91ddf780
610 41.721,201,2.9,7cfd91ddf780
611 41.829,201,4.8,7cfd91ddf780
612 41.942,201,3.0,7cfd91ddf780
613 42.053,201,5.6,7cfd91ddf780
614 42.164,201,6.5,7cfd91ddf780
615 42.281,201,5.6,5bbf259ff136
616 42.389,201,4.5,5bbf259ff136
617 42.503,201,4.6,5bbf259ff136
618 42.618,201,5.8,7cfd91ddf780
619 42.734,201,5.2,7cfd91ddf780
620 42.849,201,5.9,5bbf259ff136
621 42.959,201,5.0,5bbf259ff136
622 43.066,201,4.9,5bbf259ff136
623 43.181,201,6.2,5bbf259ff136
624 43.289,201,5.7,5bbf259ff136
625 43.4,201,6.0,5bbf259ff136
626 43.516,201,7.1,7cfd91ddf780
627 43.631,201,6.1,7cfd91ddf780
628 43.746,201,6.7,5bbf259ff136
629 43.862,201,5.9,5bbf259ff136
630 43.969,201,5.5,5bbf259ff136
631 44.084,201,5.4,7cfd91ddf780
632 44.2,201,4.6,7cfd91ddf780
633 44.305,201,5.8,7cfd91ddf780
634 44.421,201,6.1,7cfd91ddf780
635 44.53,201,5.2,7cfd91ddf780
636 44.637,201,3.7,7cfd91ddf780
637 44.75,201,3.3,7cfd91ddf780
638 44.855,201,3.8,5bbf259ff136
639 44.961,201,4.7,5bbf259ff136
640 45.072,201,6.1,7cfd91ddf780
641 45.184,201,7.3,5bbf259ff136
642 45.296,201,7.5,7cfd91ddf780
643 45.409,201,5.2,5bbf259ff136
644 45.521,201,5.5,7cfd91ddf780
@@ -0,0 +1,71 @@
{"date": "2026-11-05T19:18:13.470607+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"label": "5bbf259ff136",
"action": [
"sh",
"-c",
"scripts/stop-one.sh && docker compose ps -a --format '{{.ID}} {{.Service}} {{.Status}}' api && { scripts/stand-usage.sh & sleep 10; wait; } && scripts/restore.sh"
],
"action_exit": 0,
"action_seconds": 15.2,
"action_finished_before_load_end": true,
"lb_up_before": 2,
"lb_up_min": 1,
"lb_samples_missing": 0,
"detection_seconds": 1.498,
"rejoin_seconds": 14.99,
"probe": {
"attempts": 642,
"failed_attempts": 0,
"first_failure_s": null,
"first_success_s": 0.111,
"recovery_s": 0.0,
"instances_before": {
"7cfd91ddf780": 114,
"5bbf259ff136": 119
},
"instances_during_outage": {
"7cfd91ddf780": 121
},
"instances_after": {
"7cfd91ddf780": 149,
"5bbf259ff136": 124
}
},
"history": {
"confirmed": 642,
"stored": 642,
"exact_match": true,
"unique": true
},
"lb_counters": {
"wretr": 0,
"wredis": 0,
"econ": 0,
"eresp": 0,
"hrsp_5xx": 0
},
"k6": {
"requests": 6001,
"rps": 100.02,
"ok_rps": 100.02,
"failed_ratio": 0,
"dropped_iterations": 0,
"p50_ms": 2.98,
"p95_ms": 3.88,
"p99_ms": 4.53,
"exit": 0,
"rate": 100
},
"checks": {
"action_ok": true,
"load_errors_ok": true,
"load_delivered": true,
"lb_observed": true,
"write_recovery_ok": true,
"history_preserved": true,
"detected": true,
"target_excluded": true,
"rejoined": true
},
"pass": true
}
@@ -0,0 +1,15 @@
date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
7cfd91ddf780 api Exited (0) Less than a second ago
5bbf259ff136 api Up 3 minutes (healthy)
19:18:43 database bytes,messages: 9385651,2542
mtusi2026lab3-grade4-20261105-api-2 cpu=23.33% mem=217.6MiB / 512MiB
mtusi2026lab3-grade4-20261105-lb-1 cpu=3.14% mem=30.17MiB / 128MiB
mtusi2026lab3-grade4-20261105-postgres-1 cpu=6.03% mem=58.73MiB / 1GiB
mtusi2026lab3-grade4-20261105-valkey-1 cpu=0.42% mem=12.97MiB / 128MiB
Container mtusi2026lab3-grade4-20261105-api-2 Running
Container mtusi2026lab3-grade4-20261105-api-1 Starting
Container mtusi2026lab3-grade4-20261105-api-1 Started
Container mtusi2026lab3-grade4-20261105-api-2 Waiting
Container mtusi2026lab3-grade4-20261105-api-1 Waiting
Container mtusi2026lab3-grade4-20261105-api-2 Healthy
Container mtusi2026lab3-grade4-20261105-api-1 Healthy
@@ -0,0 +1,3 @@
date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
time="2026-11-05T19:18:15Z" level=info msg=LOAD-START source=console
measure: 6001 requests, failed ratio 0
@@ -0,0 +1,309 @@
{"date": "2026-11-05T19:18:13.470607+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"root_group": {
"checks": [
{
"name": "message 201",
"path": "::message 201",
"id": "6445aff337e1ade0bc10402fbac79882",
"passes": 5614,
"fails": 0
},
{
"name": "history 200",
"path": "::history 200",
"id": "fe27b3b8edd292c7f2170a6357b087d0",
"passes": 1388,
"fails": 0
}
],
"name": "",
"path": "",
"id": "d41d8cd98f00b204e9800998ecf8427e",
"groups": []
},
"options": {
"summaryTrendStats": [
"avg",
"min",
"med",
"p(90)",
"p(95)",
"p(99)",
"max"
],
"summaryTimeUnit": "",
"noColor": false
},
"state": {
"isStdOutTTY": false,
"isStdErrTTY": false,
"testRunDurationMs": 70538.919963
},
"metrics": {
"http_req_tls_handshaking": {
"type": "trend",
"contains": "time",
"values": {
"min": 0,
"med": 0,
"p(90)": 0,
"p(95)": 0,
"p(99)": 0,
"max": 0,
"avg": 0
}
},
"http_req_failed": {
"type": "rate",
"contains": "default",
"values": {
"rate": 0,
"passes": 0,
"fails": 7052
}
},
"http_req_duration": {
"type": "trend",
"contains": "time",
"values": {
"p(95)": 3.943366299999998,
"p(99)": 7.941836829999957,
"max": 52.49775,
"avg": 3.1087168380601247,
"min": 1.647417,
"med": 2.9766874999999997,
"p(90)": 3.6688584
}
},
"checks": {
"type": "rate",
"contains": "default",
"values": {
"rate": 1,
"passes": 7002,
"fails": 0
}
},
"iterations{scenario:measure}": {
"type": "counter",
"contains": "default",
"values": {
"count": 6001,
"rate": 85.07360196537917
}
},
"http_req_duration{expected_response:true}": {
"type": "trend",
"contains": "time",
"values": {
"avg": 3.1087168380601247,
"min": 1.647417,
"med": 2.9766874999999997,
"p(90)": 3.6688584,
"p(95)": 3.943366299999998,
"p(99)": 7.941836829999957,
"max": 52.49775
}
},
"http_req_sending": {
"contains": "time",
"values": {
"max": 0.354375,
"avg": 0.01261951956891675,
"min": 0.00275,
"med": 0.011459,
"p(90)": 0.015125,
"p(95)": 0.01775,
"p(99)": 0.033123339999999994
},
"type": "trend"
},
"http_req_receiving": {
"type": "trend",
"contains": "time",
"values": {
"avg": 0.022591304452637576,
"min": 0.008958,
"med": 0.019917,
"p(90)": 0.032,
"p(95)": 0.03826844999999999,
"p(99)": 0.06532848999999998,
"max": 0.210875
}
},
"data_sent": {
"values": {
"count": 2053317,
"rate": 29108.994028786277
},
"type": "counter",
"contains": "data"
},
"data_received": {
"values": {
"count": 17042097,
"rate": 241598.4963894988
},
"type": "counter",
"contains": "data"
},
"http_req_duration{scenario:measure}": {
"type": "trend",
"contains": "time",
"values": {
"p(95)": 3.862542,
"p(99)": 4.473916,
"max": 52.49775,
"avg": 3.049362646058984,
"min": 1.684875,
"med": 2.9745,
"p(90)": 3.631833
}
},
"http_req_connecting": {
"type": "trend",
"contains": "time",
"values": {
"max": 0.224708,
"avg": 0.00034415215541690295,
"min": 0,
"med": 0,
"p(90)": 0,
"p(95)": 0,
"p(99)": 0
}
},
"vus": {
"type": "gauge",
"contains": "default",
"values": {
"value": 1,
"min": 0,
"max": 1
}
},
"http_req_failed{scenario:measure}": {
"values": {
"fails": 6001,
"rate": 0,
"passes": 0
},
"type": "rate",
"contains": "default"
},
"dropped_iterations{scenario:measure}": {
"type": "counter",
"contains": "default",
"values": {
"count": 0,
"rate": 0
}
},
"http_req_duration{scenario:measure,name:post_message}": {
"type": "trend",
"contains": "time",
"values": {
"med": 2.940125,
"p(90)": 3.551959,
"p(95)": 3.786104,
"p(99)": 4.346612499999998,
"max": 52.49775,
"avg": 3.0002135637081597,
"min": 1.684875
}
},
"http_req_waiting": {
"values": {
"max": 52.465792,
"avg": 3.073506014038576,
"min": 1.628417,
"med": 2.941875,
"p(90)": 3.6291869,
"p(95)": 3.8983437499999996,
"p(99)": 7.918076749999957
},
"type": "trend",
"contains": "time"
},
"http_reqs{scenario:measure}": {
"type": "counter",
"contains": "default",
"values": {
"count": 6001,
"rate": 85.07360196537917
}
},
"dropped_iterations": {
"type": "counter",
"contains": "default",
"values": {
"count": 0,
"rate": 0
}
},
"iterations": {
"type": "counter",
"contains": "default",
"values": {
"count": 7002,
"rate": 99.26434943535826
}
},
"http_req_blocked": {
"type": "trend",
"contains": "time",
"values": {
"min": 0.000708,
"med": 0.001667,
"p(90)": 0.0025,
"p(95)": 0.002959,
"p(99)": 0.006083,
"max": 1.817417,
"avg": 0.0025302468803176624
}
},
"vus_max": {
"type": "gauge",
"contains": "default",
"values": {
"min": 20,
"max": 20,
"value": 20
}
},
"http_reqs": {
"type": "counter",
"contains": "default",
"values": {
"count": 7052,
"rate": 99.97317798031226
}
},
"iteration_duration": {
"values": {
"avg": 3.1430217764924397,
"min": 1.686459,
"med": 3.0623125,
"p(90)": 3.7471503,
"p(95)": 4.001064599999999,
"p(99)": 4.828713,
"max": 52.592501
},
"type": "trend",
"contains": "time"
},
"http_req_duration{scenario:measure,name:get_history}": {
"type": "trend",
"contains": "time",
"values": {
"avg": 3.2480653647058872,
"min": 1.856375,
"med": 3.166271,
"p(90)": 3.8746452,
"p(95)": 4.109600299999999,
"p(99)": 5.1775943700000004,
"max": 17.655583
}
}
}
}
@@ -0,0 +1,663 @@
# date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
t_rel_s,up
-25.852,2
-25.745,2
-25.643,2
-25.532,2
-25.426,2
-25.319,2
-25.21,2
-25.1,2
-24.995,2
-24.894,2
-24.79,2
-24.686,2
-24.577,2
-24.469,2
-24.358,2
-24.255,2
-24.148,2
-24.042,2
-23.939,2
-23.836,2
-23.727,2
-23.625,2
-23.521,2
-23.412,2
-23.303,2
-23.192,2
-23.088,2
-22.978,2
-22.868,2
-22.757,2
-22.648,2
-22.541,2
-22.44,2
-22.33,2
-22.227,2
-22.121,2
-22.019,2
-21.908,2
-21.804,2
-21.699,2
-21.588,2
-21.476,2
-21.363,2
-21.259,2
-21.148,2
-21.036,2
-20.928,2
-20.82,2
-20.709,2
-20.604,2
-20.493,2
-20.381,2
-20.269,2
-20.158,2
-20.053,2
-19.944,2
-19.837,2
-19.729,2
-19.619,2
-19.511,2
-19.406,2
-19.304,2
-19.191,2
-19.083,2
-18.972,2
-18.869,2
-18.761,2
-18.649,2
-18.542,2
-18.43,2
-18.318,2
-18.216,2
-18.114,2
-18.005,2
-17.899,2
-17.792,2
-17.685,2
-17.581,2
-17.472,2
-17.359,2
-17.253,2
-17.148,2
-17.043,2
-16.936,2
-16.823,2
-16.712,2
-16.602,2
-16.495,2
-16.386,2
-16.276,2
-16.164,2
-16.061,2
-15.957,2
-15.846,2
-15.736,2
-15.632,2
-15.526,2
-15.422,2
-15.311,2
-15.205,2
-15.101,2
-14.991,2
-14.879,2
-14.768,2
-14.661,2
-14.554,2
-14.444,2
-14.335,2
-14.228,2
-14.123,2
-14.018,2
-13.906,2
-13.792,2
-13.68,2
-13.574,2
-13.465,2
-13.358,2
-13.247,2
-13.141,2
-13.029,2
-12.922,2
-12.809,2
-12.697,2
-12.586,2
-12.474,2
-12.37,2
-12.26,2
-12.154,2
-12.041,2
-11.935,2
-11.824,2
-11.719,2
-11.608,2
-11.506,2
-11.397,2
-11.283,2
-11.176,2
-11.07,2
-10.962,2
-10.851,2
-10.741,2
-10.629,2
-10.519,2
-10.408,2
-10.296,2
-10.185,2
-10.078,2
-9.975,2
-9.87,2
-9.76,2
-9.653,2
-9.54,2
-9.43,2
-9.319,2
-9.211,2
-9.102,2
-8.989,2
-8.876,2
-8.765,2
-8.656,2
-8.546,2
-8.437,2
-8.328,2
-8.22,2
-8.106,2
-7.995,2
-7.886,2
-7.782,2
-7.67,2
-7.563,2
-7.452,2
-7.345,2
-7.236,2
-7.13,2
-7.018,2
-6.911,2
-6.798,2
-6.686,2
-6.579,2
-6.471,2
-6.367,2
-6.256,2
-6.149,2
-6.042,2
-5.934,2
-5.828,2
-5.72,2
-5.616,2
-5.503,2
-5.395,2
-5.286,2
-5.178,2
-5.067,2
-4.961,2
-4.855,2
-4.742,2
-4.635,2
-4.526,2
-4.418,2
-4.304,2
-4.195,2
-4.083,2
-3.971,2
-3.86,2
-3.755,2
-3.641,2
-3.532,2
-3.428,2
-3.32,2
-3.209,2
-3.098,2
-2.994,2
-2.882,2
-2.777,2
-2.665,2
-2.553,2
-2.446,2
-2.337,2
-2.228,2
-2.124,2
-2.018,2
-1.908,2
-1.803,2
-1.693,2
-1.587,2
-1.476,2
-1.37,2
-1.261,2
-1.149,2
-1.037,2
-0.929,2
-0.816,2
-0.713,2
-0.601,2
-0.495,2
-0.387,2
-0.277,2
-0.17,2
-0.066,2
0.036,2
0.142,2
0.253,2
0.358,2
0.461,2
0.571,2
0.675,2
0.78,2
0.884,1
0.992,1
1.098,1
1.205,1
1.314,1
1.426,1
1.538,1
1.645,1
1.755,1
1.864,1
1.972,1
2.078,1
2.188,1
2.302,1
2.411,1
2.522,1
2.626,1
2.737,1
2.838,1
2.941,1
3.049,1
3.156,1
3.268,1
3.377,1
3.49,1
3.604,1
3.716,1
3.825,1
3.937,1
4.044,1
4.154,1
4.259,1
4.369,1
4.479,1
4.588,1
4.693,1
4.8,1
4.911,1
5.018,1
5.127,1
5.239,1
5.35,1
5.457,1
5.563,1
5.664,1
5.767,1
5.871,1
5.976,1
6.086,1
6.198,1
6.306,1
6.413,1
6.525,1
6.638,1
6.747,1
6.859,1
6.97,1
7.081,1
7.189,1
7.302,1
7.411,1
7.518,1
7.631,1
7.74,1
7.847,1
7.96,1
8.071,1
8.184,1
8.288,1
8.396,1
8.501,1
8.605,1
8.711,1
8.825,1
8.93,1
9.034,1
9.147,1
9.259,1
9.367,1
9.48,1
9.584,1
9.687,1
9.792,1
9.904,1
10.011,1
10.115,1
10.225,1
10.327,1
10.429,1
10.539,1
10.64,1
10.752,1
10.862,1
10.967,1
11.073,1
11.185,1
11.297,1
11.411,1
11.521,1
11.631,1
11.737,1
11.85,1
11.955,1
12.063,1
12.175,1
12.288,1
12.397,1
12.502,1
12.616,1
12.72,1
12.83,1
12.934,1
13.042,1
13.151,1
13.264,1
13.371,1
13.485,1
13.587,1
13.696,1
13.807,1
13.912,1
14.024,1
14.136,1
14.237,1
14.346,1
14.46,1
14.571,1
14.678,1
14.783,1
14.885,2
14.996,2
15.101,2
15.203,2
15.309,2
15.423,2
15.533,2
15.647,2
15.759,2
15.87,2
15.976,2
16.088,2
16.196,2
16.305,2
16.411,2
16.523,2
16.629,2
16.742,2
16.854,2
16.962,2
17.067,2
17.181,2
17.288,2
17.397,2
17.509,2
17.621,2
17.733,2
17.84,2
17.946,2
18.054,2
18.163,2
18.267,2
18.379,2
18.492,2
18.606,2
18.71,2
18.819,2
18.93,2
19.038,2
19.14,2
19.242,2
19.346,2
19.455,2
19.565,2
19.672,2
19.786,2
19.891,2
20.0,2
20.103,2
20.211,2
20.315,2
20.42,2
20.529,2
20.64,2
20.747,2
20.854,2
20.965,2
21.073,2
21.181,2
21.286,2
21.396,2
21.505,2
21.618,2
21.73,2
21.843,2
21.948,2
22.055,2
22.16,2
22.273,2
22.385,2
22.496,2
22.602,2
22.712,2
22.82,2
22.93,2
23.042,2
23.146,2
23.253,2
23.36,2
23.469,2
23.581,2
23.69,2
23.803,2
23.917,2
24.025,2
24.13,2
24.236,2
24.348,2
24.455,2
24.567,2
24.678,2
24.786,2
24.9,2
25.013,2
25.126,2
25.241,2
25.353,2
25.463,2
25.574,2
25.686,2
25.796,2
25.908,2
26.017,2
26.129,2
26.239,2
26.35,2
26.454,2
26.563,2
26.672,2
26.784,2
26.889,2
26.994,2
27.103,2
27.212,2
27.325,2
27.437,2
27.545,2
27.658,2
27.762,2
27.874,2
27.978,2
28.085,2
28.192,2
28.301,2
28.412,2
28.524,2
28.638,2
28.749,2
28.856,2
28.966,2
29.072,2
29.182,2
29.29,2
29.394,2
29.499,2
29.607,2
29.711,2
29.821,2
29.929,2
30.035,2
30.144,2
30.255,2
30.367,2
30.472,2
30.585,2
30.691,2
30.799,2
30.911,2
31.021,2
31.13,2
31.238,2
31.352,2
31.462,2
31.576,2
31.69,2
31.797,2
31.909,2
32.021,2
32.129,2
32.233,2
32.337,2
32.444,2
32.555,2
32.658,2
32.769,2
32.881,2
32.987,2
33.095,2
33.209,2
33.317,2
33.42,2
33.532,2
33.64,2
33.751,2
33.86,2
33.963,2
34.074,2
34.179,2
34.282,2
34.387,2
34.495,2
34.601,2
34.709,2
34.821,2
34.926,2
35.037,2
35.141,2
35.244,2
35.354,2
35.462,2
35.575,2
35.687,2
35.796,2
35.908,2
36.013,2
36.12,2
36.233,2
36.34,2
36.446,2
36.553,2
36.663,2
36.771,2
36.881,2
36.989,2
37.095,2
37.208,2
37.312,2
37.424,2
37.534,2
37.644,2
37.748,2
37.859,2
37.964,2
38.068,2
38.173,2
38.28,2
38.392,2
38.496,2
38.601,2
38.712,2
38.82,2
38.933,2
39.045,2
39.155,2
39.263,2
39.376,2
39.49,2
39.606,2
39.72,2
39.829,2
39.939,2
40.042,2
40.154,2
40.259,2
40.365,2
40.478,2
40.582,2
40.692,2
40.796,2
40.907,2
41.015,2
41.118,2
41.228,2
41.338,2
41.445,2
41.551,2
41.662,2
41.774,2
41.882,2
41.993,2
42.099,2
42.204,2
42.312,2
42.419,2
42.527,2
42.637,2
42.739,2
42.85,2
42.961,2
43.068,2
43.176,2
43.278,2
43.39,2
43.494,2
43.599,2
43.71,2
43.816,2
43.92,2
44.025,2
44.128,2
44.233,2
44.337,2
44.445,2
44.552,2
44.663,2
44.77,2
44.881,2
44.988,2
45.1,2
45.203,2
45.312,2
45.416,2
45.523,2
45.634,2
45.737,2
1 # date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
2 t_rel_s,up
3 -25.852,2
4 -25.745,2
5 -25.643,2
6 -25.532,2
7 -25.426,2
8 -25.319,2
9 -25.21,2
10 -25.1,2
11 -24.995,2
12 -24.894,2
13 -24.79,2
14 -24.686,2
15 -24.577,2
16 -24.469,2
17 -24.358,2
18 -24.255,2
19 -24.148,2
20 -24.042,2
21 -23.939,2
22 -23.836,2
23 -23.727,2
24 -23.625,2
25 -23.521,2
26 -23.412,2
27 -23.303,2
28 -23.192,2
29 -23.088,2
30 -22.978,2
31 -22.868,2
32 -22.757,2
33 -22.648,2
34 -22.541,2
35 -22.44,2
36 -22.33,2
37 -22.227,2
38 -22.121,2
39 -22.019,2
40 -21.908,2
41 -21.804,2
42 -21.699,2
43 -21.588,2
44 -21.476,2
45 -21.363,2
46 -21.259,2
47 -21.148,2
48 -21.036,2
49 -20.928,2
50 -20.82,2
51 -20.709,2
52 -20.604,2
53 -20.493,2
54 -20.381,2
55 -20.269,2
56 -20.158,2
57 -20.053,2
58 -19.944,2
59 -19.837,2
60 -19.729,2
61 -19.619,2
62 -19.511,2
63 -19.406,2
64 -19.304,2
65 -19.191,2
66 -19.083,2
67 -18.972,2
68 -18.869,2
69 -18.761,2
70 -18.649,2
71 -18.542,2
72 -18.43,2
73 -18.318,2
74 -18.216,2
75 -18.114,2
76 -18.005,2
77 -17.899,2
78 -17.792,2
79 -17.685,2
80 -17.581,2
81 -17.472,2
82 -17.359,2
83 -17.253,2
84 -17.148,2
85 -17.043,2
86 -16.936,2
87 -16.823,2
88 -16.712,2
89 -16.602,2
90 -16.495,2
91 -16.386,2
92 -16.276,2
93 -16.164,2
94 -16.061,2
95 -15.957,2
96 -15.846,2
97 -15.736,2
98 -15.632,2
99 -15.526,2
100 -15.422,2
101 -15.311,2
102 -15.205,2
103 -15.101,2
104 -14.991,2
105 -14.879,2
106 -14.768,2
107 -14.661,2
108 -14.554,2
109 -14.444,2
110 -14.335,2
111 -14.228,2
112 -14.123,2
113 -14.018,2
114 -13.906,2
115 -13.792,2
116 -13.68,2
117 -13.574,2
118 -13.465,2
119 -13.358,2
120 -13.247,2
121 -13.141,2
122 -13.029,2
123 -12.922,2
124 -12.809,2
125 -12.697,2
126 -12.586,2
127 -12.474,2
128 -12.37,2
129 -12.26,2
130 -12.154,2
131 -12.041,2
132 -11.935,2
133 -11.824,2
134 -11.719,2
135 -11.608,2
136 -11.506,2
137 -11.397,2
138 -11.283,2
139 -11.176,2
140 -11.07,2
141 -10.962,2
142 -10.851,2
143 -10.741,2
144 -10.629,2
145 -10.519,2
146 -10.408,2
147 -10.296,2
148 -10.185,2
149 -10.078,2
150 -9.975,2
151 -9.87,2
152 -9.76,2
153 -9.653,2
154 -9.54,2
155 -9.43,2
156 -9.319,2
157 -9.211,2
158 -9.102,2
159 -8.989,2
160 -8.876,2
161 -8.765,2
162 -8.656,2
163 -8.546,2
164 -8.437,2
165 -8.328,2
166 -8.22,2
167 -8.106,2
168 -7.995,2
169 -7.886,2
170 -7.782,2
171 -7.67,2
172 -7.563,2
173 -7.452,2
174 -7.345,2
175 -7.236,2
176 -7.13,2
177 -7.018,2
178 -6.911,2
179 -6.798,2
180 -6.686,2
181 -6.579,2
182 -6.471,2
183 -6.367,2
184 -6.256,2
185 -6.149,2
186 -6.042,2
187 -5.934,2
188 -5.828,2
189 -5.72,2
190 -5.616,2
191 -5.503,2
192 -5.395,2
193 -5.286,2
194 -5.178,2
195 -5.067,2
196 -4.961,2
197 -4.855,2
198 -4.742,2
199 -4.635,2
200 -4.526,2
201 -4.418,2
202 -4.304,2
203 -4.195,2
204 -4.083,2
205 -3.971,2
206 -3.86,2
207 -3.755,2
208 -3.641,2
209 -3.532,2
210 -3.428,2
211 -3.32,2
212 -3.209,2
213 -3.098,2
214 -2.994,2
215 -2.882,2
216 -2.777,2
217 -2.665,2
218 -2.553,2
219 -2.446,2
220 -2.337,2
221 -2.228,2
222 -2.124,2
223 -2.018,2
224 -1.908,2
225 -1.803,2
226 -1.693,2
227 -1.587,2
228 -1.476,2
229 -1.37,2
230 -1.261,2
231 -1.149,2
232 -1.037,2
233 -0.929,2
234 -0.816,2
235 -0.713,2
236 -0.601,2
237 -0.495,2
238 -0.387,2
239 -0.277,2
240 -0.17,2
241 -0.066,2
242 0.036,2
243 0.142,2
244 0.253,2
245 0.358,2
246 0.461,2
247 0.571,2
248 0.675,2
249 0.78,2
250 0.884,1
251 0.992,1
252 1.098,1
253 1.205,1
254 1.314,1
255 1.426,1
256 1.538,1
257 1.645,1
258 1.755,1
259 1.864,1
260 1.972,1
261 2.078,1
262 2.188,1
263 2.302,1
264 2.411,1
265 2.522,1
266 2.626,1
267 2.737,1
268 2.838,1
269 2.941,1
270 3.049,1
271 3.156,1
272 3.268,1
273 3.377,1
274 3.49,1
275 3.604,1
276 3.716,1
277 3.825,1
278 3.937,1
279 4.044,1
280 4.154,1
281 4.259,1
282 4.369,1
283 4.479,1
284 4.588,1
285 4.693,1
286 4.8,1
287 4.911,1
288 5.018,1
289 5.127,1
290 5.239,1
291 5.35,1
292 5.457,1
293 5.563,1
294 5.664,1
295 5.767,1
296 5.871,1
297 5.976,1
298 6.086,1
299 6.198,1
300 6.306,1
301 6.413,1
302 6.525,1
303 6.638,1
304 6.747,1
305 6.859,1
306 6.97,1
307 7.081,1
308 7.189,1
309 7.302,1
310 7.411,1
311 7.518,1
312 7.631,1
313 7.74,1
314 7.847,1
315 7.96,1
316 8.071,1
317 8.184,1
318 8.288,1
319 8.396,1
320 8.501,1
321 8.605,1
322 8.711,1
323 8.825,1
324 8.93,1
325 9.034,1
326 9.147,1
327 9.259,1
328 9.367,1
329 9.48,1
330 9.584,1
331 9.687,1
332 9.792,1
333 9.904,1
334 10.011,1
335 10.115,1
336 10.225,1
337 10.327,1
338 10.429,1
339 10.539,1
340 10.64,1
341 10.752,1
342 10.862,1
343 10.967,1
344 11.073,1
345 11.185,1
346 11.297,1
347 11.411,1
348 11.521,1
349 11.631,1
350 11.737,1
351 11.85,1
352 11.955,1
353 12.063,1
354 12.175,1
355 12.288,1
356 12.397,1
357 12.502,1
358 12.616,1
359 12.72,1
360 12.83,1
361 12.934,1
362 13.042,1
363 13.151,1
364 13.264,1
365 13.371,1
366 13.485,1
367 13.587,1
368 13.696,1
369 13.807,1
370 13.912,1
371 14.024,1
372 14.136,1
373 14.237,1
374 14.346,1
375 14.46,1
376 14.571,1
377 14.678,1
378 14.783,1
379 14.885,2
380 14.996,2
381 15.101,2
382 15.203,2
383 15.309,2
384 15.423,2
385 15.533,2
386 15.647,2
387 15.759,2
388 15.87,2
389 15.976,2
390 16.088,2
391 16.196,2
392 16.305,2
393 16.411,2
394 16.523,2
395 16.629,2
396 16.742,2
397 16.854,2
398 16.962,2
399 17.067,2
400 17.181,2
401 17.288,2
402 17.397,2
403 17.509,2
404 17.621,2
405 17.733,2
406 17.84,2
407 17.946,2
408 18.054,2
409 18.163,2
410 18.267,2
411 18.379,2
412 18.492,2
413 18.606,2
414 18.71,2
415 18.819,2
416 18.93,2
417 19.038,2
418 19.14,2
419 19.242,2
420 19.346,2
421 19.455,2
422 19.565,2
423 19.672,2
424 19.786,2
425 19.891,2
426 20.0,2
427 20.103,2
428 20.211,2
429 20.315,2
430 20.42,2
431 20.529,2
432 20.64,2
433 20.747,2
434 20.854,2
435 20.965,2
436 21.073,2
437 21.181,2
438 21.286,2
439 21.396,2
440 21.505,2
441 21.618,2
442 21.73,2
443 21.843,2
444 21.948,2
445 22.055,2
446 22.16,2
447 22.273,2
448 22.385,2
449 22.496,2
450 22.602,2
451 22.712,2
452 22.82,2
453 22.93,2
454 23.042,2
455 23.146,2
456 23.253,2
457 23.36,2
458 23.469,2
459 23.581,2
460 23.69,2
461 23.803,2
462 23.917,2
463 24.025,2
464 24.13,2
465 24.236,2
466 24.348,2
467 24.455,2
468 24.567,2
469 24.678,2
470 24.786,2
471 24.9,2
472 25.013,2
473 25.126,2
474 25.241,2
475 25.353,2
476 25.463,2
477 25.574,2
478 25.686,2
479 25.796,2
480 25.908,2
481 26.017,2
482 26.129,2
483 26.239,2
484 26.35,2
485 26.454,2
486 26.563,2
487 26.672,2
488 26.784,2
489 26.889,2
490 26.994,2
491 27.103,2
492 27.212,2
493 27.325,2
494 27.437,2
495 27.545,2
496 27.658,2
497 27.762,2
498 27.874,2
499 27.978,2
500 28.085,2
501 28.192,2
502 28.301,2
503 28.412,2
504 28.524,2
505 28.638,2
506 28.749,2
507 28.856,2
508 28.966,2
509 29.072,2
510 29.182,2
511 29.29,2
512 29.394,2
513 29.499,2
514 29.607,2
515 29.711,2
516 29.821,2
517 29.929,2
518 30.035,2
519 30.144,2
520 30.255,2
521 30.367,2
522 30.472,2
523 30.585,2
524 30.691,2
525 30.799,2
526 30.911,2
527 31.021,2
528 31.13,2
529 31.238,2
530 31.352,2
531 31.462,2
532 31.576,2
533 31.69,2
534 31.797,2
535 31.909,2
536 32.021,2
537 32.129,2
538 32.233,2
539 32.337,2
540 32.444,2
541 32.555,2
542 32.658,2
543 32.769,2
544 32.881,2
545 32.987,2
546 33.095,2
547 33.209,2
548 33.317,2
549 33.42,2
550 33.532,2
551 33.64,2
552 33.751,2
553 33.86,2
554 33.963,2
555 34.074,2
556 34.179,2
557 34.282,2
558 34.387,2
559 34.495,2
560 34.601,2
561 34.709,2
562 34.821,2
563 34.926,2
564 35.037,2
565 35.141,2
566 35.244,2
567 35.354,2
568 35.462,2
569 35.575,2
570 35.687,2
571 35.796,2
572 35.908,2
573 36.013,2
574 36.12,2
575 36.233,2
576 36.34,2
577 36.446,2
578 36.553,2
579 36.663,2
580 36.771,2
581 36.881,2
582 36.989,2
583 37.095,2
584 37.208,2
585 37.312,2
586 37.424,2
587 37.534,2
588 37.644,2
589 37.748,2
590 37.859,2
591 37.964,2
592 38.068,2
593 38.173,2
594 38.28,2
595 38.392,2
596 38.496,2
597 38.601,2
598 38.712,2
599 38.82,2
600 38.933,2
601 39.045,2
602 39.155,2
603 39.263,2
604 39.376,2
605 39.49,2
606 39.606,2
607 39.72,2
608 39.829,2
609 39.939,2
610 40.042,2
611 40.154,2
612 40.259,2
613 40.365,2
614 40.478,2
615 40.582,2
616 40.692,2
617 40.796,2
618 40.907,2
619 41.015,2
620 41.118,2
621 41.228,2
622 41.338,2
623 41.445,2
624 41.551,2
625 41.662,2
626 41.774,2
627 41.882,2
628 41.993,2
629 42.099,2
630 42.204,2
631 42.312,2
632 42.419,2
633 42.527,2
634 42.637,2
635 42.739,2
636 42.85,2
637 42.961,2
638 43.068,2
639 43.176,2
640 43.278,2
641 43.39,2
642 43.494,2
643 43.599,2
644 43.71,2
645 43.816,2
646 43.92,2
647 44.025,2
648 44.128,2
649 44.233,2
650 44.337,2
651 44.445,2
652 44.552,2
653 44.663,2
654 44.77,2
655 44.881,2
656 44.988,2
657 45.1,2
658 45.203,2
659 45.312,2
660 45.416,2
661 45.523,2
662 45.634,2
663 45.737,2
@@ -0,0 +1,647 @@
# date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
t_rel_s,status,latency_ms,instance
-25.852,201,5.2,5bbf259ff136
-25.744,201,9.9,7cfd91ddf780
-25.632,201,5.8,5bbf259ff136
-25.522,201,3.8,7cfd91ddf780
-25.411,201,12.4,7cfd91ddf780
-25.288,201,3.8,7cfd91ddf780
-25.181,201,5.8,5bbf259ff136
-25.074,201,3.9,5bbf259ff136
-24.97,201,3.4,5bbf259ff136
-24.858,201,3.5,5bbf259ff136
-24.752,201,3.9,7cfd91ddf780
-24.641,201,6.8,5bbf259ff136
-24.525,201,4.5,5bbf259ff136
-24.411,201,4.0,7cfd91ddf780
-24.302,201,3.2,5bbf259ff136
-24.192,201,2.8,5bbf259ff136
-24.087,201,3.0,5bbf259ff136
-23.974,201,5.0,5bbf259ff136
-23.859,201,5.2,7cfd91ddf780
-23.753,201,5.8,5bbf259ff136
-23.644,201,4.1,5bbf259ff136
-23.537,201,3.4,5bbf259ff136
-23.426,201,3.4,5bbf259ff136
-23.32,201,3.4,5bbf259ff136
-23.21,201,3.2,5bbf259ff136
-23.099,201,4.2,5bbf259ff136
-22.994,201,4.0,7cfd91ddf780
-22.89,201,3.0,7cfd91ddf780
-22.777,201,3.0,7cfd91ddf780
-22.673,201,3.1,5bbf259ff136
-22.56,201,8.6,7cfd91ddf780
-22.444,201,4.4,7cfd91ddf780
-22.33,201,3.6,5bbf259ff136
-22.223,201,2.7,7cfd91ddf780
-22.12,201,3.6,7cfd91ddf780
-22.016,201,2.8,5bbf259ff136
-21.909,201,7.4,5bbf259ff136
-21.791,201,4.0,7cfd91ddf780
-21.678,201,5.2,7cfd91ddf780
-21.571,201,5.3,5bbf259ff136
-21.458,201,5.3,7cfd91ddf780
-21.343,201,5.7,7cfd91ddf780
-21.227,201,5.0,5bbf259ff136
-21.12,201,5.7,5bbf259ff136
-21.009,201,4.8,5bbf259ff136
-20.894,201,5.2,5bbf259ff136
-20.787,201,5.9,5bbf259ff136
-20.676,201,5.1,5bbf259ff136
-20.569,201,5.8,5bbf259ff136
-20.461,201,4.7,5bbf259ff136
-20.355,201,4.5,7cfd91ddf780
-20.243,201,4.7,5bbf259ff136
-20.128,201,3.5,5bbf259ff136
-20.015,201,4.0,5bbf259ff136
-19.91,201,5.1,5bbf259ff136
-19.795,201,3.7,5bbf259ff136
-19.683,201,5.2,5bbf259ff136
-19.569,201,6.1,7cfd91ddf780
-19.463,201,6.1,7cfd91ddf780
-19.35,201,4.4,7cfd91ddf780
-19.245,201,3.9,5bbf259ff136
-19.137,201,3.7,5bbf259ff136
-19.029,201,3.4,5bbf259ff136
-18.925,201,4.8,7cfd91ddf780
-18.811,201,5.3,5bbf259ff136
-18.701,201,5.2,5bbf259ff136
-18.588,201,4.5,5bbf259ff136
-18.477,201,5.4,5bbf259ff136
-18.362,201,5.6,7cfd91ddf780
-18.25,201,4.7,7cfd91ddf780
-18.136,201,5.0,7cfd91ddf780
-18.022,201,5.6,5bbf259ff136
-17.91,201,5.7,5bbf259ff136
-17.794,201,5.1,5bbf259ff136
-17.681,201,5.1,7cfd91ddf780
-17.575,201,6.1,5bbf259ff136
-17.461,201,5.2,7cfd91ddf780
-17.348,201,5.9,7cfd91ddf780
-17.238,201,4.8,7cfd91ddf780
-17.126,201,5.6,7cfd91ddf780
-17.011,201,6.3,5bbf259ff136
-16.898,201,5.2,5bbf259ff136
-16.785,201,5.4,5bbf259ff136
-16.671,201,5.4,7cfd91ddf780
-16.56,201,4.4,7cfd91ddf780
-16.453,201,4.8,5bbf259ff136
-16.342,201,6.1,7cfd91ddf780
-16.226,201,4.5,7cfd91ddf780
-16.118,201,4.7,7cfd91ddf780
-16.009,201,5.1,7cfd91ddf780
-15.898,201,5.1,7cfd91ddf780
-15.789,201,5.6,7cfd91ddf780
-15.673,201,4.5,7cfd91ddf780
-15.567,201,5.1,7cfd91ddf780
-15.453,201,5.1,5bbf259ff136
-15.339,201,6.0,5bbf259ff136
-15.233,201,5.6,5bbf259ff136
-15.117,201,3.4,5bbf259ff136
-15.004,201,10.7,5bbf259ff136
-14.886,201,6.0,5bbf259ff136
-14.773,201,5.8,5bbf259ff136
-14.659,201,5.1,7cfd91ddf780
-14.548,201,4.0,7cfd91ddf780
-14.437,201,4.3,7cfd91ddf780
-14.328,201,3.7,7cfd91ddf780
-14.217,201,3.2,7cfd91ddf780
-14.106,201,5.6,7cfd91ddf780
-13.997,201,5.7,7cfd91ddf780
-13.886,201,4.7,7cfd91ddf780
-13.771,201,5.3,5bbf259ff136
-13.659,201,4.6,5bbf259ff136
-13.545,201,6.1,5bbf259ff136
-13.437,201,5.0,5bbf259ff136
-13.328,201,4.8,5bbf259ff136
-13.221,201,4.7,5bbf259ff136
-13.11,201,4.8,5bbf259ff136
-12.995,201,5.4,5bbf259ff136
-12.886,201,5.8,5bbf259ff136
-12.771,201,4.9,7cfd91ddf780
-12.66,201,5.4,7cfd91ddf780
-12.553,201,6.2,5bbf259ff136
-12.438,201,5.3,7cfd91ddf780
-12.322,201,4.5,5bbf259ff136
-12.215,201,5.2,7cfd91ddf780
-12.1,201,5.2,5bbf259ff136
-11.987,201,4.8,5bbf259ff136
-11.878,201,5.2,5bbf259ff136
-11.766,201,5.1,5bbf259ff136
-11.657,201,4.4,5bbf259ff136
-11.545,201,6.1,5bbf259ff136
-11.436,201,4.6,5bbf259ff136
-11.325,201,5.4,5bbf259ff136
-11.209,201,4.7,7cfd91ddf780
-11.098,201,5.5,7cfd91ddf780
-10.982,201,5.2,5bbf259ff136
-10.87,201,5.4,5bbf259ff136
-10.76,201,5.7,5bbf259ff136
-10.646,201,6.0,5bbf259ff136
-10.535,201,5.0,5bbf259ff136
-10.425,201,4.7,5bbf259ff136
-10.311,201,5.4,7cfd91ddf780
-10.2,201,4.5,7cfd91ddf780
-10.085,201,5.1,7cfd91ddf780
-9.974,201,4.6,7cfd91ddf780
-9.864,201,3.1,7cfd91ddf780
-9.753,201,3.5,7cfd91ddf780
-9.649,201,4.3,7cfd91ddf780
-9.536,201,4.2,7cfd91ddf780
-9.427,201,4.4,7cfd91ddf780
-9.319,201,6.4,7cfd91ddf780
-9.211,201,6.6,7cfd91ddf780
-9.101,201,4.1,7cfd91ddf780
-8.989,201,5.0,7cfd91ddf780
-8.876,201,5.0,7cfd91ddf780
-8.765,201,6.1,7cfd91ddf780
-8.653,201,5.9,5bbf259ff136
-8.537,201,6.2,5bbf259ff136
-8.426,201,5.2,5bbf259ff136
-8.312,201,5.0,5bbf259ff136
-8.2,201,6.5,7cfd91ddf780
-8.088,201,4.4,7cfd91ddf780
-7.973,201,4.4,7cfd91ddf780
-7.861,201,5.0,5bbf259ff136
-7.746,201,5.8,5bbf259ff136
-7.636,201,5.9,5bbf259ff136
-7.52,201,5.3,7cfd91ddf780
-7.411,201,5.2,7cfd91ddf780
-7.298,201,4.9,7cfd91ddf780
-7.186,201,5.1,7cfd91ddf780
-7.075,201,5.5,7cfd91ddf780
-6.966,201,4.8,7cfd91ddf780
-6.853,201,7.0,5bbf259ff136
-6.741,201,5.7,5bbf259ff136
-6.632,201,4.9,5bbf259ff136
-6.517,201,5.1,5bbf259ff136
-6.402,201,5.1,7cfd91ddf780
-6.287,201,4.3,7cfd91ddf780
-6.176,201,5.5,7cfd91ddf780
-6.06,201,5.7,5bbf259ff136
-5.945,201,5.3,5bbf259ff136
-5.829,201,5.0,7cfd91ddf780
-5.719,201,4.2,7cfd91ddf780
-5.604,201,5.3,7cfd91ddf780
-5.49,201,4.7,5bbf259ff136
-5.385,201,3.9,7cfd91ddf780
-5.28,201,4.5,7cfd91ddf780
-5.171,201,4.4,7cfd91ddf780
-5.058,201,5.1,7cfd91ddf780
-4.951,201,4.9,7cfd91ddf780
-4.845,201,6.2,5bbf259ff136
-4.729,201,2.9,7cfd91ddf780
-4.626,201,3.3,5bbf259ff136
-4.516,201,3.9,5bbf259ff136
-4.412,201,4.6,5bbf259ff136
-4.303,201,5.2,5bbf259ff136
-4.194,201,4.7,7cfd91ddf780
-4.089,201,4.7,7cfd91ddf780
-3.974,201,5.0,7cfd91ddf780
-3.862,201,5.4,5bbf259ff136
-3.755,201,6.0,7cfd91ddf780
-3.641,201,6.5,5bbf259ff136
-3.532,201,5.1,5bbf259ff136
-3.418,201,4.7,5bbf259ff136
-3.305,201,4.8,5bbf259ff136
-3.195,201,5.7,5bbf259ff136
-3.085,201,4.4,5bbf259ff136
-2.975,201,5.2,5bbf259ff136
-2.862,201,4.9,7cfd91ddf780
-2.748,201,4.5,7cfd91ddf780
-2.637,201,4.6,7cfd91ddf780
-2.529,201,6.1,7cfd91ddf780
-2.413,201,5.2,5bbf259ff136
-2.302,201,5.6,5bbf259ff136
-2.187,201,4.9,5bbf259ff136
-2.072,201,5.5,7cfd91ddf780
-1.956,201,5.1,7cfd91ddf780
-1.849,201,5.2,7cfd91ddf780
-1.737,201,5.4,7cfd91ddf780
-1.625,201,5.3,7cfd91ddf780
-1.51,201,5.3,5bbf259ff136
-1.404,201,6.3,7cfd91ddf780
-1.291,201,5.5,5bbf259ff136
-1.185,201,5.8,7cfd91ddf780
-1.078,201,6.7,7cfd91ddf780
-0.964,201,7.0,5bbf259ff136
-0.855,201,5.7,7cfd91ddf780
-0.748,201,4.7,7cfd91ddf780
-0.642,201,5.7,7cfd91ddf780
-0.526,201,5.6,7cfd91ddf780
-0.42,201,5.1,7cfd91ddf780
-0.309,201,5.5,7cfd91ddf780
-0.199,201,5.0,7cfd91ddf780
-0.088,201,5.2,7cfd91ddf780
0.019,201,4.0,7cfd91ddf780
0.133,201,3.5,7cfd91ddf780
0.241,201,3.7,7cfd91ddf780
0.352,201,6.6,7cfd91ddf780
0.46,201,5.2,7cfd91ddf780
0.569,201,4.6,7cfd91ddf780
0.675,201,5.2,5bbf259ff136
0.79,201,4.4,7cfd91ddf780
0.896,201,4.5,5bbf259ff136
1.005,201,5.1,5bbf259ff136
1.118,201,5.4,5bbf259ff136
1.233,201,3.4,5bbf259ff136
1.343,201,4.1,5bbf259ff136
1.457,201,4.8,5bbf259ff136
1.565,201,4.2,5bbf259ff136
1.679,201,4.6,5bbf259ff136
1.785,201,6.0,5bbf259ff136
1.895,201,6.1,5bbf259ff136
2.011,201,4.3,5bbf259ff136
2.117,201,6.7,5bbf259ff136
2.234,201,4.1,5bbf259ff136
2.342,201,5.5,5bbf259ff136
2.457,201,6.2,5bbf259ff136
2.57,201,3.6,5bbf259ff136
2.675,201,4.6,5bbf259ff136
2.78,201,4.0,5bbf259ff136
2.892,201,3.2,5bbf259ff136
2.995,201,3.1,5bbf259ff136
3.106,201,6.7,5bbf259ff136
3.218,201,6.4,5bbf259ff136
3.334,201,6.6,5bbf259ff136
3.451,201,6.3,5bbf259ff136
3.563,201,5.6,5bbf259ff136
3.669,201,7.1,5bbf259ff136
3.78,201,4.6,5bbf259ff136
3.887,201,5.8,5bbf259ff136
4.003,201,6.2,5bbf259ff136
4.12,201,5.4,5bbf259ff136
4.234,201,5.0,5bbf259ff136
4.348,201,6.5,5bbf259ff136
4.458,201,6.0,5bbf259ff136
4.564,201,5.9,5bbf259ff136
4.674,201,5.4,5bbf259ff136
4.787,201,7.3,5bbf259ff136
4.9,201,5.7,5bbf259ff136
5.014,201,5.3,5bbf259ff136
5.126,201,5.2,5bbf259ff136
5.241,201,5.2,5bbf259ff136
5.352,201,3.0,5bbf259ff136
5.458,201,3.6,5bbf259ff136
5.566,201,2.8,5bbf259ff136
5.672,201,3.0,5bbf259ff136
5.776,201,4.3,5bbf259ff136
5.882,201,4.2,5bbf259ff136
5.99,201,5.6,5bbf259ff136
6.097,201,3.8,5bbf259ff136
6.204,201,5.5,5bbf259ff136
6.31,201,5.2,5bbf259ff136
6.421,201,3.4,5bbf259ff136
6.529,201,4.4,5bbf259ff136
6.636,201,3.7,5bbf259ff136
6.747,201,4.6,5bbf259ff136
6.861,201,4.7,5bbf259ff136
6.974,201,3.9,5bbf259ff136
7.081,201,5.7,5bbf259ff136
7.189,201,6.0,5bbf259ff136
7.302,201,4.3,5bbf259ff136
7.412,201,4.8,5bbf259ff136
7.522,201,4.8,5bbf259ff136
7.631,201,4.5,5bbf259ff136
7.739,201,6.3,5bbf259ff136
7.854,201,3.7,5bbf259ff136
7.959,201,6.1,5bbf259ff136
8.075,201,6.3,5bbf259ff136
8.191,201,5.6,5bbf259ff136
8.306,201,6.0,5bbf259ff136
8.422,201,5.4,5bbf259ff136
8.537,201,5.8,5bbf259ff136
8.651,201,5.3,5bbf259ff136
8.758,201,6.5,5bbf259ff136
8.869,201,5.5,5bbf259ff136
8.977,201,7.0,5bbf259ff136
9.094,201,4.9,5bbf259ff136
9.202,201,5.9,5bbf259ff136
9.313,201,5.8,5bbf259ff136
9.421,201,4.4,5bbf259ff136
9.526,201,6.7,5bbf259ff136
9.64,201,7.0,5bbf259ff136
9.753,201,7.7,5bbf259ff136
9.869,201,5.7,5bbf259ff136
9.979,201,5.9,5bbf259ff136
10.086,201,6.6,5bbf259ff136
10.194,201,4.8,5bbf259ff136
10.309,201,4.9,5bbf259ff136
10.419,201,3.7,5bbf259ff136
10.529,201,4.7,5bbf259ff136
10.64,201,5.3,5bbf259ff136
10.751,201,4.1,5bbf259ff136
10.862,201,4.6,5bbf259ff136
10.968,201,6.9,5bbf259ff136
11.084,201,5.0,5bbf259ff136
11.191,201,4.0,5bbf259ff136
11.296,201,7.0,5bbf259ff136
11.411,201,6.1,5bbf259ff136
11.527,201,6.0,5bbf259ff136
11.641,201,4.6,5bbf259ff136
11.753,201,4.7,5bbf259ff136
11.868,201,5.1,5bbf259ff136
11.976,201,6.4,5bbf259ff136
12.088,201,4.8,5bbf259ff136
12.199,201,3.8,5bbf259ff136
12.313,201,4.7,5bbf259ff136
12.424,201,4.9,5bbf259ff136
12.535,201,4.9,5bbf259ff136
12.65,201,5.2,5bbf259ff136
12.762,201,4.6,5bbf259ff136
12.869,201,4.6,5bbf259ff136
12.977,201,3.8,5bbf259ff136
13.089,201,3.6,5bbf259ff136
13.196,201,5.8,5bbf259ff136
13.304,201,4.5,5bbf259ff136
13.413,201,4.0,5bbf259ff136
13.526,201,5.2,5bbf259ff136
13.633,201,5.3,5bbf259ff136
13.747,201,3.9,5bbf259ff136
13.859,201,3.8,5bbf259ff136
13.969,201,3.9,5bbf259ff136
14.076,201,3.7,5bbf259ff136
14.184,201,3.5,5bbf259ff136
14.296,201,5.1,5bbf259ff136
14.409,201,4.4,5bbf259ff136
14.519,201,7.0,5bbf259ff136
14.632,201,6.5,5bbf259ff136
14.746,201,6.7,5bbf259ff136
14.855,201,5.9,5bbf259ff136
14.968,201,11.9,7cfd91ddf780
15.088,201,5.6,5bbf259ff136
15.203,201,4.0,5bbf259ff136
15.31,201,4.9,5bbf259ff136
15.423,201,6.8,5bbf259ff136
15.538,201,4.5,7cfd91ddf780
15.646,201,5.7,7cfd91ddf780
15.758,201,7.3,7cfd91ddf780
15.868,201,8.8,7cfd91ddf780
15.987,201,3.4,5bbf259ff136
16.1,201,4.6,5bbf259ff136
16.207,201,4.9,5bbf259ff136
16.319,201,4.9,5bbf259ff136
16.432,201,4.5,5bbf259ff136
16.538,201,4.7,5bbf259ff136
16.651,201,6.0,5bbf259ff136
16.763,201,4.9,5bbf259ff136
16.874,201,4.4,5bbf259ff136
16.979,201,4.1,5bbf259ff136
17.09,201,5.1,5bbf259ff136
17.197,201,5.2,5bbf259ff136
17.312,201,5.9,5bbf259ff136
17.426,201,5.1,5bbf259ff136
17.541,201,5.4,7cfd91ddf780
17.652,201,5.2,7cfd91ddf780
17.761,201,5.1,7cfd91ddf780
17.877,201,6.2,5bbf259ff136
17.993,201,6.3,5bbf259ff136
18.109,201,6.2,7cfd91ddf780
18.218,201,5.0,7cfd91ddf780
18.334,201,5.3,7cfd91ddf780
18.442,201,5.2,7cfd91ddf780
18.55,201,5.5,7cfd91ddf780
18.663,201,5.2,7cfd91ddf780
18.775,201,5.8,5bbf259ff136
18.891,201,4.3,5bbf259ff136
19.006,201,11.6,7cfd91ddf780
19.127,201,4.9,5bbf259ff136
19.241,201,6.1,5bbf259ff136
19.35,201,5.0,5bbf259ff136
19.463,201,4.9,5bbf259ff136
19.571,201,5.1,5bbf259ff136
19.682,201,4.7,5bbf259ff136
19.795,201,5.0,5bbf259ff136
19.91,201,4.4,7cfd91ddf780
20.02,201,6.2,7cfd91ddf780
20.129,201,5.4,7cfd91ddf780
20.244,201,5.8,7cfd91ddf780
20.354,201,5.2,7cfd91ddf780
20.468,201,5.6,5bbf259ff136
20.579,201,7.0,5bbf259ff136
20.686,201,4.7,7cfd91ddf780
20.793,201,4.9,7cfd91ddf780
20.901,201,5.0,7cfd91ddf780
21.009,201,3.9,7cfd91ddf780
21.121,201,4.9,7cfd91ddf780
21.229,201,5.9,7cfd91ddf780
21.337,201,4.9,7cfd91ddf780
21.446,201,5.4,7cfd91ddf780
21.562,201,6.4,7cfd91ddf780
21.671,201,6.0,7cfd91ddf780
21.787,201,5.8,5bbf259ff136
21.893,201,4.9,7cfd91ddf780
22.007,201,5.0,7cfd91ddf780
22.116,201,5.8,5bbf259ff136
22.222,201,5.2,7cfd91ddf780
22.329,201,4.9,7cfd91ddf780
22.438,201,5.8,7cfd91ddf780
22.553,201,6.3,7cfd91ddf780
22.669,201,5.5,5bbf259ff136
22.785,201,6.3,7cfd91ddf780
22.896,201,5.5,5bbf259ff136
23.006,201,6.2,7cfd91ddf780
23.115,201,6.6,5bbf259ff136
23.232,201,5.3,7cfd91ddf780
23.347,201,6.7,5bbf259ff136
23.461,201,6.8,5bbf259ff136
23.578,201,4.1,7cfd91ddf780
23.687,201,6.1,7cfd91ddf780
23.802,201,6.8,7cfd91ddf780
23.917,201,6.0,5bbf259ff136
24.033,201,3.2,5bbf259ff136
24.145,201,3.9,5bbf259ff136
24.256,201,4.2,7cfd91ddf780
24.363,201,4.9,5bbf259ff136
24.468,201,5.0,5bbf259ff136
24.577,201,4.5,5bbf259ff136
24.687,201,5.2,7cfd91ddf780
24.797,201,5.3,5bbf259ff136
24.909,201,4.7,5bbf259ff136
25.021,201,5.0,5bbf259ff136
25.126,201,7.2,5bbf259ff136
25.239,201,6.0,5bbf259ff136
25.35,201,5.8,5bbf259ff136
25.463,201,6.2,5bbf259ff136
25.573,201,5.5,5bbf259ff136
25.687,201,5.2,7cfd91ddf780
25.797,201,5.8,7cfd91ddf780
25.912,201,5.2,7cfd91ddf780
26.021,201,4.8,7cfd91ddf780
26.131,201,3.5,7cfd91ddf780
26.235,201,3.7,5bbf259ff136
26.349,201,5.5,7cfd91ddf780
26.465,201,4.8,7cfd91ddf780
26.57,201,4.5,7cfd91ddf780
26.679,201,4.2,7cfd91ddf780
26.793,201,4.8,7cfd91ddf780
26.903,201,4.9,7cfd91ddf780
27.012,201,5.4,7cfd91ddf780
27.121,201,5.7,7cfd91ddf780
27.227,201,5.0,7cfd91ddf780
27.342,201,5.4,7cfd91ddf780
27.452,201,5.1,7cfd91ddf780
27.565,201,4.8,7cfd91ddf780
27.675,201,5.0,7cfd91ddf780
27.79,201,5.4,5bbf259ff136
27.904,201,5.8,5bbf259ff136
28.014,201,6.5,5bbf259ff136
28.131,201,5.5,7cfd91ddf780
28.246,201,5.7,5bbf259ff136
28.354,201,5.8,7cfd91ddf780
28.467,201,6.2,5bbf259ff136
28.581,201,5.6,5bbf259ff136
28.69,201,4.3,5bbf259ff136
28.795,201,5.1,7cfd91ddf780
28.907,201,5.1,5bbf259ff136
29.015,201,6.1,7cfd91ddf780
29.123,201,6.4,7cfd91ddf780
29.237,201,3.9,5bbf259ff136
29.349,201,4.2,5bbf259ff136
29.455,201,7.7,5bbf259ff136
29.569,201,7.2,5bbf259ff136
29.679,201,6.3,5bbf259ff136
29.794,201,7.5,5bbf259ff136
29.91,201,5.8,7cfd91ddf780
30.022,201,6.9,7cfd91ddf780
30.136,201,6.7,5bbf259ff136
30.252,201,6.3,5bbf259ff136
30.367,201,6.1,7cfd91ddf780
30.479,201,6.8,7cfd91ddf780
30.595,201,4.9,7cfd91ddf780
30.703,201,5.1,7cfd91ddf780
30.813,201,4.7,7cfd91ddf780
30.924,201,5.3,7cfd91ddf780
31.032,201,5.1,7cfd91ddf780
31.143,201,4.5,7cfd91ddf780
31.257,201,4.8,5bbf259ff136
31.371,201,3.3,5bbf259ff136
31.477,201,4.6,5bbf259ff136
31.586,201,3.6,5bbf259ff136
31.699,201,4.4,5bbf259ff136
31.813,201,4.8,5bbf259ff136
31.927,201,4.6,7cfd91ddf780
32.042,201,5.3,7cfd91ddf780
32.158,201,5.0,5bbf259ff136
32.27,201,4.2,5bbf259ff136
32.378,201,5.6,5bbf259ff136
32.487,201,5.7,5bbf259ff136
32.603,201,4.3,5bbf259ff136
32.712,201,6.5,5bbf259ff136
32.828,201,6.7,7cfd91ddf780
32.945,201,5.3,7cfd91ddf780
33.054,201,5.8,7cfd91ddf780
33.168,201,6.3,5bbf259ff136
33.279,201,8.0,5bbf259ff136
33.393,201,5.6,5bbf259ff136
33.503,201,4.7,5bbf259ff136
33.616,201,6.1,7cfd91ddf780
33.731,201,4.7,7cfd91ddf780
33.845,201,7.1,7cfd91ddf780
33.954,201,5.1,7cfd91ddf780
34.069,201,5.8,5bbf259ff136
34.179,201,5.2,5bbf259ff136
34.288,201,4.7,5bbf259ff136
34.399,201,3.1,5bbf259ff136
34.512,201,3.5,5bbf259ff136
34.625,201,4.0,5bbf259ff136
34.733,201,4.6,5bbf259ff136
34.848,201,5.8,7cfd91ddf780
34.957,201,4.2,7cfd91ddf780
35.062,201,6.9,5bbf259ff136
35.169,201,5.2,5bbf259ff136
35.282,201,6.0,5bbf259ff136
35.395,201,5.3,5bbf259ff136
35.503,201,5.2,5bbf259ff136
35.614,201,5.8,5bbf259ff136
35.72,201,4.9,5bbf259ff136
35.83,201,6.2,5bbf259ff136
35.938,201,5.5,5bbf259ff136
36.05,201,6.1,5bbf259ff136
36.159,201,5.0,5bbf259ff136
36.273,201,5.2,5bbf259ff136
36.387,201,6.5,7cfd91ddf780
36.495,201,8.3,5bbf259ff136
36.611,201,3.5,7cfd91ddf780
36.716,201,4.1,7cfd91ddf780
36.823,201,7.0,5bbf259ff136
36.937,201,4.8,7cfd91ddf780
37.046,201,6.6,7cfd91ddf780
37.16,201,5.2,7cfd91ddf780
37.271,201,4.7,7cfd91ddf780
37.378,201,5.3,7cfd91ddf780
37.487,201,6.1,7cfd91ddf780
37.596,201,4.5,7cfd91ddf780
37.71,201,4.0,7cfd91ddf780
37.821,201,5.1,7cfd91ddf780
37.927,201,4.4,7cfd91ddf780
38.037,201,6.3,7cfd91ddf780
38.145,201,6.1,5bbf259ff136
38.259,201,5.1,7cfd91ddf780
38.373,201,8.5,7cfd91ddf780
38.485,201,5.3,7cfd91ddf780
38.592,201,4.8,7cfd91ddf780
38.706,201,4.2,7cfd91ddf780
38.82,201,4.7,5bbf259ff136
38.933,201,5.9,5bbf259ff136
39.041,201,6.6,5bbf259ff136
39.155,201,6.8,7cfd91ddf780
39.269,201,5.0,7cfd91ddf780
39.379,201,5.5,7cfd91ddf780
39.49,201,11.5,7cfd91ddf780
39.605,201,6.4,5bbf259ff136
39.72,201,5.7,5bbf259ff136
39.828,201,6.5,5bbf259ff136
39.939,201,6.0,5bbf259ff136
40.054,201,5.0,5bbf259ff136
40.161,201,4.9,5bbf259ff136
40.27,201,5.6,5bbf259ff136
40.377,201,6.1,7cfd91ddf780
40.493,201,4.2,5bbf259ff136
40.604,201,6.2,5bbf259ff136
40.71,201,4.2,5bbf259ff136
40.817,201,4.2,5bbf259ff136
40.927,201,5.0,5bbf259ff136
41.039,201,5.1,5bbf259ff136
41.154,201,5.1,5bbf259ff136
41.264,201,4.9,5bbf259ff136
41.372,201,5.1,5bbf259ff136
41.487,201,6.8,7cfd91ddf780
41.597,201,3.7,7cfd91ddf780
41.711,201,3.9,7cfd91ddf780
41.824,201,4.3,7cfd91ddf780
41.939,201,5.0,5bbf259ff136
42.049,201,6.2,5bbf259ff136
42.16,201,5.7,5bbf259ff136
42.27,201,3.6,5bbf259ff136
42.378,201,3.7,5bbf259ff136
42.492,201,4.5,5bbf259ff136
42.599,201,4.3,5bbf259ff136
42.711,201,4.0,5bbf259ff136
42.821,201,6.6,5bbf259ff136
42.937,201,4.8,7cfd91ddf780
43.048,201,6.1,7cfd91ddf780
43.16,201,3.3,7cfd91ddf780
43.265,201,3.2,5bbf259ff136
43.37,201,3.5,5bbf259ff136
43.482,201,3.4,5bbf259ff136
43.59,201,3.1,5bbf259ff136
43.698,201,7.8,5bbf259ff136
43.816,201,7.4,7cfd91ddf780
43.928,201,5.0,7cfd91ddf780
44.036,201,5.0,7cfd91ddf780
44.149,201,5.4,7cfd91ddf780
44.264,201,6.0,7cfd91ddf780
44.371,201,5.1,7cfd91ddf780
44.481,201,4.8,7cfd91ddf780
44.595,201,3.4,7cfd91ddf780
44.7,201,3.2,7cfd91ddf780
44.812,201,2.8,7cfd91ddf780
44.925,201,4.3,7cfd91ddf780
45.034,201,5.7,5bbf259ff136
45.145,201,8.0,7cfd91ddf780
45.26,201,8.5,5bbf259ff136
45.369,201,8.2,7cfd91ddf780
45.487,201,11.7,5bbf259ff136
45.607,201,11.0,7cfd91ddf780
45.727,201,4.3,5bbf259ff136
1 # date: 2026-11-05T19:18:13.470607+00:00 commit: 252e3e3ffeea5004cfd9dc342e686af1417a77a2
2 t_rel_s,status,latency_ms,instance
3 -25.852,201,5.2,5bbf259ff136
4 -25.744,201,9.9,7cfd91ddf780
5 -25.632,201,5.8,5bbf259ff136
6 -25.522,201,3.8,7cfd91ddf780
7 -25.411,201,12.4,7cfd91ddf780
8 -25.288,201,3.8,7cfd91ddf780
9 -25.181,201,5.8,5bbf259ff136
10 -25.074,201,3.9,5bbf259ff136
11 -24.97,201,3.4,5bbf259ff136
12 -24.858,201,3.5,5bbf259ff136
13 -24.752,201,3.9,7cfd91ddf780
14 -24.641,201,6.8,5bbf259ff136
15 -24.525,201,4.5,5bbf259ff136
16 -24.411,201,4.0,7cfd91ddf780
17 -24.302,201,3.2,5bbf259ff136
18 -24.192,201,2.8,5bbf259ff136
19 -24.087,201,3.0,5bbf259ff136
20 -23.974,201,5.0,5bbf259ff136
21 -23.859,201,5.2,7cfd91ddf780
22 -23.753,201,5.8,5bbf259ff136
23 -23.644,201,4.1,5bbf259ff136
24 -23.537,201,3.4,5bbf259ff136
25 -23.426,201,3.4,5bbf259ff136
26 -23.32,201,3.4,5bbf259ff136
27 -23.21,201,3.2,5bbf259ff136
28 -23.099,201,4.2,5bbf259ff136
29 -22.994,201,4.0,7cfd91ddf780
30 -22.89,201,3.0,7cfd91ddf780
31 -22.777,201,3.0,7cfd91ddf780
32 -22.673,201,3.1,5bbf259ff136
33 -22.56,201,8.6,7cfd91ddf780
34 -22.444,201,4.4,7cfd91ddf780
35 -22.33,201,3.6,5bbf259ff136
36 -22.223,201,2.7,7cfd91ddf780
37 -22.12,201,3.6,7cfd91ddf780
38 -22.016,201,2.8,5bbf259ff136
39 -21.909,201,7.4,5bbf259ff136
40 -21.791,201,4.0,7cfd91ddf780
41 -21.678,201,5.2,7cfd91ddf780
42 -21.571,201,5.3,5bbf259ff136
43 -21.458,201,5.3,7cfd91ddf780
44 -21.343,201,5.7,7cfd91ddf780
45 -21.227,201,5.0,5bbf259ff136
46 -21.12,201,5.7,5bbf259ff136
47 -21.009,201,4.8,5bbf259ff136
48 -20.894,201,5.2,5bbf259ff136
49 -20.787,201,5.9,5bbf259ff136
50 -20.676,201,5.1,5bbf259ff136
51 -20.569,201,5.8,5bbf259ff136
52 -20.461,201,4.7,5bbf259ff136
53 -20.355,201,4.5,7cfd91ddf780
54 -20.243,201,4.7,5bbf259ff136
55 -20.128,201,3.5,5bbf259ff136
56 -20.015,201,4.0,5bbf259ff136
57 -19.91,201,5.1,5bbf259ff136
58 -19.795,201,3.7,5bbf259ff136
59 -19.683,201,5.2,5bbf259ff136
60 -19.569,201,6.1,7cfd91ddf780
61 -19.463,201,6.1,7cfd91ddf780
62 -19.35,201,4.4,7cfd91ddf780
63 -19.245,201,3.9,5bbf259ff136
64 -19.137,201,3.7,5bbf259ff136
65 -19.029,201,3.4,5bbf259ff136
66 -18.925,201,4.8,7cfd91ddf780
67 -18.811,201,5.3,5bbf259ff136
68 -18.701,201,5.2,5bbf259ff136
69 -18.588,201,4.5,5bbf259ff136
70 -18.477,201,5.4,5bbf259ff136
71 -18.362,201,5.6,7cfd91ddf780
72 -18.25,201,4.7,7cfd91ddf780
73 -18.136,201,5.0,7cfd91ddf780
74 -18.022,201,5.6,5bbf259ff136
75 -17.91,201,5.7,5bbf259ff136
76 -17.794,201,5.1,5bbf259ff136
77 -17.681,201,5.1,7cfd91ddf780
78 -17.575,201,6.1,5bbf259ff136
79 -17.461,201,5.2,7cfd91ddf780
80 -17.348,201,5.9,7cfd91ddf780
81 -17.238,201,4.8,7cfd91ddf780
82 -17.126,201,5.6,7cfd91ddf780
83 -17.011,201,6.3,5bbf259ff136
84 -16.898,201,5.2,5bbf259ff136
85 -16.785,201,5.4,5bbf259ff136
86 -16.671,201,5.4,7cfd91ddf780
87 -16.56,201,4.4,7cfd91ddf780
88 -16.453,201,4.8,5bbf259ff136
89 -16.342,201,6.1,7cfd91ddf780
90 -16.226,201,4.5,7cfd91ddf780
91 -16.118,201,4.7,7cfd91ddf780
92 -16.009,201,5.1,7cfd91ddf780
93 -15.898,201,5.1,7cfd91ddf780
94 -15.789,201,5.6,7cfd91ddf780
95 -15.673,201,4.5,7cfd91ddf780
96 -15.567,201,5.1,7cfd91ddf780
97 -15.453,201,5.1,5bbf259ff136
98 -15.339,201,6.0,5bbf259ff136
99 -15.233,201,5.6,5bbf259ff136
100 -15.117,201,3.4,5bbf259ff136
101 -15.004,201,10.7,5bbf259ff136
102 -14.886,201,6.0,5bbf259ff136
103 -14.773,201,5.8,5bbf259ff136
104 -14.659,201,5.1,7cfd91ddf780
105 -14.548,201,4.0,7cfd91ddf780
106 -14.437,201,4.3,7cfd91ddf780
107 -14.328,201,3.7,7cfd91ddf780
108 -14.217,201,3.2,7cfd91ddf780
109 -14.106,201,5.6,7cfd91ddf780
110 -13.997,201,5.7,7cfd91ddf780
111 -13.886,201,4.7,7cfd91ddf780
112 -13.771,201,5.3,5bbf259ff136
113 -13.659,201,4.6,5bbf259ff136
114 -13.545,201,6.1,5bbf259ff136
115 -13.437,201,5.0,5bbf259ff136
116 -13.328,201,4.8,5bbf259ff136
117 -13.221,201,4.7,5bbf259ff136
118 -13.11,201,4.8,5bbf259ff136
119 -12.995,201,5.4,5bbf259ff136
120 -12.886,201,5.8,5bbf259ff136
121 -12.771,201,4.9,7cfd91ddf780
122 -12.66,201,5.4,7cfd91ddf780
123 -12.553,201,6.2,5bbf259ff136
124 -12.438,201,5.3,7cfd91ddf780
125 -12.322,201,4.5,5bbf259ff136
126 -12.215,201,5.2,7cfd91ddf780
127 -12.1,201,5.2,5bbf259ff136
128 -11.987,201,4.8,5bbf259ff136
129 -11.878,201,5.2,5bbf259ff136
130 -11.766,201,5.1,5bbf259ff136
131 -11.657,201,4.4,5bbf259ff136
132 -11.545,201,6.1,5bbf259ff136
133 -11.436,201,4.6,5bbf259ff136
134 -11.325,201,5.4,5bbf259ff136
135 -11.209,201,4.7,7cfd91ddf780
136 -11.098,201,5.5,7cfd91ddf780
137 -10.982,201,5.2,5bbf259ff136
138 -10.87,201,5.4,5bbf259ff136
139 -10.76,201,5.7,5bbf259ff136
140 -10.646,201,6.0,5bbf259ff136
141 -10.535,201,5.0,5bbf259ff136
142 -10.425,201,4.7,5bbf259ff136
143 -10.311,201,5.4,7cfd91ddf780
144 -10.2,201,4.5,7cfd91ddf780
145 -10.085,201,5.1,7cfd91ddf780
146 -9.974,201,4.6,7cfd91ddf780
147 -9.864,201,3.1,7cfd91ddf780
148 -9.753,201,3.5,7cfd91ddf780
149 -9.649,201,4.3,7cfd91ddf780
150 -9.536,201,4.2,7cfd91ddf780
151 -9.427,201,4.4,7cfd91ddf780
152 -9.319,201,6.4,7cfd91ddf780
153 -9.211,201,6.6,7cfd91ddf780
154 -9.101,201,4.1,7cfd91ddf780
155 -8.989,201,5.0,7cfd91ddf780
156 -8.876,201,5.0,7cfd91ddf780
157 -8.765,201,6.1,7cfd91ddf780
158 -8.653,201,5.9,5bbf259ff136
159 -8.537,201,6.2,5bbf259ff136
160 -8.426,201,5.2,5bbf259ff136
161 -8.312,201,5.0,5bbf259ff136
162 -8.2,201,6.5,7cfd91ddf780
163 -8.088,201,4.4,7cfd91ddf780
164 -7.973,201,4.4,7cfd91ddf780
165 -7.861,201,5.0,5bbf259ff136
166 -7.746,201,5.8,5bbf259ff136
167 -7.636,201,5.9,5bbf259ff136
168 -7.52,201,5.3,7cfd91ddf780
169 -7.411,201,5.2,7cfd91ddf780
170 -7.298,201,4.9,7cfd91ddf780
171 -7.186,201,5.1,7cfd91ddf780
172 -7.075,201,5.5,7cfd91ddf780
173 -6.966,201,4.8,7cfd91ddf780
174 -6.853,201,7.0,5bbf259ff136
175 -6.741,201,5.7,5bbf259ff136
176 -6.632,201,4.9,5bbf259ff136
177 -6.517,201,5.1,5bbf259ff136
178 -6.402,201,5.1,7cfd91ddf780
179 -6.287,201,4.3,7cfd91ddf780
180 -6.176,201,5.5,7cfd91ddf780
181 -6.06,201,5.7,5bbf259ff136
182 -5.945,201,5.3,5bbf259ff136
183 -5.829,201,5.0,7cfd91ddf780
184 -5.719,201,4.2,7cfd91ddf780
185 -5.604,201,5.3,7cfd91ddf780
186 -5.49,201,4.7,5bbf259ff136
187 -5.385,201,3.9,7cfd91ddf780
188 -5.28,201,4.5,7cfd91ddf780
189 -5.171,201,4.4,7cfd91ddf780
190 -5.058,201,5.1,7cfd91ddf780
191 -4.951,201,4.9,7cfd91ddf780
192 -4.845,201,6.2,5bbf259ff136
193 -4.729,201,2.9,7cfd91ddf780
194 -4.626,201,3.3,5bbf259ff136
195 -4.516,201,3.9,5bbf259ff136
196 -4.412,201,4.6,5bbf259ff136
197 -4.303,201,5.2,5bbf259ff136
198 -4.194,201,4.7,7cfd91ddf780
199 -4.089,201,4.7,7cfd91ddf780
200 -3.974,201,5.0,7cfd91ddf780
201 -3.862,201,5.4,5bbf259ff136
202 -3.755,201,6.0,7cfd91ddf780
203 -3.641,201,6.5,5bbf259ff136
204 -3.532,201,5.1,5bbf259ff136
205 -3.418,201,4.7,5bbf259ff136
206 -3.305,201,4.8,5bbf259ff136
207 -3.195,201,5.7,5bbf259ff136
208 -3.085,201,4.4,5bbf259ff136
209 -2.975,201,5.2,5bbf259ff136
210 -2.862,201,4.9,7cfd91ddf780
211 -2.748,201,4.5,7cfd91ddf780
212 -2.637,201,4.6,7cfd91ddf780
213 -2.529,201,6.1,7cfd91ddf780
214 -2.413,201,5.2,5bbf259ff136
215 -2.302,201,5.6,5bbf259ff136
216 -2.187,201,4.9,5bbf259ff136
217 -2.072,201,5.5,7cfd91ddf780
218 -1.956,201,5.1,7cfd91ddf780
219 -1.849,201,5.2,7cfd91ddf780
220 -1.737,201,5.4,7cfd91ddf780
221 -1.625,201,5.3,7cfd91ddf780
222 -1.51,201,5.3,5bbf259ff136
223 -1.404,201,6.3,7cfd91ddf780
224 -1.291,201,5.5,5bbf259ff136
225 -1.185,201,5.8,7cfd91ddf780
226 -1.078,201,6.7,7cfd91ddf780
227 -0.964,201,7.0,5bbf259ff136
228 -0.855,201,5.7,7cfd91ddf780
229 -0.748,201,4.7,7cfd91ddf780
230 -0.642,201,5.7,7cfd91ddf780
231 -0.526,201,5.6,7cfd91ddf780
232 -0.42,201,5.1,7cfd91ddf780
233 -0.309,201,5.5,7cfd91ddf780
234 -0.199,201,5.0,7cfd91ddf780
235 -0.088,201,5.2,7cfd91ddf780
236 0.019,201,4.0,7cfd91ddf780
237 0.133,201,3.5,7cfd91ddf780
238 0.241,201,3.7,7cfd91ddf780
239 0.352,201,6.6,7cfd91ddf780
240 0.46,201,5.2,7cfd91ddf780
241 0.569,201,4.6,7cfd91ddf780
242 0.675,201,5.2,5bbf259ff136
243 0.79,201,4.4,7cfd91ddf780
244 0.896,201,4.5,5bbf259ff136
245 1.005,201,5.1,5bbf259ff136
246 1.118,201,5.4,5bbf259ff136
247 1.233,201,3.4,5bbf259ff136
248 1.343,201,4.1,5bbf259ff136
249 1.457,201,4.8,5bbf259ff136
250 1.565,201,4.2,5bbf259ff136
251 1.679,201,4.6,5bbf259ff136
252 1.785,201,6.0,5bbf259ff136
253 1.895,201,6.1,5bbf259ff136
254 2.011,201,4.3,5bbf259ff136
255 2.117,201,6.7,5bbf259ff136
256 2.234,201,4.1,5bbf259ff136
257 2.342,201,5.5,5bbf259ff136
258 2.457,201,6.2,5bbf259ff136
259 2.57,201,3.6,5bbf259ff136
260 2.675,201,4.6,5bbf259ff136
261 2.78,201,4.0,5bbf259ff136
262 2.892,201,3.2,5bbf259ff136
263 2.995,201,3.1,5bbf259ff136
264 3.106,201,6.7,5bbf259ff136
265 3.218,201,6.4,5bbf259ff136
266 3.334,201,6.6,5bbf259ff136
267 3.451,201,6.3,5bbf259ff136
268 3.563,201,5.6,5bbf259ff136
269 3.669,201,7.1,5bbf259ff136
270 3.78,201,4.6,5bbf259ff136
271 3.887,201,5.8,5bbf259ff136
272 4.003,201,6.2,5bbf259ff136
273 4.12,201,5.4,5bbf259ff136
274 4.234,201,5.0,5bbf259ff136
275 4.348,201,6.5,5bbf259ff136
276 4.458,201,6.0,5bbf259ff136
277 4.564,201,5.9,5bbf259ff136
278 4.674,201,5.4,5bbf259ff136
279 4.787,201,7.3,5bbf259ff136
280 4.9,201,5.7,5bbf259ff136
281 5.014,201,5.3,5bbf259ff136
282 5.126,201,5.2,5bbf259ff136
283 5.241,201,5.2,5bbf259ff136
284 5.352,201,3.0,5bbf259ff136
285 5.458,201,3.6,5bbf259ff136
286 5.566,201,2.8,5bbf259ff136
287 5.672,201,3.0,5bbf259ff136
288 5.776,201,4.3,5bbf259ff136
289 5.882,201,4.2,5bbf259ff136
290 5.99,201,5.6,5bbf259ff136
291 6.097,201,3.8,5bbf259ff136
292 6.204,201,5.5,5bbf259ff136
293 6.31,201,5.2,5bbf259ff136
294 6.421,201,3.4,5bbf259ff136
295 6.529,201,4.4,5bbf259ff136
296 6.636,201,3.7,5bbf259ff136
297 6.747,201,4.6,5bbf259ff136
298 6.861,201,4.7,5bbf259ff136
299 6.974,201,3.9,5bbf259ff136
300 7.081,201,5.7,5bbf259ff136
301 7.189,201,6.0,5bbf259ff136
302 7.302,201,4.3,5bbf259ff136
303 7.412,201,4.8,5bbf259ff136
304 7.522,201,4.8,5bbf259ff136
305 7.631,201,4.5,5bbf259ff136
306 7.739,201,6.3,5bbf259ff136
307 7.854,201,3.7,5bbf259ff136
308 7.959,201,6.1,5bbf259ff136
309 8.075,201,6.3,5bbf259ff136
310 8.191,201,5.6,5bbf259ff136
311 8.306,201,6.0,5bbf259ff136
312 8.422,201,5.4,5bbf259ff136
313 8.537,201,5.8,5bbf259ff136
314 8.651,201,5.3,5bbf259ff136
315 8.758,201,6.5,5bbf259ff136
316 8.869,201,5.5,5bbf259ff136
317 8.977,201,7.0,5bbf259ff136
318 9.094,201,4.9,5bbf259ff136
319 9.202,201,5.9,5bbf259ff136
320 9.313,201,5.8,5bbf259ff136
321 9.421,201,4.4,5bbf259ff136
322 9.526,201,6.7,5bbf259ff136
323 9.64,201,7.0,5bbf259ff136
324 9.753,201,7.7,5bbf259ff136
325 9.869,201,5.7,5bbf259ff136
326 9.979,201,5.9,5bbf259ff136
327 10.086,201,6.6,5bbf259ff136
328 10.194,201,4.8,5bbf259ff136
329 10.309,201,4.9,5bbf259ff136
330 10.419,201,3.7,5bbf259ff136
331 10.529,201,4.7,5bbf259ff136
332 10.64,201,5.3,5bbf259ff136
333 10.751,201,4.1,5bbf259ff136
334 10.862,201,4.6,5bbf259ff136
335 10.968,201,6.9,5bbf259ff136
336 11.084,201,5.0,5bbf259ff136
337 11.191,201,4.0,5bbf259ff136
338 11.296,201,7.0,5bbf259ff136
339 11.411,201,6.1,5bbf259ff136
340 11.527,201,6.0,5bbf259ff136
341 11.641,201,4.6,5bbf259ff136
342 11.753,201,4.7,5bbf259ff136
343 11.868,201,5.1,5bbf259ff136
344 11.976,201,6.4,5bbf259ff136
345 12.088,201,4.8,5bbf259ff136
346 12.199,201,3.8,5bbf259ff136
347 12.313,201,4.7,5bbf259ff136
348 12.424,201,4.9,5bbf259ff136
349 12.535,201,4.9,5bbf259ff136
350 12.65,201,5.2,5bbf259ff136
351 12.762,201,4.6,5bbf259ff136
352 12.869,201,4.6,5bbf259ff136
353 12.977,201,3.8,5bbf259ff136
354 13.089,201,3.6,5bbf259ff136
355 13.196,201,5.8,5bbf259ff136
356 13.304,201,4.5,5bbf259ff136
357 13.413,201,4.0,5bbf259ff136
358 13.526,201,5.2,5bbf259ff136
359 13.633,201,5.3,5bbf259ff136
360 13.747,201,3.9,5bbf259ff136
361 13.859,201,3.8,5bbf259ff136
362 13.969,201,3.9,5bbf259ff136
363 14.076,201,3.7,5bbf259ff136
364 14.184,201,3.5,5bbf259ff136
365 14.296,201,5.1,5bbf259ff136
366 14.409,201,4.4,5bbf259ff136
367 14.519,201,7.0,5bbf259ff136
368 14.632,201,6.5,5bbf259ff136
369 14.746,201,6.7,5bbf259ff136
370 14.855,201,5.9,5bbf259ff136
371 14.968,201,11.9,7cfd91ddf780
372 15.088,201,5.6,5bbf259ff136
373 15.203,201,4.0,5bbf259ff136
374 15.31,201,4.9,5bbf259ff136
375 15.423,201,6.8,5bbf259ff136
376 15.538,201,4.5,7cfd91ddf780
377 15.646,201,5.7,7cfd91ddf780
378 15.758,201,7.3,7cfd91ddf780
379 15.868,201,8.8,7cfd91ddf780
380 15.987,201,3.4,5bbf259ff136
381 16.1,201,4.6,5bbf259ff136
382 16.207,201,4.9,5bbf259ff136
383 16.319,201,4.9,5bbf259ff136
384 16.432,201,4.5,5bbf259ff136
385 16.538,201,4.7,5bbf259ff136
386 16.651,201,6.0,5bbf259ff136
387 16.763,201,4.9,5bbf259ff136
388 16.874,201,4.4,5bbf259ff136
389 16.979,201,4.1,5bbf259ff136
390 17.09,201,5.1,5bbf259ff136
391 17.197,201,5.2,5bbf259ff136
392 17.312,201,5.9,5bbf259ff136
393 17.426,201,5.1,5bbf259ff136
394 17.541,201,5.4,7cfd91ddf780
395 17.652,201,5.2,7cfd91ddf780
396 17.761,201,5.1,7cfd91ddf780
397 17.877,201,6.2,5bbf259ff136
398 17.993,201,6.3,5bbf259ff136
399 18.109,201,6.2,7cfd91ddf780
400 18.218,201,5.0,7cfd91ddf780
401 18.334,201,5.3,7cfd91ddf780
402 18.442,201,5.2,7cfd91ddf780
403 18.55,201,5.5,7cfd91ddf780
404 18.663,201,5.2,7cfd91ddf780
405 18.775,201,5.8,5bbf259ff136
406 18.891,201,4.3,5bbf259ff136
407 19.006,201,11.6,7cfd91ddf780
408 19.127,201,4.9,5bbf259ff136
409 19.241,201,6.1,5bbf259ff136
410 19.35,201,5.0,5bbf259ff136
411 19.463,201,4.9,5bbf259ff136
412 19.571,201,5.1,5bbf259ff136
413 19.682,201,4.7,5bbf259ff136
414 19.795,201,5.0,5bbf259ff136
415 19.91,201,4.4,7cfd91ddf780
416 20.02,201,6.2,7cfd91ddf780
417 20.129,201,5.4,7cfd91ddf780
418 20.244,201,5.8,7cfd91ddf780
419 20.354,201,5.2,7cfd91ddf780
420 20.468,201,5.6,5bbf259ff136
421 20.579,201,7.0,5bbf259ff136
422 20.686,201,4.7,7cfd91ddf780
423 20.793,201,4.9,7cfd91ddf780
424 20.901,201,5.0,7cfd91ddf780
425 21.009,201,3.9,7cfd91ddf780
426 21.121,201,4.9,7cfd91ddf780
427 21.229,201,5.9,7cfd91ddf780
428 21.337,201,4.9,7cfd91ddf780
429 21.446,201,5.4,7cfd91ddf780
430 21.562,201,6.4,7cfd91ddf780
431 21.671,201,6.0,7cfd91ddf780
432 21.787,201,5.8,5bbf259ff136
433 21.893,201,4.9,7cfd91ddf780
434 22.007,201,5.0,7cfd91ddf780
435 22.116,201,5.8,5bbf259ff136
436 22.222,201,5.2,7cfd91ddf780
437 22.329,201,4.9,7cfd91ddf780
438 22.438,201,5.8,7cfd91ddf780
439 22.553,201,6.3,7cfd91ddf780
440 22.669,201,5.5,5bbf259ff136
441 22.785,201,6.3,7cfd91ddf780
442 22.896,201,5.5,5bbf259ff136
443 23.006,201,6.2,7cfd91ddf780
444 23.115,201,6.6,5bbf259ff136
445 23.232,201,5.3,7cfd91ddf780
446 23.347,201,6.7,5bbf259ff136
447 23.461,201,6.8,5bbf259ff136
448 23.578,201,4.1,7cfd91ddf780
449 23.687,201,6.1,7cfd91ddf780
450 23.802,201,6.8,7cfd91ddf780
451 23.917,201,6.0,5bbf259ff136
452 24.033,201,3.2,5bbf259ff136
453 24.145,201,3.9,5bbf259ff136
454 24.256,201,4.2,7cfd91ddf780
455 24.363,201,4.9,5bbf259ff136
456 24.468,201,5.0,5bbf259ff136
457 24.577,201,4.5,5bbf259ff136
458 24.687,201,5.2,7cfd91ddf780
459 24.797,201,5.3,5bbf259ff136
460 24.909,201,4.7,5bbf259ff136
461 25.021,201,5.0,5bbf259ff136
462 25.126,201,7.2,5bbf259ff136
463 25.239,201,6.0,5bbf259ff136
464 25.35,201,5.8,5bbf259ff136
465 25.463,201,6.2,5bbf259ff136
466 25.573,201,5.5,5bbf259ff136
467 25.687,201,5.2,7cfd91ddf780
468 25.797,201,5.8,7cfd91ddf780
469 25.912,201,5.2,7cfd91ddf780
470 26.021,201,4.8,7cfd91ddf780
471 26.131,201,3.5,7cfd91ddf780
472 26.235,201,3.7,5bbf259ff136
473 26.349,201,5.5,7cfd91ddf780
474 26.465,201,4.8,7cfd91ddf780
475 26.57,201,4.5,7cfd91ddf780
476 26.679,201,4.2,7cfd91ddf780
477 26.793,201,4.8,7cfd91ddf780
478 26.903,201,4.9,7cfd91ddf780
479 27.012,201,5.4,7cfd91ddf780
480 27.121,201,5.7,7cfd91ddf780
481 27.227,201,5.0,7cfd91ddf780
482 27.342,201,5.4,7cfd91ddf780
483 27.452,201,5.1,7cfd91ddf780
484 27.565,201,4.8,7cfd91ddf780
485 27.675,201,5.0,7cfd91ddf780
486 27.79,201,5.4,5bbf259ff136
487 27.904,201,5.8,5bbf259ff136
488 28.014,201,6.5,5bbf259ff136
489 28.131,201,5.5,7cfd91ddf780
490 28.246,201,5.7,5bbf259ff136
491 28.354,201,5.8,7cfd91ddf780
492 28.467,201,6.2,5bbf259ff136
493 28.581,201,5.6,5bbf259ff136
494 28.69,201,4.3,5bbf259ff136
495 28.795,201,5.1,7cfd91ddf780
496 28.907,201,5.1,5bbf259ff136
497 29.015,201,6.1,7cfd91ddf780
498 29.123,201,6.4,7cfd91ddf780
499 29.237,201,3.9,5bbf259ff136
500 29.349,201,4.2,5bbf259ff136
501 29.455,201,7.7,5bbf259ff136
502 29.569,201,7.2,5bbf259ff136
503 29.679,201,6.3,5bbf259ff136
504 29.794,201,7.5,5bbf259ff136
505 29.91,201,5.8,7cfd91ddf780
506 30.022,201,6.9,7cfd91ddf780
507 30.136,201,6.7,5bbf259ff136
508 30.252,201,6.3,5bbf259ff136
509 30.367,201,6.1,7cfd91ddf780
510 30.479,201,6.8,7cfd91ddf780
511 30.595,201,4.9,7cfd91ddf780
512 30.703,201,5.1,7cfd91ddf780
513 30.813,201,4.7,7cfd91ddf780
514 30.924,201,5.3,7cfd91ddf780
515 31.032,201,5.1,7cfd91ddf780
516 31.143,201,4.5,7cfd91ddf780
517 31.257,201,4.8,5bbf259ff136
518 31.371,201,3.3,5bbf259ff136
519 31.477,201,4.6,5bbf259ff136
520 31.586,201,3.6,5bbf259ff136
521 31.699,201,4.4,5bbf259ff136
522 31.813,201,4.8,5bbf259ff136
523 31.927,201,4.6,7cfd91ddf780
524 32.042,201,5.3,7cfd91ddf780
525 32.158,201,5.0,5bbf259ff136
526 32.27,201,4.2,5bbf259ff136
527 32.378,201,5.6,5bbf259ff136
528 32.487,201,5.7,5bbf259ff136
529 32.603,201,4.3,5bbf259ff136
530 32.712,201,6.5,5bbf259ff136
531 32.828,201,6.7,7cfd91ddf780
532 32.945,201,5.3,7cfd91ddf780
533 33.054,201,5.8,7cfd91ddf780
534 33.168,201,6.3,5bbf259ff136
535 33.279,201,8.0,5bbf259ff136
536 33.393,201,5.6,5bbf259ff136
537 33.503,201,4.7,5bbf259ff136
538 33.616,201,6.1,7cfd91ddf780
539 33.731,201,4.7,7cfd91ddf780
540 33.845,201,7.1,7cfd91ddf780
541 33.954,201,5.1,7cfd91ddf780
542 34.069,201,5.8,5bbf259ff136
543 34.179,201,5.2,5bbf259ff136
544 34.288,201,4.7,5bbf259ff136
545 34.399,201,3.1,5bbf259ff136
546 34.512,201,3.5,5bbf259ff136
547 34.625,201,4.0,5bbf259ff136
548 34.733,201,4.6,5bbf259ff136
549 34.848,201,5.8,7cfd91ddf780
550 34.957,201,4.2,7cfd91ddf780
551 35.062,201,6.9,5bbf259ff136
552 35.169,201,5.2,5bbf259ff136
553 35.282,201,6.0,5bbf259ff136
554 35.395,201,5.3,5bbf259ff136
555 35.503,201,5.2,5bbf259ff136
556 35.614,201,5.8,5bbf259ff136
557 35.72,201,4.9,5bbf259ff136
558 35.83,201,6.2,5bbf259ff136
559 35.938,201,5.5,5bbf259ff136
560 36.05,201,6.1,5bbf259ff136
561 36.159,201,5.0,5bbf259ff136
562 36.273,201,5.2,5bbf259ff136
563 36.387,201,6.5,7cfd91ddf780
564 36.495,201,8.3,5bbf259ff136
565 36.611,201,3.5,7cfd91ddf780
566 36.716,201,4.1,7cfd91ddf780
567 36.823,201,7.0,5bbf259ff136
568 36.937,201,4.8,7cfd91ddf780
569 37.046,201,6.6,7cfd91ddf780
570 37.16,201,5.2,7cfd91ddf780
571 37.271,201,4.7,7cfd91ddf780
572 37.378,201,5.3,7cfd91ddf780
573 37.487,201,6.1,7cfd91ddf780
574 37.596,201,4.5,7cfd91ddf780
575 37.71,201,4.0,7cfd91ddf780
576 37.821,201,5.1,7cfd91ddf780
577 37.927,201,4.4,7cfd91ddf780
578 38.037,201,6.3,7cfd91ddf780
579 38.145,201,6.1,5bbf259ff136
580 38.259,201,5.1,7cfd91ddf780
581 38.373,201,8.5,7cfd91ddf780
582 38.485,201,5.3,7cfd91ddf780
583 38.592,201,4.8,7cfd91ddf780
584 38.706,201,4.2,7cfd91ddf780
585 38.82,201,4.7,5bbf259ff136
586 38.933,201,5.9,5bbf259ff136
587 39.041,201,6.6,5bbf259ff136
588 39.155,201,6.8,7cfd91ddf780
589 39.269,201,5.0,7cfd91ddf780
590 39.379,201,5.5,7cfd91ddf780
591 39.49,201,11.5,7cfd91ddf780
592 39.605,201,6.4,5bbf259ff136
593 39.72,201,5.7,5bbf259ff136
594 39.828,201,6.5,5bbf259ff136
595 39.939,201,6.0,5bbf259ff136
596 40.054,201,5.0,5bbf259ff136
597 40.161,201,4.9,5bbf259ff136
598 40.27,201,5.6,5bbf259ff136
599 40.377,201,6.1,7cfd91ddf780
600 40.493,201,4.2,5bbf259ff136
601 40.604,201,6.2,5bbf259ff136
602 40.71,201,4.2,5bbf259ff136
603 40.817,201,4.2,5bbf259ff136
604 40.927,201,5.0,5bbf259ff136
605 41.039,201,5.1,5bbf259ff136
606 41.154,201,5.1,5bbf259ff136
607 41.264,201,4.9,5bbf259ff136
608 41.372,201,5.1,5bbf259ff136
609 41.487,201,6.8,7cfd91ddf780
610 41.597,201,3.7,7cfd91ddf780
611 41.711,201,3.9,7cfd91ddf780
612 41.824,201,4.3,7cfd91ddf780
613 41.939,201,5.0,5bbf259ff136
614 42.049,201,6.2,5bbf259ff136
615 42.16,201,5.7,5bbf259ff136
616 42.27,201,3.6,5bbf259ff136
617 42.378,201,3.7,5bbf259ff136
618 42.492,201,4.5,5bbf259ff136
619 42.599,201,4.3,5bbf259ff136
620 42.711,201,4.0,5bbf259ff136
621 42.821,201,6.6,5bbf259ff136
622 42.937,201,4.8,7cfd91ddf780
623 43.048,201,6.1,7cfd91ddf780
624 43.16,201,3.3,7cfd91ddf780
625 43.265,201,3.2,5bbf259ff136
626 43.37,201,3.5,5bbf259ff136
627 43.482,201,3.4,5bbf259ff136
628 43.59,201,3.1,5bbf259ff136
629 43.698,201,7.8,5bbf259ff136
630 43.816,201,7.4,7cfd91ddf780
631 43.928,201,5.0,7cfd91ddf780
632 44.036,201,5.0,7cfd91ddf780
633 44.149,201,5.4,7cfd91ddf780
634 44.264,201,6.0,7cfd91ddf780
635 44.371,201,5.1,7cfd91ddf780
636 44.481,201,4.8,7cfd91ddf780
637 44.595,201,3.4,7cfd91ddf780
638 44.7,201,3.2,7cfd91ddf780
639 44.812,201,2.8,7cfd91ddf780
640 44.925,201,4.3,7cfd91ddf780
641 45.034,201,5.7,5bbf259ff136
642 45.145,201,8.0,7cfd91ddf780
643 45.26,201,8.5,5bbf259ff136
644 45.369,201,8.2,7cfd91ddf780
645 45.487,201,11.7,5bbf259ff136
646 45.607,201,11.0,7cfd91ddf780
647 45.727,201,4.3,5bbf259ff136
@@ -0,0 +1,71 @@
{"date": "2026-11-05T19:18:13.470607+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"label": "7cfd91ddf780",
"action": [
"sh",
"-c",
"scripts/stop-one.sh && docker compose ps -a --format '{{.ID}} {{.Service}} {{.Status}}' api && { scripts/stand-usage.sh & sleep 10; wait; } && scripts/restore.sh"
],
"action_exit": 0,
"action_seconds": 15.204,
"action_finished_before_load_end": true,
"lb_up_before": 2,
"lb_up_min": 1,
"lb_samples_missing": 0,
"detection_seconds": 0.884,
"rejoin_seconds": 14.885,
"probe": {
"attempts": 645,
"failed_attempts": 0,
"first_failure_s": null,
"first_success_s": 0.023,
"recovery_s": 0.0,
"instances_before": {
"5bbf259ff136": 118,
"7cfd91ddf780": 115
},
"instances_during_outage": {
"5bbf259ff136": 126
},
"instances_after": {
"5bbf259ff136": 142,
"7cfd91ddf780": 132
}
},
"history": {
"confirmed": 645,
"stored": 645,
"exact_match": true,
"unique": true
},
"lb_counters": {
"wretr": 0,
"wredis": 0,
"econ": 0,
"eresp": 0,
"hrsp_5xx": 0
},
"k6": {
"requests": 6001,
"rps": 100.02,
"ok_rps": 100.02,
"failed_ratio": 0,
"dropped_iterations": 0,
"p50_ms": 2.97,
"p95_ms": 3.86,
"p99_ms": 4.47,
"exit": 0,
"rate": 100
},
"checks": {
"action_ok": true,
"load_errors_ok": true,
"load_delivered": true,
"lb_observed": true,
"write_recovery_ok": true,
"history_preserved": true,
"detected": true,
"target_excluded": true,
"rejoined": true
},
"pass": true
}
+142
View File
@@ -0,0 +1,142 @@
[{"date": "2026-11-05T19:18:13.470607+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"label": "7cfd91ddf780",
"action": [
"sh",
"-c",
"scripts/stop-one.sh && docker compose ps -a --format '{{.ID}} {{.Service}} {{.Status}}' api && { scripts/stand-usage.sh & sleep 10; wait; } && scripts/restore.sh"
],
"action_exit": 0,
"action_seconds": 15.204,
"action_finished_before_load_end": true,
"lb_up_before": 2,
"lb_up_min": 1,
"lb_samples_missing": 0,
"detection_seconds": 0.884,
"rejoin_seconds": 14.885,
"probe": {
"attempts": 645,
"failed_attempts": 0,
"first_failure_s": null,
"first_success_s": 0.023,
"recovery_s": 0.0,
"instances_before": {
"5bbf259ff136": 118,
"7cfd91ddf780": 115
},
"instances_during_outage": {
"5bbf259ff136": 126
},
"instances_after": {
"5bbf259ff136": 142,
"7cfd91ddf780": 132
}
},
"history": {
"confirmed": 645,
"stored": 645,
"exact_match": true,
"unique": true
},
"lb_counters": {
"wretr": 0,
"wredis": 0,
"econ": 0,
"eresp": 0,
"hrsp_5xx": 0
},
"k6": {
"requests": 6001,
"rps": 100.02,
"ok_rps": 100.02,
"failed_ratio": 0,
"dropped_iterations": 0,
"p50_ms": 2.97,
"p95_ms": 3.86,
"p99_ms": 4.47,
"exit": 0,
"rate": 100
},
"checks": {
"action_ok": true,
"load_errors_ok": true,
"load_delivered": true,
"lb_observed": true,
"write_recovery_ok": true,
"history_preserved": true,
"detected": true,
"target_excluded": true,
"rejoined": true
},
"pass": true
},
{"date": "2026-11-05T19:18:13.470607+00:00", "commit": "252e3e3ffeea5004cfd9dc342e686af1417a77a2",
"label": "5bbf259ff136",
"action": [
"sh",
"-c",
"scripts/stop-one.sh && docker compose ps -a --format '{{.ID}} {{.Service}} {{.Status}}' api && { scripts/stand-usage.sh & sleep 10; wait; } && scripts/restore.sh"
],
"action_exit": 0,
"action_seconds": 15.2,
"action_finished_before_load_end": true,
"lb_up_before": 2,
"lb_up_min": 1,
"lb_samples_missing": 0,
"detection_seconds": 1.498,
"rejoin_seconds": 14.99,
"probe": {
"attempts": 642,
"failed_attempts": 0,
"first_failure_s": null,
"first_success_s": 0.111,
"recovery_s": 0.0,
"instances_before": {
"7cfd91ddf780": 114,
"5bbf259ff136": 119
},
"instances_during_outage": {
"7cfd91ddf780": 121
},
"instances_after": {
"7cfd91ddf780": 149,
"5bbf259ff136": 124
}
},
"history": {
"confirmed": 642,
"stored": 642,
"exact_match": true,
"unique": true
},
"lb_counters": {
"wretr": 0,
"wredis": 0,
"econ": 0,
"eresp": 0,
"hrsp_5xx": 0
},
"k6": {
"requests": 6001,
"rps": 100.02,
"ok_rps": 100.02,
"failed_ratio": 0,
"dropped_iterations": 0,
"p50_ms": 2.98,
"p95_ms": 3.88,
"p99_ms": 4.53,
"exit": 0,
"rate": 100
},
"checks": {
"action_ok": true,
"load_errors_ok": true,
"load_delivered": true,
"lb_observed": true,
"write_recovery_ok": true,
"history_preserved": true,
"detected": true,
"target_excluded": true,
"rejoined": true
},
"pass": true
}]
+39
View File
@@ -0,0 +1,39 @@
date: 2026-10-30T02:45:26Z commit: e049c26
== before: api replicas (hostname = X-Instance-Id), state, StartedAt, FinishedAt
ff81f93e8b0d running started=2026-10-30T02:45:24.812924046Z finished=2026-10-30T02:45:12.063064055Z
fe75c6c37e5a running started=2026-10-30T02:44:57.104698047Z finished=0001-01-01T00:00:00Z
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18083"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 136, "failed_attempts": 0, "first_recovery_seconds": 2.682}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
== after: api replicas
ff81f93e8b0d running started=2026-10-30T02:45:24.812924046Z finished=2026-10-30T02:45:12.063064055Z
fe75c6c37e5a exited started=2026-10-30T02:44:57.104698047Z finished=2026-10-30T02:45:29.956667174Z
== stopped replica: fe75c6c37e5a; shutdown: signal, drain window with readiness 503, close, exit
{"event":"Handling signal: term","level":"info","ts":"2026-10-30T02:45:27.514055Z"}
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T02:45:27.518751Z"}
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T02:45:27.528773Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"efd64978c14445e3a460198e8ef66a32","duration_ms":0.358,"event":"request.handled","level":"info","ts":"2026-10-30T02:45:27.962664Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"53b0876587c14b2cbae1936781f79fc0","duration_ms":0.401,"event":"request.handled","level":"info","ts":"2026-10-30T02:45:28.465375Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"947ec9729b774819b5de906d12ad0f91","duration_ms":0.306,"event":"request.handled","level":"info","ts":"2026-10-30T02:45:29.005486Z"}
{"event":"Shutting down","level":"info","ts":"2026-10-30T02:45:29.520790Z"}
{"event":"Shutting down","level":"info","ts":"2026-10-30T02:45:29.529825Z"}
{"event":"service.stopped","level":"info","ts":"2026-10-30T02:45:29.623527Z"}
{"event":"Finished server process [14]","level":"info","ts":"2026-10-30T02:45:29.623728Z"}
{"event":"service.stopped","level":"info","ts":"2026-10-30T02:45:29.632649Z"}
{"event":"Finished server process [13]","level":"info","ts":"2026-10-30T02:45:29.632781Z"}
{"event":"Worker (pid:13) was sent SIGTERM!","level":"info","ts":"2026-10-30T02:45:29.635792Z"}
{"event":"Worker (pid:14) was sent SIGTERM!","level":"info","ts":"2026-10-30T02:45:29.635917Z"}
{"event":"Shutting down: Master","level":"info","ts":"2026-10-30T02:45:29.737995Z"}
$ scripts/restore.sh
Container mtusi2026lab3-api-1 Running
Container mtusi2026lab3-api-2 Starting
Container mtusi2026lab3-api-2 Started
Container mtusi2026lab3-api-1 Waiting
Container mtusi2026lab3-api-2 Waiting
Container mtusi2026lab3-api-1 Healthy
Container mtusi2026lab3-api-2 Healthy
== after restore
ff81f93e8b0d running started=2026-10-30T02:45:24.812924046Z finished=2026-10-30T02:45:12.063064055Z
fe75c6c37e5a running started=2026-10-30T02:45:42.870380597Z finished=2026-10-30T02:45:29.956667174Z
+38
View File
@@ -0,0 +1,38 @@
date: 2026-10-30T02:45:08Z commit: e049c26
== before: api replicas (hostname = X-Instance-Id), state, StartedAt, FinishedAt
ff81f93e8b0d running started=2026-10-30T02:44:57.018195381Z finished=0001-01-01T00:00:00Z
fe75c6c37e5a running started=2026-10-30T02:44:57.104698047Z finished=0001-01-01T00:00:00Z
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18083"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 132, "failed_attempts": 0, "first_recovery_seconds": 2.858}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
== after: api replicas
ff81f93e8b0d exited started=2026-10-30T02:44:57.018195381Z finished=2026-10-30T02:45:12.063064055Z
fe75c6c37e5a running started=2026-10-30T02:44:57.104698047Z finished=0001-01-01T00:00:00Z
== stopped replica: ff81f93e8b0d; shutdown: signal, drain window with readiness 503, close, exit
{"event":"Handling signal: term","level":"info","ts":"2026-10-30T02:45:09.656770Z"}
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T02:45:09.714534Z"}
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T02:45:09.722268Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"468d7db9c6d24e9797a43d901e9bdbb1","duration_ms":0.312,"event":"request.handled","level":"info","ts":"2026-10-30T02:45:10.589248Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"771ff57a02304724b1de9c40033c5157","duration_ms":0.304,"event":"request.handled","level":"info","ts":"2026-10-30T02:45:11.091968Z"}
{"event":"Shutting down","level":"info","ts":"2026-10-30T02:45:11.715907Z"}
{"event":"Shutting down","level":"info","ts":"2026-10-30T02:45:11.728294Z"}
{"event":"service.stopped","level":"info","ts":"2026-10-30T02:45:11.826861Z"}
{"event":"Finished server process [14]","level":"info","ts":"2026-10-30T02:45:11.827508Z"}
{"event":"service.stopped","level":"info","ts":"2026-10-30T02:45:11.835753Z"}
{"event":"Finished server process [13]","level":"info","ts":"2026-10-30T02:45:11.835868Z"}
{"event":"Worker (pid:13) was sent SIGTERM!","level":"info","ts":"2026-10-30T02:45:11.887319Z"}
{"event":"Worker (pid:14) was sent SIGTERM!","level":"info","ts":"2026-10-30T02:45:11.887404Z"}
{"event":"Shutting down: Master","level":"info","ts":"2026-10-30T02:45:11.990788Z"}
$ scripts/restore.sh
Container mtusi2026lab3-api-2 Running
Container mtusi2026lab3-api-1 Starting
Container mtusi2026lab3-api-1 Started
Container mtusi2026lab3-api-1 Waiting
Container mtusi2026lab3-api-2 Waiting
Container mtusi2026lab3-api-1 Healthy
Container mtusi2026lab3-api-2 Healthy
== after restore
ff81f93e8b0d running started=2026-10-30T02:45:24.812924046Z finished=2026-10-30T02:45:12.063064055Z
fe75c6c37e5a running started=2026-10-30T02:44:57.104698047Z finished=0001-01-01T00:00:00Z
-10
View File
@@ -1,10 +0,0 @@
{
"ok": true,
"scenario": "failover-upload",
"upload_instance": "70cfba801128",
"download_instance": "095cb9cc85bd",
"sha256_equal": true,
"hook_seconds": 2.877,
"download_ok_after_seconds": 2.886,
"restored": true
}
-39
View File
@@ -1,39 +0,0 @@
date: 2026-10-29T00:33:32Z commit: 0b85edf
== before: api replicas (hostname = X-Instance-Id), state, StartedAt, FinishedAt
095cb9cc85bd running started=2026-10-29T00:33:12.427394173Z finished=2026-10-29T00:33:03.853644169Z
70cfba801128 running started=2026-10-29T00:33:12.607365173Z finished=2026-10-29T00:33:03.666460502Z
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://localhost:18085"
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 133, "failed_attempts": 0, "first_recovery_seconds": 3.005}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
== after: api replicas
095cb9cc85bd running started=2026-10-29T00:33:12.427394173Z finished=2026-10-29T00:33:03.853644169Z
70cfba801128 exited started=2026-10-29T00:33:12.607365173Z finished=2026-10-29T00:33:39.052027754Z
== stopped replica: 70cfba801128; shutdown: signal, drain window with readiness 503, close, exit
{"event":"Handling signal: term","level":"info","ts":"2026-10-29T00:33:36.479119Z"}
{"event":"Draining for 2.0 s","level":"info","ts":"2026-10-29T00:33:36.486455Z"}
{"event":"Draining for 2.0 s","level":"info","ts":"2026-10-29T00:33:36.558375Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"b3d9118f93b94471b698d536dda11072","duration_ms":0.772,"event":"request.handled","level":"info","ts":"2026-10-29T00:33:37.011026Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"3c1324def7e74f3484c4fbd242f7f70b","duration_ms":0.655,"event":"request.handled","level":"info","ts":"2026-10-29T00:33:37.515877Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"e56c9504272b45a89b7372020eb80d15","duration_ms":0.733,"event":"request.handled","level":"info","ts":"2026-10-29T00:33:37.588547Z"}
{"event":"Shutting down","level":"info","ts":"2026-10-29T00:33:38.487910Z"}
{"event":"Shutting down","level":"info","ts":"2026-10-29T00:33:38.560300Z"}
{"event":"service.stopped","level":"info","ts":"2026-10-29T00:33:38.592638Z"}
{"event":"Finished server process [13]","level":"info","ts":"2026-10-29T00:33:38.592934Z"}
{"event":"Worker (pid:13) was sent SIGTERM!","level":"info","ts":"2026-10-29T00:33:38.624866Z"}
{"event":"service.stopped","level":"info","ts":"2026-10-29T00:33:38.664775Z"}
{"event":"Finished server process [14]","level":"info","ts":"2026-10-29T00:33:38.665027Z"}
{"event":"Worker (pid:14) was sent SIGTERM!","level":"info","ts":"2026-10-29T00:33:38.725950Z"}
{"event":"Shutting down: Master","level":"info","ts":"2026-10-29T00:33:38.834487Z"}
$ scripts/restore.sh
Container mtusi2026lab5-api-1 Running
Container mtusi2026lab5-api-2 Starting
Container mtusi2026lab5-api-2 Started
Container mtusi2026lab5-api-1 Waiting
Container mtusi2026lab5-api-2 Waiting
Container mtusi2026lab5-api-1 Healthy
Container mtusi2026lab5-api-2 Healthy
== after restore
095cb9cc85bd running started=2026-10-29T00:33:12.427394173Z finished=2026-10-29T00:33:03.853644169Z
70cfba801128 running started=2026-10-29T00:33:51.687259594Z finished=2026-10-29T00:33:39.052027754Z
+55
View File
@@ -0,0 +1,55 @@
date: 2026-11-05T18:55:40.945444+00:00 commit: 0a1f48bd73e5780c732356880c23fdb476bcff98
environment: COMPOSE_PROJECT_NAME=mtusi2026lab3-grade3-final-20261105 LB_HTTP_PORT=18083 LB_STATS_PORT=23083 API_REPLICAS=2 NO_PROXY=localhost,127.0.0.1,::1
$ make test-faults
BASE_URL="http://127.0.0.1:18083" .venv/bin/pytest -m stack -v -s student-tests/stack
============================= test session starts ==============================
platform darwin -- Python 3.14.5, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv/bin/python
cachedir: .pytest_cache
rootdir: <repo>
configfile: pyproject.toml
plugins: asyncio-1.4.0, anyio-4.15.1
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
collecting ... collected 12 items
student-tests/stack/test_faults.py::test_registration_race_yields_exactly_one_user гонка 50 регистраций: 201×1, 409×49
PASSED
student-tests/stack/test_faults.py::test_chat_creation_rolls_back_on_unknown_member чат с несуществующим участником: 400 invalid_request
PASSED
student-tests/stack/test_faults.py::test_wrong_password_and_unknown_user_get_same_basic_challenge неверный пароль: 401 за 0.013 с
неизвестный пользователь: 401 за 0.012 с
PASSED
student-tests/stack/test_faults.py::test_logout_then_token_is_dead logout: 204, затем 401, повтор 401
PASSED
student-tests/stack/test_faults.py::test_valkey_serves_repeated_checks 5 проверок токена: keyspace_hits 3 → 8
PASSED
student-tests/stack/test_faults.py::test_valkey_stopped_is_bypassed_but_logout_waits valkey остановлен, действующий токен: 200 за 0.014 с
valkey остановлен, неизвестный токен: 401 за 0.004 с
valkey остановлен: ready 200, вход 201, выход 503
valkey снова запущен: повторный выход 204
PASSED
student-tests/stack/test_faults.py::test_valkey_restart_keeps_cache_and_revocations рестарт valkey: ключей 6 → 6
рестарт valkey: живой 200, отозванный 401
PASSED
student-tests/stack/test_faults.py::test_valkey_flushall_loses_nothing FLUSHALL: живой 200, отозванный 401
PASSED
student-tests/stack/test_faults.py::test_postgres_stopped_gives_bounded_503_and_recovers postgres остановлен, ready: 503 за 0.004 с
postgres остановлен, me: 503 за 0.003 с
postgres остановлен, stranger: 503 за 0.003 с
postgres остановлен, login: 503 за 0.003 с
postgres остановлен, register: 503 за 0.014 с
live внутри реплики 200; LB исключил все реплики ещё через 0.86 с
live через LB без реплик: 503 за 0.000 с
postgres запущен: готовность через 2.0 с, прежний токен 200
postgres запущен: 40 регистраций 201 через 2 реплики без рестарта API
PASSED
student-tests/stack/test_faults.py::test_api_restart_keeps_session_without_extending_it рестарт api: готовность через 1.5 с, токен 200
expires_at при входе 2026-11-05T19:55:55.410073Z, в БД после рестарта 2026-11-05T19:55:55.410073Z
PASSED
student-tests/stack/test_faults.py::test_full_restart_keeps_session_until_logout restart.sh: готовность через 0.0 с, прежний токен 200, выход 204, затем 401
PASSED
student-tests/stack/test_faults.py::test_session_expires_after_restart_at_original_time срок сессии сокращён до 20 с, рестарт api: готовность через 2.8 с, до срока 200, после срока 401
PASSED
============================= 12 passed in 47.62s ==============================
exit_code: 0 elapsed_seconds: 48.183
@@ -1,55 +0,0 @@
# Адресная validation исправления памяти PNG
**PASS. Прежний major закрыт динамически.** Измерен HEAD `74c33160ac01618f70e9e1c6a121a5e45c3c7155`, runtime побайтно равен прошедшему review `6096aa7948b8b666aa3892397a96d044a55f02c6` (M). Реализация, конфигурация и нормативные файлы в этой фазе не менялись. Публикация не выполнялась и требует отдельной фазы.
Исходные четыре HTTP FAIL и Linux MemoryError на V сохранены в [предыдущей validation](../final-validation-2026-11-07/RESULTS.md). Они не переименованы в PASS: здесь новый образ и новые результаты после отдельного fix/re-review. От V изменён только renderer, поэтому три пригодных повтора track2 на D, PostgreSQL outage и delivery subset на V приняты без повторения. Решения по каждой серии — [load-validation.md](../load-validation.md); SHA, diff, 53 прежних и 28 fix-хешей — [audit.json](audit.json).
## Условия и методика
2026-11-07 по локальной дате; UTC команд — 2026-11-06. Darwin arm64, Docker 28.3.2, Linux aarch64, VM 16 CPU / 8 217 190 400 байт. Project `mtusi2026lab5`, API 18085, stats/UI 25085/25086, все published ports на loopback. Использованы сохранённые volumes, не чистый clone. Worker: 2 CPU / 1 GiB; renderer RLIMIT_AS=448 MiB, 30 с, concurrency=2. Значения не повышались. Образ и закреплённые версии — [build.txt](build.txt), [runtime.txt](runtime.txt), [conditions.json](conditions.json).
Снимки до/после и внутри серий не показывают чужих работающих контейнеров. Специальная фоновая нагрузка не запускалась. Load average хоста: 1.67/2.11/2.34 перед этапом, 2.62/2.65/2.53 перед проверками, 6.41/4.18/3.19 после. Непрерывного контроля хоста нет; throughput, RPS, p95 и новые SLO не заявляются. Длительности — диагностические монотонные интервалы конкретных команд/рендеров, не benchmark. [Условия после](conditions-after.json) подтверждают восстановленные 448 MiB/30 с/concurrency=2 и отсутствие OOM.
Linux-пробник запускает новые процессы с **импортами messenger.worker**, настоящим RLIMIT_AS и alarm=30 с на каждый рендер. Пять файлов проходят v1/v2, затем обычное изображение в том же процессе. Два дополнительных процесса одновременно рендерят RGB16; пересечение измерено монотонными отметками внутри render. Пробник использует те же четыре сохранённые PNG 8000×5000; пятый получается удалением только tRNS из grayscale16. Входные SHA сохранены. Реальный HTTP использует LB/API/relay/broker/worker и ту же конфигурацию. При concurrency=2 журнал БД подтверждает перекрытие claimed→done в обеих версиях.
После диагностических процессов worker пересоздан без изменения образа/лимитов для отдельного cgroup peak HTTP — [reset-worker-peak.txt](reset-worker-peak.txt). Это не повтор нагрузки. Ожидаемые отказы создаются только на двух новых заданиях: 0.01 с и 200 MiB. После каждого лимиты восстановлены и обычная загрузка достигает ready. Перед SQL/S3-инвариантами дождались штатного sweep. Старые данные и DLQ не удалялись.
## Команды и результаты
Команды выполнялись из корня репозитория. `source evidence/final-memory-validation-2026-11-07/env.sh` задаёт несекретные Compose-параметры. Обёртка `python3 scripts/capture_evidence.py <новый-файл> -- <команда>` сохраняет SHA, команду, exit code и elapsed; существующие результаты не перезаписывает.
| Команда / проверка | Результат | Первичные данные |
| --- | --- | --- |
| `make check validate lint` | PASS | [checks.txt](checks.txt) |
| `openspec validate --all --strict`; `openspec list --json` | 22 PASS, changes=[] | [openspec.txt](openspec.txt), [active-changes.txt](active-changes.txt) |
| `make student-test PYTEST_ARGS='-q --tb=short'` | 984 passed, 2 Linux-only skipped, 19 stack deselected; 171.26 с pytest. Пропуски не названы PASS; Linux-эквивалент выполнен отдельно | [student-test.txt](student-test.txt) |
| `docker compose build`; `docker compose up -d --wait`; `make test` | Сборка, миграции/готовность и публичный smoke PASS | [build.txt](build.txt), [up.txt](up.txt), [smoke.txt](smoke.txt) |
| `.venv/bin/python evidence/final-memory-validation-2026-11-07/linux-runner.py` | Пять 40 Мп PNG × v1/v2 и обычный render после каждого PASS; два параллельных RGB16 × v1/v2 PASS | [linux.txt](linux.txt), [linux.json](linux.json), [child](linux-child.py), [runner](linux-runner.py) |
| `PYTHONPATH=src .venv/bin/python evidence/final-memory-validation-2026-11-07/targeted.py png-boundary` | 8 сценариев и 8 инвариантов PASS; 11 ID, 9 ready, 2 ожидаемых failed, 59 S3-объектов | [команда](png-boundary.txt), [JSON](png-boundary.json), [обвязка](targeted.py) |
| `PYTHONPATH=src .venv/bin/python evidence/final-memory-validation-2026-11-07/targeted.py png-http` | 8 fixtures, 48 сравнений вариантов v1/v2 с независимым белым эталоном; 8 инвариантов PASS, 8 ID / 56 объектов | [команда](png-http.txt), [JSON](png-http.json) |
Граничные HTTP-проверки: четыре прежних tRNS и grayscale16 без tRNS достигают ready в v1/v2; исходник неизменен; SHA, длина, MIME и размеры вариантов согласованы с байтами. Белая область белая, соседний RGB16 с отличием младшего бита остаётся непрозрачным. Все 30 хешей пяти файлов × двух версий × трёх вариантов совпали между Linux и HTTP. Поздняя v1 и дубль v2 не изменили опубликованный снимок; по шесть строк вариантов на ID. Два одновременных HTTP-вложения также проходят обе версии. Малая матрица включает grayscale 1/2/4/8/16, RGB8/16 и RGB16 Adam7/filter=4/EXIF=6.
| Linux-файл | Peak RSS процесса, KiB | HTTP: обе версии |
| --- | ---: | --- |
| grayscale2 tRNS | 343 076 | ready |
| grayscale4 tRNS | 342 956 | ready |
| grayscale16 tRNS | 365 508 | ready |
| grayscale16 без tRNS | 365 492 | ready |
| RGB16 tRNS | 406 312 | ready |
При двух **реальных HTTP jobs** cgroup peak worker = **885 428 224 байт (844.41 MiB) / 1 GiB**, все memory.events=0. Диагностический Linux-запуск, включающий idle worker, управляющий процесс и два дополнительных дочерних процесса, дал другой peak: 1 026 285 568 байт / 1 GiB, также без OOM. Это разные условия; диагностический peak не выдаётся за память штатных HTTP jobs. RSS не равен RLIMIT_AS или cgroup memory.
Инварианты обеих HTTP-серий проверяют terminal, single_effect, no_ready_rollback, monotonic_publication, bytes_match_metadata, bounded_attempts, no_orphans и no_lost_202. После sweep нет staging/pending outbox/дублирующих эффектов для новых ID; S3 inventory согласован с БД. DLQ осталась 16: это сохранённые исторические/ожидаемые записи, не новые ошибки этой серии. Проверка scoped, не глобальная очистка.
## Завершение и ограничения
Свои контейнеры остановлены через `make down` без `-v`: [до остановки](final-containers.txt), [down](down.txt), [пустой project](stopped-containers.txt), [четыре volumes](retained-volumes.txt), [пустой общий running list](running-containers-after.txt). [Ретроспективный срез Docker events](container-events.json) ограничен последними 256 событиями daemon и подтверждает финальную остановку; он не используется как полная история промежуточных worker-stop.
Семь minor независимого review сохранены в [findings.json](findings.json) и ограничениях [REPORT](../../REPORT.md). Новых major не обнаружено. Потеря publisher confirm, все возможные отказы диска/сети и изолированная throughput-матрица не доказаны; выбран трек 2. Исходный major памяти закрывается только для проверенных сценариев и закреплённой версии Pillow.
Context7 вызван для Pillow, но вернул исчерпание месячной квоты. Использован официальный [Pillow: Image lifecycle](https://pillow.readthedocs.io/en/stable/reference/open_files.html): `Image.close()` уничтожает image core, контекстный менеджер сам core не уничтожает. Эта справка не заменяет выполненные Linux-проверки.
Перед коммитом проверены разрешённый scope, отсутствие секретов в tracked файлах, локальные ссылки и сохранность runtime/нормативных файлов; результаты — [final-audit.json](final-audit.json), [secret-scan.json](secret-scan.json), [final-checks.txt](final-checks.txt). Контрольные суммы новых артефактов — [sha256.json](sha256.json). Push и Telegram не выполнялись.
Полный `git diff --cached --check` обнаружил 17 trailing-whitespace строк в сыром выводе smoke и двух HTTP-команд. Вывод сохранён без редактирования; отдельная проверка документов/обвязки проходит. Это формат console output, не результат сценария приложения.
@@ -1,11 +0,0 @@
date: 2026-11-06T22:54:44Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {}
$ openspec list --json
{
"changes": [],
"root": {
"path": "<repo>",
"source": "nearest"
}
}
exit_code: 0 elapsed_seconds: 0.974
@@ -1,153 +0,0 @@
import ast
import hashlib
import json
import re
import subprocess
from pathlib import Path
ROOT = Path.cwd()
FOLDER = Path(__file__).resolve().parent.relative_to(ROOT)
MEASURED = "74c33160ac01618f70e9e1c6a121a5e45c3c7155"
REVIEWED = "6096aa7948b8b666aa3892397a96d044a55f02c6"
def git(*args):
return subprocess.check_output(["git", *args], text=True).strip()
assert git("rev-parse", "HEAD") == MEASURED
assert git("branch", "--show-current") == "solution/bpi2303-kostylev"
paths = git("diff", "--name-only", MEASURED).splitlines()
assert all(p in {"REPORT.md", "AGENTS.md"} or p.startswith("evidence/") for p in paths)
runtime_diff = git(
"diff",
"--name-only",
REVIEWED,
"--",
"src",
"infra",
"compose.yaml",
"Dockerfile",
"pyproject.toml",
"uv.lock",
"alembic",
"scripts",
"student-tests",
"openspec",
)
assert not runtime_diff
assert not git(
"diff", "--name-only", "upstream/main", "--", "contracts", "tests", "README.md", "COURSE.md"
)
subprocess.run(["git", "diff", "--check"], check=True)
whitespace = subprocess.run(["git", "diff", "--cached", "--check"], capture_output=True, text=True)
raw_whitespace = re.findall(r"^([^\n:]+):\d+: trailing whitespace\.$", whitespace.stdout, re.M)
assert whitespace.returncode in (0, 2)
assert set(raw_whitespace) <= {
str(FOLDER / "png-boundary.txt"),
str(FOLDER / "png-http.txt"),
str(FOLDER / "smoke.txt"),
}
subprocess.run(
["git", "diff", "--cached", "--check", "--", ".", f":(exclude){FOLDER}/*.txt"],
check=True,
)
tracked = set(git("ls-files").splitlines())
docs = [
Path("REPORT.md"),
Path("AGENTS.md"),
Path("evidence/README.md"),
Path("evidence/load-validation.md"),
FOLDER / "RESULTS.md",
]
links = 0
for doc in docs:
content = doc.read_text()
assert len(re.findall(r"^```", content, re.M)) % 2 == 0, doc
for target in re.findall(r"\]\(([^)]+)\)", content):
if re.match(r"(?:https?://|#)", target):
continue
destination = (doc.parent / target.split("#")[0]).resolve()
assert destination.exists(), (doc, target)
relative = str(destination.relative_to(ROOT))
assert relative in tracked or any(p.startswith(relative + "/") for p in tracked), relative
links += 1
json_files = 0
for name in tracked:
if name.endswith(".json"):
json.loads(Path(name).read_text())
json_files += 1
for path in FOLDER.glob("*.py"):
ast.parse(path.read_text(), filename=str(path))
hashes = 0
for manifest, key in [
("evidence/final-validation-2026-11-07/sha256.json", None),
("evidence/review-png-memory-2026-11-07/checks.json", "sha256"),
]:
entries = json.loads(Path(manifest).read_text())
if key:
entries = entries[key]
for name, expected in entries.items():
assert hashlib.sha256(Path(name).read_bytes()).hexdigest() == expected, name
hashes += 1
accepted = []
for name in [
"evidence/forward-final-2026-11-06/level4.json",
*[f"evidence/forward-final-2026-11-06/track-run{n}.json" for n in (1, 2, 3)],
"evidence/final-validation-2026-11-07/postgres-outage-retry.json",
"evidence/final-validation-2026-11-07/delivery-targeted.json",
]:
data = json.loads(Path(name).read_text())
scenarios = data.get("scenarios", data.get("cases"))
assert data["result"] == "PASS"
assert all(x["result"] == "PASS" for x in scenarios + data["invariants"])
accepted.append(
{
"file": name,
"source_sha": data["source_sha"],
"scenarios": len(scenarios),
"invariants": len(data["invariants"]),
}
)
own_containers = subprocess.check_output(
["docker", "ps", "-aq", "--filter", "label=com.docker.compose.project=mtusi2026lab5"], text=True
).strip()
assert not own_containers
volumes = subprocess.check_output(
[
"docker",
"volume",
"ls",
"--filter",
"label=com.docker.compose.project=mtusi2026lab5",
"--format",
"{{.Name}}",
],
text=True,
).splitlines()
assert len(volumes) == 4
report = {
"measured_sha": MEASURED,
"reviewed_runtime": REVIEWED,
"runtime_equal": True,
"normative_equal_to_upstream_main": True,
"changed_paths": paths,
"local_links_verified": links,
"tracked_json_parsed": json_files,
"previous_and_fix_hashes_verified": hashes,
"raw_diff_check_exit": whitespace.returncode,
"preserved_raw_trailing_whitespace_lines": len(raw_whitespace),
"docs_and_helpers_diff_check": "PASS",
"accepted_without_rerun": accepted,
"own_containers": [],
"retained_volumes": volumes,
"result": "PASS",
}
(FOLDER / "final-audit.json").write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n")
print(
json.dumps(
{k: v for k, v in report.items() if k not in {"changed_paths", "accepted_without_rerun"}},
ensure_ascii=False,
indent=2,
)
)
@@ -1,25 +0,0 @@
{
"head": "74c33160ac01618f70e9e1c6a121a5e45c3c7155",
"reviewed_runtime": "6096aa7948b8b666aa3892397a96d044a55f02c6",
"runtime_review_equal": true,
"runtime_diff_since_previous_validation": [
"src/messenger/attachments/infrastructure/images.py"
],
"previous_artifact_hashes_verified": 53,
"platform": "macOS-15.1-arm64-arm-64bit-Mach-O",
"host_load": [
1.66796875,
2.11279296875,
2.3447265625
],
"docker": "28.3.2 16 8217190400",
"running_containers_before": "",
"image_fixture_hashes": {
"rgb16-40mp.png": "3c4418a6f6a7676298b30b70ebbde9d12929f1345152f4fae979caf5ba1cb3f7",
"gray4-40mp.png": "64e665928c08ac3721324af4f8e2e9d2e98230dc1be389dde7bbf365650b08cd",
"gray2-40mp.png": "e1238de6a5be9dbce36abee904f7233f8ff7be59e405b528671940630cce11f6",
"gray16-40mp.png": "b59784ae916d50b41b15afa4bdd474a8699f099da55efd53580d3f97180f3800"
},
"fix_hashes_verified": 28,
"linux_http_variant_hashes_equal": 30
}
@@ -1,90 +0,0 @@
date: 2026-11-06T22:54:41Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ docker compose build
#1 [internal] load local bake definitions
#1 reading from stdin 441B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.65kB done
#2 DONE 0.0s
#3 [internal] load metadata for ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142
#3 DONE 0.0s
#4 [internal] load metadata for docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d
#4 DONE 1.6s
#5 [internal] load .dockerignore
#5 transferring context: 453B done
#5 DONE 0.0s
#6 [builder 1/7] FROM docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d
#6 resolve docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d 0.0s done
#6 DONE 0.0s
#7 FROM ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142
#7 resolve ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 0.0s done
#7 DONE 0.0s
#8 [internal] load build context
#8 transferring context: 55.94kB 0.0s done
#8 DONE 0.0s
#9 [builder 4/7] COPY pyproject.toml uv.lock README.md ./
#9 CACHED
#10 [builder 3/7] WORKDIR /build
#10 CACHED
#11 [builder 2/7] COPY --from=ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 /uv /usr/local/bin/uv
#11 CACHED
#12 [builder 5/7] RUN uv sync --locked --no-dev --no-install-project --no-editable
#12 CACHED
#13 [builder 6/7] COPY src ./src
#13 DONE 0.1s
#14 [builder 7/7] RUN uv sync --locked --no-dev --no-editable
#14 0.246 Resolved 84 packages in 9ms
#14 0.261 Building mtusi-messenger @ file:///build
#14 3.076 Built mtusi-messenger @ file:///build
#14 3.081 Prepared 1 package in 2.82s
#14 3.086 Installed 1 package in 4ms
#14 3.306 Bytecode compiled 2301 files in 220ms
#14 3.306 + mtusi-messenger==1.5.0 (from file:///build)
#14 DONE 3.3s
#15 [runtime 2/6] RUN groupadd --system --gid 10001 messenger && useradd --system --uid 10001 --gid messenger --no-create-home --shell /usr/sbin/nologin messenger
#15 CACHED
#16 [runtime 3/6] COPY --from=builder --chown=root:root /opt/venv /opt/venv
#16 DONE 0.7s
#17 [runtime 4/6] WORKDIR /app
#17 DONE 0.0s
#18 [runtime 5/6] COPY --chown=root:root alembic.ini ./
#18 DONE 0.0s
#19 [runtime 6/6] COPY --chown=root:root alembic ./alembic
#19 DONE 0.0s
#20 exporting to image
#20 exporting layers
#20 exporting layers 3.1s done
#20 exporting manifest sha256:822ff6808de5ef24f7f560bfbf7d074f03882cf60f79ec17b72ec20b27fcb882
#20 exporting manifest sha256:822ff6808de5ef24f7f560bfbf7d074f03882cf60f79ec17b72ec20b27fcb882 done
#20 exporting config sha256:9ad245995fe138e23bc22e352d3a77ee0bf65ae6979997c6da067bd8d5315782 done
#20 exporting attestation manifest sha256:cf7e1da74a249e6859a38405addb0350aab512b93f6e62e66405ae1936db4e38 done
#20 exporting manifest list sha256:9a4299f2f673c414d67c3cc1f1debf4a56d9358f4a1ae85703152d48e8ef1608 done
#20 naming to docker.io/library/mtusi-messenger:lab5-memory-validation done
#20 unpacking to docker.io/library/mtusi-messenger:lab5-memory-validation
#20 unpacking to docker.io/library/mtusi-messenger:lab5-memory-validation 0.6s done
#20 DONE 3.8s
#21 resolving provenance for metadata file
#21 DONE 0.0s
api Built
exit_code: 0 elapsed_seconds: 14.282
@@ -1,14 +0,0 @@
date: 2026-11-06T22:54:42Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {}
$ make check validate lint
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
.venv/bin/python scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
.venv/bin/ruff check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py scripts/make_images.py scripts/pipeline_suite.py scripts/pipeline_evidence.py
All checks passed!
.venv/bin/ruff format --check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py scripts/make_images.py scripts/pipeline_suite.py scripts/pipeline_evidence.py
198 files already formatted
.venv/bin/ty check src student-tests alembic
All checks passed!
exit_code: 0 elapsed_seconds: 0.650
@@ -1,11 +0,0 @@
{
"at": "2026-11-06T23:18:53.248018+00:00",
"host_load": [
6.40869140625,
4.1806640625,
3.1875
],
"running": "mtusi2026lab5-worker-1 Up About a minute (healthy)\nmtusi2026lab5-api-1 Up 6 minutes (healthy)\nmtusi2026lab5-api-2 Up 6 minutes (healthy)\nmtusi2026lab5-relay-1 Up 6 minutes (healthy)\nmtusi2026lab5-broker-1 Up 6 minutes (healthy)\nmtusi2026lab5-postgres-1 Up 6 minutes (healthy)\nmtusi2026lab5-lb-1 Up 6 minutes (healthy)\nmtusi2026lab5-s3-1 Up 6 minutes (healthy)\nmtusi2026lab5-valkey-1 Up 6 minutes (healthy)",
"worker_limits": "448\n30\n2",
"memory_events": "low 0\nhigh 0\nmax 0\noom 0\noom_kill 0\noom_group_kill 0"
}
@@ -1,166 +0,0 @@
{
"platform": "macOS-15.1-arm64-arm-64bit-Mach-O",
"host_load": [
2.62109375,
2.6455078125,
2.53076171875
],
"running": "mtusi2026lab5-api-1 Up 2 minutes (healthy)\nmtusi2026lab5-api-2 Up 2 minutes (healthy)\nmtusi2026lab5-worker-1 Up 2 minutes (healthy)\nmtusi2026lab5-relay-1 Up 2 minutes (healthy)\nmtusi2026lab5-broker-1 Up 2 minutes (healthy)\nmtusi2026lab5-postgres-1 Up 2 minutes (healthy)\nmtusi2026lab5-lb-1 Up 2 minutes (healthy)\nmtusi2026lab5-s3-1 Up 2 minutes (healthy)\nmtusi2026lab5-valkey-1 Up 2 minutes (healthy)",
"containers": [
{
"name": "/mtusi2026lab5-api-1",
"id": "d1dcbda238669764840b38a8b9cf22d889ffc356945e165f019c4814bb96553d",
"image": "sha256:9a4299f2f673c414d67c3cc1f1debf4a56d9358f4a1ae85703152d48e8ef1608",
"limits": {
"Memory": 536870912,
"NanoCpus": 1000000000,
"PortBindings": {}
},
"volumes": []
},
{
"name": "/mtusi2026lab5-api-2",
"id": "1eaf0fe05de650a1c2f7ced3e360aabbf8ad59566f40beefc2d745fa5277688e",
"image": "sha256:9a4299f2f673c414d67c3cc1f1debf4a56d9358f4a1ae85703152d48e8ef1608",
"limits": {
"Memory": 536870912,
"NanoCpus": 1000000000,
"PortBindings": {}
},
"volumes": []
},
{
"name": "/mtusi2026lab5-worker-1",
"id": "a9afb0e9cc245a54eb871ddd44fcf8661949789de59603ded5236b5f9e9f1d7c",
"image": "sha256:9a4299f2f673c414d67c3cc1f1debf4a56d9358f4a1ae85703152d48e8ef1608",
"limits": {
"Memory": 1073741824,
"NanoCpus": 2000000000,
"PortBindings": {}
},
"volumes": []
},
{
"name": "/mtusi2026lab5-relay-1",
"id": "63225b5eff8e1eb2870376c487b8925f9dbbe9bb8a636f4e2cdc12a24becf577",
"image": "sha256:9a4299f2f673c414d67c3cc1f1debf4a56d9358f4a1ae85703152d48e8ef1608",
"limits": {
"Memory": 201326592,
"NanoCpus": 500000000,
"PortBindings": {}
},
"volumes": []
},
{
"name": "/mtusi2026lab5-migrate-1",
"id": "34901a09ef7b837c6ce87116d2035d842762d2c86f8562cade96d3915a80e6f0",
"image": "sha256:9a4299f2f673c414d67c3cc1f1debf4a56d9358f4a1ae85703152d48e8ef1608",
"limits": {
"Memory": 268435456,
"NanoCpus": 500000000,
"PortBindings": {}
},
"volumes": []
},
{
"name": "/mtusi2026lab5-s3-init-1",
"id": "6e80e83ed5c5470bf22c5a9cd7c835788fbdff94f1f28f8649df6b667c153f89",
"image": "sha256:ab024bfebee49a750ce886b4c70963ccd9ddaa03f491704a90710641d7a26699",
"limits": {
"Memory": 67108864,
"NanoCpus": 500000000,
"PortBindings": {}
},
"volumes": []
},
{
"name": "/mtusi2026lab5-broker-1",
"id": "be1942ad69b426f8b7c9189b0209021b169c6c8d0ee633f302d6d659a4143377",
"image": "sha256:54ef5ff505352c1beef05a97ee9bd6ab95b3559a76629ba0119b1d63d8d6e4b2",
"limits": {
"Memory": 536870912,
"NanoCpus": 1000000000,
"PortBindings": {
"15672/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "25086"
}
]
}
},
"volumes": [
"mtusi2026lab5_broker-data"
]
},
{
"name": "/mtusi2026lab5-postgres-1",
"id": "a0b9f2742b9737ac0e32423d767438cab12af55a00cc1d68515ca9f07c2758ed",
"image": "sha256:266d3c257594e195b309ae9d1805e54c788f1d7b375f56bf6c367aac6c917e7b",
"limits": {
"Memory": 1073741824,
"NanoCpus": 2000000000,
"PortBindings": {}
},
"volumes": [
"mtusi2026lab5_pgdata"
]
},
{
"name": "/mtusi2026lab5-lb-1",
"id": "4ddcc75182e6044288c7a8c7bddca7b8c5fc6dab9b18c429e61ab1d292680111",
"image": "sha256:5893840576071e6217cd52ebdcc14f454c6f7a631056fa6f4e0f40d3cac4215b",
"limits": {
"Memory": 134217728,
"NanoCpus": 1000000000,
"PortBindings": {
"8080/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "18085"
}
],
"8404/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "25085"
}
]
}
},
"volumes": []
},
{
"name": "/mtusi2026lab5-s3-1",
"id": "29b3e7c3a260f035ba99f5ce5f3e8a8b7077c56377b17e25503f16fb7fbad7c3",
"image": "sha256:ed9210d6411d77ffceb70d94b964a3cb7d225983368bc25bebe01e995293dcc2",
"limits": {
"Memory": 536870912,
"NanoCpus": 1000000000,
"PortBindings": {}
},
"volumes": [
"mtusi2026lab5_s3data"
]
},
{
"name": "/mtusi2026lab5-valkey-1",
"id": "647882797ca28a7e07453f00bd198433c80d8a8b8e7013279bc8954eafe11db6",
"image": "sha256:2602ace2be6569bc2bbb06f873776516ab9089821525be34f4305c67eb9db02a",
"limits": {
"Memory": 134217728,
"NanoCpus": 500000000,
"PortBindings": {}
},
"volumes": [
"mtusi2026lab5_valkeydata"
]
}
],
"fixture_sha256": {
"rgb16-40mp.png": "3c4418a6f6a7676298b30b70ebbde9d12929f1345152f4fae979caf5ba1cb3f7",
"gray4-40mp.png": "64e665928c08ac3721324af4f8e2e9d2e98230dc1be389dde7bbf365650b08cd",
"gray2-40mp.png": "e1238de6a5be9dbce36abee904f7233f8ff7be59e405b528671940630cce11f6",
"gray16-40mp.png": "b59784ae916d50b41b15afa4bdd474a8699f099da55efd53580d3f97180f3800"
}
}
@@ -1,151 +0,0 @@
{
"command": [
"docker",
"events",
"--since",
"2026-11-06T23:15:45Z",
"--until",
"2026-11-06T23:19:06.697928+00:00",
"--filter",
"type=container",
"--filter",
"label=com.docker.compose.project=mtusi2026lab5",
"--filter",
"event=stop",
"--filter",
"event=die",
"--filter",
"event=start",
"--format",
"{{json .}}"
],
"events": [
{
"timeNano": 1794007134212820139,
"id": "4ddcc75182e6044288c7a8c7bddca7b8c5fc6dab9b18c429e61ab1d292680111",
"action": "stop",
"name": "mtusi2026lab5-lb-1",
"exitCode": null
},
{
"timeNano": 1794007134214724139,
"id": "4ddcc75182e6044288c7a8c7bddca7b8c5fc6dab9b18c429e61ab1d292680111",
"action": "die",
"name": "mtusi2026lab5-lb-1",
"exitCode": "0"
},
{
"timeNano": 1794007134242785764,
"id": "63225b5eff8e1eb2870376c487b8925f9dbbe9bb8a636f4e2cdc12a24becf577",
"action": "stop",
"name": "mtusi2026lab5-relay-1",
"exitCode": null
},
{
"timeNano": 1794007134243810347,
"id": "63225b5eff8e1eb2870376c487b8925f9dbbe9bb8a636f4e2cdc12a24becf577",
"action": "die",
"name": "mtusi2026lab5-relay-1",
"exitCode": "0"
},
{
"timeNano": 1794007134261625889,
"id": "cb275ee9574b376bec77f26bdc24d2149568a272094f374b866d89d42b8067ce",
"action": "stop",
"name": "mtusi2026lab5-worker-1",
"exitCode": null
},
{
"timeNano": 1794007134263340222,
"id": "cb275ee9574b376bec77f26bdc24d2149568a272094f374b866d89d42b8067ce",
"action": "die",
"name": "mtusi2026lab5-worker-1",
"exitCode": "0"
},
{
"timeNano": 1794007135530081667,
"id": "be1942ad69b426f8b7c9189b0209021b169c6c8d0ee633f302d6d659a4143377",
"action": "stop",
"name": "mtusi2026lab5-broker-1",
"exitCode": null
},
{
"timeNano": 1794007135532395875,
"id": "be1942ad69b426f8b7c9189b0209021b169c6c8d0ee633f302d6d659a4143377",
"action": "die",
"name": "mtusi2026lab5-broker-1",
"exitCode": "0"
},
{
"timeNano": 1794007136578760167,
"id": "d1dcbda238669764840b38a8b9cf22d889ffc356945e165f019c4814bb96553d",
"action": "stop",
"name": "mtusi2026lab5-api-1",
"exitCode": null
},
{
"timeNano": 1794007136581296250,
"id": "d1dcbda238669764840b38a8b9cf22d889ffc356945e165f019c4814bb96553d",
"action": "die",
"name": "mtusi2026lab5-api-1",
"exitCode": "0"
},
{
"timeNano": 1794007136606883084,
"id": "1eaf0fe05de650a1c2f7ced3e360aabbf8ad59566f40beefc2d745fa5277688e",
"action": "stop",
"name": "mtusi2026lab5-api-2",
"exitCode": null
},
{
"timeNano": 1794007136610714375,
"id": "1eaf0fe05de650a1c2f7ced3e360aabbf8ad59566f40beefc2d745fa5277688e",
"action": "die",
"name": "mtusi2026lab5-api-2",
"exitCode": "0"
},
{
"timeNano": 1794007136730887167,
"id": "647882797ca28a7e07453f00bd198433c80d8a8b8e7013279bc8954eafe11db6",
"action": "stop",
"name": "mtusi2026lab5-valkey-1",
"exitCode": null
},
{
"timeNano": 1794007136732779084,
"id": "647882797ca28a7e07453f00bd198433c80d8a8b8e7013279bc8954eafe11db6",
"action": "die",
"name": "mtusi2026lab5-valkey-1",
"exitCode": "0"
},
{
"timeNano": 1794007136795874292,
"id": "a0b9f2742b9737ac0e32423d767438cab12af55a00cc1d68515ca9f07c2758ed",
"action": "stop",
"name": "mtusi2026lab5-postgres-1",
"exitCode": null
},
{
"timeNano": 1794007136796859125,
"id": "a0b9f2742b9737ac0e32423d767438cab12af55a00cc1d68515ca9f07c2758ed",
"action": "die",
"name": "mtusi2026lab5-postgres-1",
"exitCode": "0"
},
{
"timeNano": 1794007136819412875,
"id": "29b3e7c3a260f035ba99f5ce5f3e8a8b7077c56377b17e25503f16fb7fbad7c3",
"action": "stop",
"name": "mtusi2026lab5-s3-1",
"exitCode": null
},
{
"timeNano": 1794007136823326750,
"id": "29b3e7c3a260f035ba99f5ce5f3e8a8b7077c56377b17e25503f16fb7fbad7c3",
"action": "die",
"name": "mtusi2026lab5-s3-1",
"exitCode": "0"
}
],
"limitation": "Docker retains only the last 256 events; this is a partial retrospective snapshot."
}
@@ -1,53 +0,0 @@
date: 2026-11-06T23:18:53Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ make down
docker compose --profile bench --profile tools --profile reconcile down
Container mtusi2026lab5-relay-1 Stopping
Container mtusi2026lab5-lb-1 Stopping
Container mtusi2026lab5-api-2 Stopping
Container mtusi2026lab5-api-1 Stopping
Container mtusi2026lab5-worker-1 Stopping
Container mtusi2026lab5-lb-1 Stopped
Container mtusi2026lab5-lb-1 Removing
Container mtusi2026lab5-relay-1 Stopped
Container mtusi2026lab5-relay-1 Removing
Container mtusi2026lab5-lb-1 Removed
Container mtusi2026lab5-worker-1 Stopped
Container mtusi2026lab5-worker-1 Removing
Container mtusi2026lab5-relay-1 Removed
Container mtusi2026lab5-worker-1 Removed
Container mtusi2026lab5-broker-1 Stopping
Container mtusi2026lab5-broker-1 Stopped
Container mtusi2026lab5-broker-1 Removing
Container mtusi2026lab5-broker-1 Removed
Container mtusi2026lab5-api-1 Stopped
Container mtusi2026lab5-api-1 Removing
Container mtusi2026lab5-api-2 Stopped
Container mtusi2026lab5-api-2 Removing
Container mtusi2026lab5-api-1 Removed
Container mtusi2026lab5-api-2 Removed
Container mtusi2026lab5-valkey-1 Stopping
Container mtusi2026lab5-migrate-1 Stopping
Container mtusi2026lab5-s3-init-1 Stopping
Container mtusi2026lab5-migrate-1 Stopped
Container mtusi2026lab5-migrate-1 Removing
Container mtusi2026lab5-s3-init-1 Stopped
Container mtusi2026lab5-s3-init-1 Removing
Container mtusi2026lab5-migrate-1 Removed
Container mtusi2026lab5-postgres-1 Stopping
Container mtusi2026lab5-s3-init-1 Removed
Container mtusi2026lab5-s3-1 Stopping
Container mtusi2026lab5-valkey-1 Stopped
Container mtusi2026lab5-valkey-1 Removing
Container mtusi2026lab5-valkey-1 Removed
Container mtusi2026lab5-postgres-1 Stopped
Container mtusi2026lab5-postgres-1 Removing
Container mtusi2026lab5-s3-1 Stopped
Container mtusi2026lab5-s3-1 Removing
Container mtusi2026lab5-postgres-1 Removed
Container mtusi2026lab5-s3-1 Removed
Network mtusi2026lab5_edge Removing
Network mtusi2026lab5_backend Removing
Network mtusi2026lab5_backend Removed
Network mtusi2026lab5_edge Removed
exit_code: 0 elapsed_seconds: 3.356
@@ -1,8 +0,0 @@
export COMPOSE_PROJECT_NAME=mtusi2026lab5
export COMPOSE_FILE=compose.yaml
export API_IMAGE=mtusi-messenger:lab5-memory-validation
export LB_HTTP_PORT=18085
export LB_STATS_PORT=25085
export BROKER_MANAGEMENT_PORT=25086
export BASE_URL=http://127.0.0.1:18085
export NO_PROXY=localhost,127.0.0.1,::1
@@ -1,100 +0,0 @@
{
"measured_sha": "74c33160ac01618f70e9e1c6a121a5e45c3c7155",
"reviewed_runtime": "6096aa7948b8b666aa3892397a96d044a55f02c6",
"runtime_equal": true,
"normative_equal_to_upstream_main": true,
"changed_paths": [
"AGENTS.md",
"REPORT.md",
"evidence/README.md",
"evidence/final-memory-validation-2026-11-07/RESULTS.md",
"evidence/final-memory-validation-2026-11-07/active-changes.txt",
"evidence/final-memory-validation-2026-11-07/audit-files.py",
"evidence/final-memory-validation-2026-11-07/audit.json",
"evidence/final-memory-validation-2026-11-07/build.txt",
"evidence/final-memory-validation-2026-11-07/checks.txt",
"evidence/final-memory-validation-2026-11-07/conditions-after.json",
"evidence/final-memory-validation-2026-11-07/conditions.json",
"evidence/final-memory-validation-2026-11-07/container-events.json",
"evidence/final-memory-validation-2026-11-07/down.txt",
"evidence/final-memory-validation-2026-11-07/env.sh",
"evidence/final-memory-validation-2026-11-07/final-audit.json",
"evidence/final-memory-validation-2026-11-07/final-checks.txt",
"evidence/final-memory-validation-2026-11-07/final-containers.txt",
"evidence/final-memory-validation-2026-11-07/findings.json",
"evidence/final-memory-validation-2026-11-07/linux-child.py",
"evidence/final-memory-validation-2026-11-07/linux-runner.py",
"evidence/final-memory-validation-2026-11-07/linux.json",
"evidence/final-memory-validation-2026-11-07/linux.txt",
"evidence/final-memory-validation-2026-11-07/openspec.txt",
"evidence/final-memory-validation-2026-11-07/png-boundary.json",
"evidence/final-memory-validation-2026-11-07/png-boundary.txt",
"evidence/final-memory-validation-2026-11-07/png-http.json",
"evidence/final-memory-validation-2026-11-07/png-http.txt",
"evidence/final-memory-validation-2026-11-07/reset-worker-peak.txt",
"evidence/final-memory-validation-2026-11-07/retained-volumes.txt",
"evidence/final-memory-validation-2026-11-07/running-containers-after.txt",
"evidence/final-memory-validation-2026-11-07/runtime.txt",
"evidence/final-memory-validation-2026-11-07/scan-secrets.py",
"evidence/final-memory-validation-2026-11-07/secret-scan.json",
"evidence/final-memory-validation-2026-11-07/sha256.json",
"evidence/final-memory-validation-2026-11-07/smoke.txt",
"evidence/final-memory-validation-2026-11-07/stopped-containers.txt",
"evidence/final-memory-validation-2026-11-07/student-test.txt",
"evidence/final-memory-validation-2026-11-07/targeted.py",
"evidence/final-memory-validation-2026-11-07/up.txt",
"evidence/load-validation.md"
],
"local_links_verified": 208,
"tracked_json_parsed": 111,
"previous_and_fix_hashes_verified": 81,
"raw_diff_check_exit": 2,
"preserved_raw_trailing_whitespace_lines": 17,
"docs_and_helpers_diff_check": "PASS",
"accepted_without_rerun": [
{
"file": "evidence/forward-final-2026-11-06/level4.json",
"source_sha": "0e921116610d289716af0a40ec823376eb8bbcbc",
"scenarios": 17,
"invariants": 8
},
{
"file": "evidence/forward-final-2026-11-06/track-run1.json",
"source_sha": "0e921116610d289716af0a40ec823376eb8bbcbc",
"scenarios": 13,
"invariants": 8
},
{
"file": "evidence/forward-final-2026-11-06/track-run2.json",
"source_sha": "0e921116610d289716af0a40ec823376eb8bbcbc",
"scenarios": 13,
"invariants": 8
},
{
"file": "evidence/forward-final-2026-11-06/track-run3.json",
"source_sha": "0e921116610d289716af0a40ec823376eb8bbcbc",
"scenarios": 13,
"invariants": 8
},
{
"file": "evidence/final-validation-2026-11-07/postgres-outage-retry.json",
"source_sha": "b30e2f6a84419d1ca64525ffb40df92183cccaef",
"scenarios": 1,
"invariants": 8
},
{
"file": "evidence/final-validation-2026-11-07/delivery-targeted.json",
"source_sha": "b30e2f6a84419d1ca64525ffb40df92183cccaef",
"scenarios": 12,
"invariants": 8
}
],
"own_containers": [],
"retained_volumes": [
"mtusi2026lab5_broker-data",
"mtusi2026lab5_pgdata",
"mtusi2026lab5_s3data",
"mtusi2026lab5_valkeydata"
],
"result": "PASS"
}
@@ -1,14 +0,0 @@
date: 2026-11-06T23:23:20Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ make check validate lint
python3 scripts/check_contract.py
PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
.venv/bin/python scripts/validate_openapi.py
PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
.venv/bin/ruff check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py scripts/make_images.py scripts/pipeline_suite.py scripts/pipeline_evidence.py
All checks passed!
.venv/bin/ruff format --check src student-tests scripts/capture_evidence.py alembic scripts/scaling.py tools scripts/probe.py scripts/limits.py scripts/lab4_scenarios.py scripts/lab4_seed.py scripts/upload_memory.py scripts/reconcile_check.py scripts/reconcile_track.py scripts/make_images.py scripts/pipeline_suite.py scripts/pipeline_evidence.py
198 files already formatted
.venv/bin/ty check src student-tests alembic
All checks passed!
exit_code: 0 elapsed_seconds: 0.530
@@ -1,15 +0,0 @@
date: 2026-11-06T23:18:53Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ docker compose ps -a --format '{{.Name}} {{.ID}} {{.State}} {{.Status}}'
mtusi2026lab5-api-1 d1dcbda23866 running Up 6 minutes (healthy)
mtusi2026lab5-api-2 1eaf0fe05de6 running Up 6 minutes (healthy)
mtusi2026lab5-broker-1 be1942ad69b4 running Up 6 minutes (healthy)
mtusi2026lab5-lb-1 4ddcc75182e6 running Up 6 minutes (healthy)
mtusi2026lab5-migrate-1 34901a09ef7b exited Exited (0) 6 minutes ago
mtusi2026lab5-postgres-1 a0b9f2742b97 running Up 6 minutes (healthy)
mtusi2026lab5-relay-1 63225b5eff8e running Up 6 minutes (healthy)
mtusi2026lab5-s3-1 29b3e7c3a260 running Up 6 minutes (healthy)
mtusi2026lab5-s3-init-1 6e80e83ed5c5 exited Exited (0) 6 minutes ago
mtusi2026lab5-valkey-1 647882797ca2 running Up 6 minutes (healthy)
mtusi2026lab5-worker-1 cb275ee9574b running Up About a minute (healthy)
exit_code: 0 elapsed_seconds: 0.127
@@ -1,37 +0,0 @@
[
{
"severity": "minor",
"file": "src/messenger/attachments/application/pipeline.py:230",
"problem": "Сохранено из ревью 544b9c78d39d0e9dd07f841894c390895896605a: after_publish_before_confirm вызывается после publish(), уже проверившего Basic.Ack. crash_publish_confirm доказывает сбой после confirm до COMMIT outbox, а не потерю подтверждения."
},
{
"severity": "minor",
"file": "scripts/limits.py:10",
"problem": "Сохранено из первого ревью: S3_CLIENT_SECRETS не отсортирован, сравнение с sorted(secrets) даёт ложные FAIL. evidence/sqlalchemy-2026-11-06/grade3/limits.txt содержит exit_code=1; этот запуск нельзя обозначать PASS."
},
{
"severity": "minor",
"file": "scripts/reconcile_track.py:396",
"problem": "Сохранено из первого ревью: демонстратор ожидает upload 201 и attachments/, поэтому непригоден для квалификации ЛР5 с 202 queued. Текущие pipeline evidence его не используют."
},
{
"severity": "minor",
"file": "src/messenger/attachments/application/pipeline.py:304",
"problem": "Сохранено из первого ревью: failed upgrade v2 при опубликованной ready v1 нельзя отдельно повторить через replay/reprocess. V1 остаётся доступной; подтверждение исходного ревью — test_failed_new_version_keeps_ready_original."
},
{
"severity": "minor",
"file": "evidence/forward-final-2026-11-06/clock-observation.json",
"problem": "Сохранено из первого ревью: в повторах 2/3 UTC-интервалы расходятся с elapsed (1161.195/186.458 с и 571.513/180.259 с). Причина не установлена; длительности исключены из выводов о производительности."
},
{
"severity": "minor",
"file": "openspec/config.yaml:130",
"problem": "Сохранена документационная неточность: design требует upload 201 вместо нормативного 202 ЛР5; конфиг вне scope validation."
},
{
"severity": "minor",
"file": "src/messenger/attachments/entrypoints/schemas.py:67",
"problem": "Независимое review M: attempt=\"1\", pipeline_version=true, числовой created_at и traceparent=null принимаются вне jobs.schema.json с bus/ack вместо reject. Штатный relay формирует корректные сообщения; критичного последствия не установлено. Источник: evidence/report-memory-audit-2026-11-07.md; повторные пробы в validation не выполнялись."
}
]
@@ -1,58 +0,0 @@
import hashlib
import json
import resource
import signal
import sys
import time
from io import BytesIO
from pathlib import Path
import messenger.worker
from PIL import Image
from messenger.attachments.infrastructure.images import RenderLimits, render_variants
resource.setrlimit(resource.RLIMIT_AS, (448 << 20, 448 << 20))
data = Path(sys.argv[1]).read_bytes()
transparent = sys.argv[3] == "true"
limits = RenderLimits(10485760, 40000000)
versions = []
for version in (1, 2):
signal.alarm(30)
began = time.monotonic()
variants = render_variants(data, limits, version)
ended = time.monotonic()
signal.alarm(0)
assert [(v.name, v.width, v.height) for v in variants] == [
("thumbnail", 256, 160),
("crop", 128, 128),
("optimized", 1280, 800),
]
for variant in variants:
with Image.open(BytesIO(variant.data)) as image:
assert (image.getpixel((5, image.height // 2)) == (255, 255, 255)) == transparent
assert image.getpixel((image.width - 6, image.height // 2)) != (255, 255, 255)
versions.append(
{
"version": version,
"started_monotonic": began,
"finished_monotonic": ended,
"render_seconds": ended - began,
"variants": {v.name: hashlib.sha256(v.data).hexdigest() for v in variants},
}
)
signal.alarm(30)
assert len(render_variants(Path(sys.argv[2]).read_bytes(), limits)) == 3
signal.alarm(0)
print(
json.dumps(
{
"file": Path(sys.argv[1]).name,
"input_sha256": hashlib.sha256(data).hexdigest(),
"versions": versions,
"ordinary_after": True,
"max_rss_kib": resource.getrusage(resource.RUSAGE_SELF).ru_maxrss,
"result": "PASS",
}
),
flush=True,
)
@@ -1,89 +0,0 @@
import base64
import hashlib
import json
import subprocess
from pathlib import Path
folder = Path(__file__).parent
fixtures = {
p.name: p.read_bytes()
for p in Path("evidence/final-validation-2026-11-07/fixtures").glob("*.png")
}
data = fixtures["gray16-40mp.png"]
result, offset = data[:8], 8
while offset < len(data):
size = int.from_bytes(data[offset : offset + 4], "big") + 12
if data[offset + 4 : offset + 8] != b"tRNS":
result += data[offset : offset + size]
offset += size
fixtures["gray16-opaque-40mp.png"] = result
fixtures["sample.png"] = Path("tests/fixtures/sample.png").read_bytes()
bootstrap = """
import base64,json,sys,subprocess,time
from pathlib import Path
p=json.load(sys.stdin)
d=Path('/tmp/png-memory-validation')
d.mkdir(exist_ok=True)
for name,data in p['fixtures'].items():
(d/name).write_bytes(base64.b64decode(data))
def command(name):
return [sys.executable,'-c',p['code'],str(d/name),str(d/'sample.png'),'false' if 'opaque' in name else 'true']
results=[]
for name in sorted(p['fixtures']):
if name=='sample.png':
continue
output=subprocess.check_output(command(name),text=True,timeout=65)
row=json.loads(output)
results.append(row)
print(json.dumps({'case':row}),flush=True)
children=[subprocess.Popen(command('rgb16-40mp.png'),stdout=subprocess.PIPE,stderr=subprocess.PIPE,text=True) for _ in range(2)]
parallel=[]
try:
for child in children:
out,err=child.communicate(timeout=65)
assert child.returncode==0,(child.returncode,err)
parallel.append(json.loads(out))
finally:
for child in children:
if child.poll() is None:
child.kill()
child.wait()
overlap=any(max(a['started_monotonic'],b['started_monotonic'])<min(a['finished_monotonic'],b['finished_monotonic']) for a in parallel[0]['versions'] for b in parallel[1]['versions'])
assert overlap
peak=int(Path('/sys/fs/cgroup/memory.peak').read_text())
limit=int(Path('/sys/fs/cgroup/memory.max').read_text())
assert peak<limit
print(json.dumps({'result':'PASS','cases':results,'concurrent':parallel,'render_overlap':overlap,'cgroup_peak':peak,'cgroup_limit':limit,'memory_events':Path('/sys/fs/cgroup/memory.events').read_text()}),flush=True)
"""
payload = {
"code": (folder / "linux-child.py").read_text(),
"fixtures": {name: base64.b64encode(data).decode() for name, data in fixtures.items()},
}
output = folder / "linux.json"
if output.exists():
raise SystemExit("Result exists; do not repeat completed evidence")
process = subprocess.Popen(
["docker", "compose", "exec", "-T", "worker", "python", "-c", bootstrap],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
)
process.stdin.write(json.dumps(payload))
process.stdin.close()
lines = []
for line in process.stdout:
print(line, end="", flush=True)
lines.append(line)
code = process.wait()
report = {
"source_sha": subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip(),
"exit_code": code,
"input_sha256": {name: hashlib.sha256(data).hexdigest() for name, data in fixtures.items()},
}
if code == 0:
report.update(json.loads(lines[-1]))
else:
report["result"] = "FAIL"
output.write_text(json.dumps(report, indent=2) + "\n")
raise SystemExit(code)
@@ -1,238 +0,0 @@
{
"source_sha": "74c33160ac01618f70e9e1c6a121a5e45c3c7155",
"exit_code": 0,
"input_sha256": {
"rgb16-40mp.png": "3c4418a6f6a7676298b30b70ebbde9d12929f1345152f4fae979caf5ba1cb3f7",
"gray4-40mp.png": "64e665928c08ac3721324af4f8e2e9d2e98230dc1be389dde7bbf365650b08cd",
"gray2-40mp.png": "e1238de6a5be9dbce36abee904f7233f8ff7be59e405b528671940630cce11f6",
"gray16-40mp.png": "b59784ae916d50b41b15afa4bdd474a8699f099da55efd53580d3f97180f3800",
"gray16-opaque-40mp.png": "e3f239d818da5f02cd7b5d1cc4a2c542ed4fc4a7fbc604598500e7810e246e2f",
"sample.png": "99ea7533c441f79cd630d15ce4d58d1ec4e06f6f0efb3b77a5c088305cd1bd75"
},
"result": "PASS",
"cases": [
{
"file": "gray16-40mp.png",
"input_sha256": "b59784ae916d50b41b15afa4bdd474a8699f099da55efd53580d3f97180f3800",
"versions": [
{
"version": 1,
"started_monotonic": 192.032995711,
"finished_monotonic": 193.256173795,
"render_seconds": 1.2231780839999828,
"variants": {
"thumbnail": "99b591aa444566a5c425692f989a8cee8621615ec6aa7911e273e2c47f07d027",
"crop": "8ea8ec6f95dc1d1697327cd4640e59c2adce777564d349703df9f581dde2f450",
"optimized": "7667253d3e6bb37c83e75c2955705e4d5b97a7ee515df4aac5099c9ab3f97335"
}
},
{
"version": 2,
"started_monotonic": 193.258607545,
"finished_monotonic": 194.489094088,
"render_seconds": 1.230486543000012,
"variants": {
"thumbnail": "a17f036f331d17eb03302ddf06741e2f22a8da3394e85e122194fe45282de352",
"crop": "9de07bd2f684859366c50919650f159adc93f9027150de2f51cafac502794daf",
"optimized": "e9d888a5186c3f4824ef866fe97606e2d0a486a312551bfd571e02d2f509a4b7"
}
}
],
"ordinary_after": true,
"max_rss_kib": 365508,
"result": "PASS"
},
{
"file": "gray16-opaque-40mp.png",
"input_sha256": "e3f239d818da5f02cd7b5d1cc4a2c542ed4fc4a7fbc604598500e7810e246e2f",
"versions": [
{
"version": 1,
"started_monotonic": 195.17374863,
"finished_monotonic": 195.46179888,
"render_seconds": 0.28805024999999773,
"variants": {
"thumbnail": "59472ec8f914eb4502ffd2591a59ac68b05cccefeedc678f5ad2919cd2c6a3a2",
"crop": "91eed7d7e1f422bfc3827903cc6801a37387d642a1b4fcc08e23b2de4840a3d7",
"optimized": "c5216644c9e66c732de7523bef11b7024bf0417adb1d134d8f2ce259fd8b6ed1"
}
},
{
"version": 2,
"started_monotonic": 195.464219338,
"finished_monotonic": 195.744785005,
"render_seconds": 0.28056566700001895,
"variants": {
"thumbnail": "13ea6ae5bd526b8b0ad4e88ff72498ff48c6d50f8d12e81ad6773a1eaa9fc177",
"crop": "2d5dbe60802e4aa1c3ff20c112df454c70a12e2c9fccc99137802a5556e270bf",
"optimized": "38fc10be1e1f56da14304d26c52c11032900002c70de502ea1b7c1639535633e"
}
}
],
"ordinary_after": true,
"max_rss_kib": 365492,
"result": "PASS"
},
{
"file": "gray2-40mp.png",
"input_sha256": "e1238de6a5be9dbce36abee904f7233f8ff7be59e405b528671940630cce11f6",
"versions": [
{
"version": 1,
"started_monotonic": 196.461284589,
"finished_monotonic": 196.747503755,
"render_seconds": 0.2862191659999951,
"variants": {
"thumbnail": "ce63aabedb566aea34ad71f5a146f8dc8f0fd27c1f28f5ad1a28ffdf4630b053",
"crop": "43956a7b626be7786964c28ae7bf0cb96b2b914335a1b8129772c7bd88ee95ed",
"optimized": "059fb496f80d0ad3c73ea4d257f754b726f5d8dacfefd42ea7cedd6ff3368b24"
}
},
{
"version": 2,
"started_monotonic": 196.750391839,
"finished_monotonic": 197.01864788,
"render_seconds": 0.2682560410000008,
"variants": {
"thumbnail": "efb7970658ce2e7ed86bf5f3d5d695b81cd396e717d699fd9e69ffb294d5d870",
"crop": "cb1ce90bf86c1178ac5f0830e3d34ed54296a0e81231b422ba9f622dd22c7daa",
"optimized": "38dc2af32b8bbc1f15fbff2fcfeb5fa0d5ac6ed75225afc575e80254d2a54d0c"
}
}
],
"ordinary_after": true,
"max_rss_kib": 343076,
"result": "PASS"
},
{
"file": "gray4-40mp.png",
"input_sha256": "64e665928c08ac3721324af4f8e2e9d2e98230dc1be389dde7bbf365650b08cd",
"versions": [
{
"version": 1,
"started_monotonic": 197.695555797,
"finished_monotonic": 197.978069006,
"render_seconds": 0.2825132090000011,
"variants": {
"thumbnail": "9130484055a59c8c813ea7e48b32e27a3c424b4107ca4f872c8711ad34234610",
"crop": "c3bbbc98cc4a63ea0aab370b4e81bdc66a353a8ca7beb9c83a2c28af9f44b641",
"optimized": "e23b8d975b6973b848ba57cb68925d6f88156f44d9dcbe52d194e006e49dcefa"
}
},
{
"version": 2,
"started_monotonic": 197.980502798,
"finished_monotonic": 198.257521881,
"render_seconds": 0.277019082999999,
"variants": {
"thumbnail": "814c3cc7149871ae2614c4e71f9ffd35e86063bad1c8f94c2aeb5c0dacd516c3",
"crop": "beb44557013393b7c1a85b1df2fd74ffaf8bd32606092bad48c300b7d7a676fb",
"optimized": "b7a3cf6655e630458fb5341d7a79e742dafcf26590724a0440e3dfe82e4986f7"
}
}
],
"ordinary_after": true,
"max_rss_kib": 342956,
"result": "PASS"
},
{
"file": "rgb16-40mp.png",
"input_sha256": "3c4418a6f6a7676298b30b70ebbde9d12929f1345152f4fae979caf5ba1cb3f7",
"versions": [
{
"version": 1,
"started_monotonic": 198.959187048,
"finished_monotonic": 199.736227465,
"render_seconds": 0.7770404170000234,
"variants": {
"thumbnail": "be34d8832da928389bfebfc6c1f16a5f871bca150c4cfcb6480c28b9734038ec",
"crop": "8d4a59604fea3ea6ad475b5bb12c0c9fd30a48297bd1b7bd41ac1ee82105df96",
"optimized": "30e30ca074118f4fd00c1670bdfe93a03b083da861af109fe863f6baab4549af"
}
},
{
"version": 2,
"started_monotonic": 199.738488423,
"finished_monotonic": 200.507796924,
"render_seconds": 0.7693085009999834,
"variants": {
"thumbnail": "ec9b5fc973ee1c36c8bbb4f11fcc376a255a8a2b52067d958b57aeb272eb7f3d",
"crop": "6d73d731eff497a00f8535f705e4a2e79a376d08d559d9afeb8b815aa56a3bb2",
"optimized": "a1de31c50abed6c9309af3cd7bc4fcabdfefe8d38d2b6eb946f91e52cb69e687"
}
}
],
"ordinary_after": true,
"max_rss_kib": 406312,
"result": "PASS"
}
],
"concurrent": [
{
"file": "rgb16-40mp.png",
"input_sha256": "3c4418a6f6a7676298b30b70ebbde9d12929f1345152f4fae979caf5ba1cb3f7",
"versions": [
{
"version": 1,
"started_monotonic": 201.216312716,
"finished_monotonic": 202.037238716,
"render_seconds": 0.8209259999999858,
"variants": {
"thumbnail": "be34d8832da928389bfebfc6c1f16a5f871bca150c4cfcb6480c28b9734038ec",
"crop": "8d4a59604fea3ea6ad475b5bb12c0c9fd30a48297bd1b7bd41ac1ee82105df96",
"optimized": "30e30ca074118f4fd00c1670bdfe93a03b083da861af109fe863f6baab4549af"
}
},
{
"version": 2,
"started_monotonic": 202.0398138,
"finished_monotonic": 202.834078508,
"render_seconds": 0.7942647080000143,
"variants": {
"thumbnail": "ec9b5fc973ee1c36c8bbb4f11fcc376a255a8a2b52067d958b57aeb272eb7f3d",
"crop": "6d73d731eff497a00f8535f705e4a2e79a376d08d559d9afeb8b815aa56a3bb2",
"optimized": "a1de31c50abed6c9309af3cd7bc4fcabdfefe8d38d2b6eb946f91e52cb69e687"
}
}
],
"ordinary_after": true,
"max_rss_kib": 404584,
"result": "PASS"
},
{
"file": "rgb16-40mp.png",
"input_sha256": "3c4418a6f6a7676298b30b70ebbde9d12929f1345152f4fae979caf5ba1cb3f7",
"versions": [
{
"version": 1,
"started_monotonic": 201.216906382,
"finished_monotonic": 202.031220341,
"render_seconds": 0.8143139590000033,
"variants": {
"thumbnail": "be34d8832da928389bfebfc6c1f16a5f871bca150c4cfcb6480c28b9734038ec",
"crop": "8d4a59604fea3ea6ad475b5bb12c0c9fd30a48297bd1b7bd41ac1ee82105df96",
"optimized": "30e30ca074118f4fd00c1670bdfe93a03b083da861af109fe863f6baab4549af"
}
},
{
"version": 2,
"started_monotonic": 202.033759216,
"finished_monotonic": 202.824064842,
"render_seconds": 0.7903056260000199,
"variants": {
"thumbnail": "ec9b5fc973ee1c36c8bbb4f11fcc376a255a8a2b52067d958b57aeb272eb7f3d",
"crop": "6d73d731eff497a00f8535f705e4a2e79a376d08d559d9afeb8b815aa56a3bb2",
"optimized": "a1de31c50abed6c9309af3cd7bc4fcabdfefe8d38d2b6eb946f91e52cb69e687"
}
}
],
"ordinary_after": true,
"max_rss_kib": 404776,
"result": "PASS"
}
],
"render_overlap": true,
"cgroup_peak": 1026285568,
"cgroup_limit": 1073741824,
"memory_events": "low 0\nhigh 0\nmax 0\noom 0\noom_kill 0\noom_group_kill 0\n"
}
File diff suppressed because one or more lines are too long
@@ -1,28 +0,0 @@
date: 2026-11-06T22:54:43Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {}
$ openspec validate --all --strict
- Validating...
✓ spec/lab1-event-publication
✓ spec/lab1-messenger-chats
✓ spec/lab1-messenger-history
✓ spec/lab1-messenger-identity
✓ spec/lab1-service-operability
✓ spec/lab2-auth-sessions
✓ spec/lab2-dependency-failures
✓ spec/lab2-history-index
✓ spec/lab2-persistent-storage
✓ spec/lab2-session-cache
✓ spec/lab3-failover-operations
✓ spec/lab3-idempotent-messages
✓ spec/lab3-load-balancing
✓ spec/lab4-attachments
✓ spec/lab4-object-storage
✓ spec/lab4-storage-reconciler
✓ spec/lab4-upload-resilience
✓ spec/lab5-attachment-pipeline
✓ spec/lab5-image-variants
✓ spec/lab5-job-delivery
✓ spec/lab5-job-recovery
✓ spec/lab5-versioned-pipeline
Totals: 22 passed, 0 failed (22 items)
exit_code: 0 elapsed_seconds: 0.921
File diff suppressed because it is too large Load Diff
@@ -1,12 +0,0 @@
date: 2026-11-06T23:15:49Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ .venv/bin/python evidence/final-memory-validation-2026-11-07/targeted.py png-boundary
gray2 PASS
gray4 PASS
gray16 PASS
rgb16 PASS
gray16-without-trns PASS
concurrent-rgb16 PASS
forced-timeout PASS
forced-memory PASS
exit_code: 0 elapsed_seconds: 95.742
@@ -1,742 +0,0 @@
{
"mode": "png-http",
"source_sha": "74c33160ac01618f70e9e1c6a121a5e45c3c7155",
"started_at": "2026-11-06T23:17:43.287659Z",
"cases": [
{
"name": "gray2",
"started_at": "2026-11-06T23:17:44.380294Z",
"observed": {
"attachment_id": "01a1b327-5f46-7727-8fb2-3d557bcfe643",
"input_sha256": "80754a3aa0c8a6014dc7a282b2009d49d064a5c02ed48e634d05d01c0aa3df6b",
"input_bytes": 189,
"versions": [
{
"version": 1,
"variants": [
{
"name": "crop",
"sha256": "def030bc04042aa23362c895a36b18d1d9748360df045decaf68eaeb0aa68deb",
"bytes": 1324,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "fd49dc2d33e7b7963cd8f8f6ef0f0b80117e2c4a7dfb4fd2ee79285e47ff789a",
"bytes": 98,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "1b6f714677aa69673da7ed9eefba73376b8df42eae70a3d91002d2aab7272d9e",
"bytes": 332,
"dimensions": [
64,
64
]
}
]
},
{
"version": 2,
"variants": [
{
"name": "crop",
"sha256": "d14b6cc6fe415bd8c4fef55402a4fb196bc4f39d493ee505d536594b28841697",
"bytes": 1489,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "c8f23e1adbf1add646f23531e270dc25d5108453d50947dec3e73700b0d6741f",
"bytes": 96,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "93107a3ba1a8aba475f8993dfb0d03658cef12a416c08a230653370921dcc00b",
"bytes": 335,
"dimensions": [
64,
64
]
}
]
}
],
"stale_and_duplicate_noop": true
},
"result": "PASS",
"elapsed_seconds": 4.395
},
{
"name": "gray4",
"started_at": "2026-11-06T23:17:48.776246Z",
"observed": {
"attachment_id": "01a1b327-706a-74da-b85b-267a032e36ab",
"input_sha256": "d0ed5b64b3b875be881dd286bedb7948fd06570f9ef1d2cfe41b665a838e426b",
"input_bytes": 263,
"versions": [
{
"version": 1,
"variants": [
{
"name": "crop",
"sha256": "43323e552e3c79e1a1688478d72c2ce370c21e55f1560958f9f6a225bc32f909",
"bytes": 1034,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "dbc0ec3736c18df42f26b8f396b2eb243a18c6a0f5b6bbef8bf4ff7e563032c3",
"bytes": 90,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "d594ebb80ca7939643379ba54392070444eb900ca40bec9c6beee75175fd81ab",
"bytes": 330,
"dimensions": [
64,
64
]
}
]
},
{
"version": 2,
"variants": [
{
"name": "crop",
"sha256": "16e27d4c1df7aac7185381a1f4c8c9cceb05647e98f941c41e38f4dfe4c5ec75",
"bytes": 1197,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "cd31de25056767d9050108c4475d4b72ccd1ab3c36c1c96159b104019e4e3493",
"bytes": 92,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "86afdaf8b40f77c2426ebf10c8d603535d75caa4fc720462be9acc43e39d4fbd",
"bytes": 330,
"dimensions": [
64,
64
]
}
]
}
],
"stale_and_duplicate_noop": true
},
"result": "PASS",
"elapsed_seconds": 4.114
},
{
"name": "rgb16",
"started_at": "2026-11-06T23:17:52.890838Z",
"observed": {
"attachment_id": "01a1b327-807d-7398-8d62-e5d3486f1e9c",
"input_sha256": "e8e5f64cf3a0312b17549aed29f062e3cb03d0c1bf327396c6a25c4b88dec89f",
"input_bytes": 724,
"versions": [
{
"version": 1,
"variants": [
{
"name": "crop",
"sha256": "6386114c3d0b89c4df88745eec318b5db1a779d488e1b08f34ac34138c7feb99",
"bytes": 671,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "fed7e1bf230e91296a5ea9a85b51d190152639bb2963b661f872029d852cc4ad",
"bytes": 94,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "af0a65a6596a0905a79403a9a6ef32a1301671a98ae3c8b5c3d6ced2d20a1009",
"bytes": 332,
"dimensions": [
64,
64
]
}
]
},
{
"version": 2,
"variants": [
{
"name": "crop",
"sha256": "dedcbac123226e3012a4dbd178b7aae7a7aba4f2c44c91be274c86804238b208",
"bytes": 696,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "bc09a1696563daae4385190d2c52afac8230e18fff4b5eba701a42f0facde043",
"bytes": 94,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "218bc7fd6fa995a3c4cf725016714408073699d13f39d0d364ffe3595856d272",
"bytes": 333,
"dimensions": [
64,
64
]
}
]
}
],
"stale_and_duplicate_noop": true
},
"result": "PASS",
"elapsed_seconds": 4.989
},
{
"name": "gray1",
"started_at": "2026-11-06T23:17:57.880356Z",
"observed": {
"attachment_id": "01a1b327-93fd-7099-8ab3-600c201aa01d",
"input_sha256": "1bf59ee41fbd12bf572ac294e0505cc00b904363dd479d643b43062199dd67e3",
"input_bytes": 154,
"versions": [
{
"version": 1,
"variants": [
{
"name": "crop",
"sha256": "1b88218fd7e3ff00d6fa3e4bb732411617126591b2b6837aecdcaef426327eda",
"bytes": 380,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "315ce3f53ac2012226bbeb906c7d37997a7f6fabb3d0ede23a797e66fe4b51ee",
"bytes": 66,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "c5745603d7848377a38bb5228f7817f5d4d4f5af26dadd7d3b2fb97b6a598a8e",
"bytes": 308,
"dimensions": [
64,
64
]
}
]
},
{
"version": 2,
"variants": [
{
"name": "crop",
"sha256": "45eac1bd54dba273142a01a3f01a1530787db5a9fe04aa2483862256dbf16132",
"bytes": 380,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "8c5e7cc56874d00f7d5731a6a89461b264ef4e3b450899b63f4608231180b5aa",
"bytes": 66,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "19305cfd18dc8f3e504d6ef5667ddd7be788836e16317dba5b2c6f590b6a2667",
"bytes": 308,
"dimensions": [
64,
64
]
}
]
}
],
"stale_and_duplicate_noop": true
},
"result": "PASS",
"elapsed_seconds": 3.969
},
{
"name": "gray8",
"started_at": "2026-11-06T23:18:01.850592Z",
"observed": {
"attachment_id": "01a1b327-a380-7186-b76d-46a83b6ad53f",
"input_sha256": "31cb54aacd9e1c324b14e38377f83a21d4084dc134aa3d3e1e5c40e3d9c5dd2f",
"input_bytes": 286,
"versions": [
{
"version": 1,
"variants": [
{
"name": "crop",
"sha256": "14a6e4afbe7e4cf766330007adda1c14168f16957f9409f402a8699a0e5de62f",
"bytes": 621,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "d35d7d59302099ea5fea9e91a405f4ebf8823aae1f53a59f88a09015619136fa",
"bytes": 74,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "d8fac3c82defab08dbb276681b1a7fb2545d883830dae5f965349474897373c7",
"bytes": 324,
"dimensions": [
64,
64
]
}
]
},
{
"version": 2,
"variants": [
{
"name": "crop",
"sha256": "dba34622f93edcd2d3449e217eb9605b1fac739c7c873a875af5d88d8e76bf2e",
"bytes": 637,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "3d4823a2da9971a444b16cf060be5f0b2b7a16bf50a26e7343283d48b562d105",
"bytes": 76,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "d9d4ed6ef383acdf227745e511acaffddf6e90c4f100d0f6ea117f5783581d29",
"bytes": 324,
"dimensions": [
64,
64
]
}
]
}
],
"stale_and_duplicate_noop": true
},
"result": "PASS",
"elapsed_seconds": 4.033
},
{
"name": "gray16",
"started_at": "2026-11-06T23:18:05.884188Z",
"observed": {
"attachment_id": "01a1b327-b349-7332-b73c-db1221023da8",
"input_sha256": "f7fbc2506997045f18891037f3cde42cf1e3b55461948f790779817317574f77",
"input_bytes": 344,
"versions": [
{
"version": 1,
"variants": [
{
"name": "crop",
"sha256": "e49a69181bafd0ebb4348db7030ad2cbf632b296a65f0ba05fe984b3d2b1ccdc",
"bytes": 632,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "751dc647784804af94b6cda23248840f73cab0cfe03227c4568b75195c00603a",
"bytes": 80,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "1bc87b25d02a179dc0cf5a883fb624fda094ba042ed9476c883ae1c12da6f1b6",
"bytes": 322,
"dimensions": [
64,
64
]
}
]
},
{
"version": 2,
"variants": [
{
"name": "crop",
"sha256": "a94faef9b08b5ab6ab239b6618c0e92b0ab92abca889acd714d6d9179dde90ef",
"bytes": 662,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "b22468814595a3ae63a9b326ed55fefb9a833a0e621be5f366b113ad6cf18a12",
"bytes": 72,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "d2267d9adb990db045e8c186737a4bdf66aa4e8d82d785c0bb3d8e87c34c5072",
"bytes": 323,
"dimensions": [
64,
64
]
}
]
}
],
"stale_and_duplicate_noop": true
},
"result": "PASS",
"elapsed_seconds": 4.099
},
{
"name": "rgb8",
"started_at": "2026-11-06T23:18:09.984254Z",
"observed": {
"attachment_id": "01a1b327-c34a-7194-8fbd-164055cfd3ef",
"input_sha256": "8c27c74500758b1a24ccc96261334265e304d55530780aefc7ef4a1109c11852",
"input_bytes": 385,
"versions": [
{
"version": 1,
"variants": [
{
"name": "crop",
"sha256": "c36bf54d185cfcf7abddd219a5e586e11b0c4d10007df5db55978f74ddcad2d0",
"bytes": 1069,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "7b78675ebd18218e82922e75072a801eea74b4d5f26cde2b5810407c86509b88",
"bytes": 116,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "f6e37a29401efe000eb8deb1490975d745f621f2ab229f4d85162945189e1791",
"bytes": 385,
"dimensions": [
64,
64
]
}
]
},
{
"version": 2,
"variants": [
{
"name": "crop",
"sha256": "fd82664e5924528cd7916b3b9e30b71517fa13c2763a8705e750fd4293f3027c",
"bytes": 1184,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "a05ef5a9f1305b2941cc99ab594eb3671bb87babe35d28ac97127791881137bb",
"bytes": 120,
"dimensions": [
64,
64
]
},
{
"name": "thumbnail",
"sha256": "3cac9f1ca274bcac1dc05cf186cd88eb622fd947fb1a7958a1175e77623a96d6",
"bytes": 392,
"dimensions": [
64,
64
]
}
]
}
],
"stale_and_duplicate_noop": true
},
"result": "PASS",
"elapsed_seconds": 3.994
},
{
"name": "rgb16-adam7-filter4-exif6",
"started_at": "2026-11-06T23:18:13.978929Z",
"observed": {
"attachment_id": "01a1b327-d2e9-72fc-9b7e-a5d8d6e2f8b7",
"input_sha256": "b9c1cb1e309024e4578d1f17bf6bb0f0645aad4d8973e16098b65c54cfd4930a",
"input_bytes": 1297,
"versions": [
{
"version": 1,
"variants": [
{
"name": "crop",
"sha256": "f257bf8fa4f30f0e4df6672de140799375cebd2bf735a656432cf702663414a1",
"bytes": 601,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "cb92af5a047068048eba212d944513edb2c454f64e124339820d46c24e69ea45",
"bytes": 112,
"dimensions": [
63,
65
]
},
{
"name": "thumbnail",
"sha256": "110cf7571f07d2db04821e8ef1861f74b5a16978851a4b0082ca60aedc1d29e4",
"bytes": 419,
"dimensions": [
63,
65
]
}
]
},
{
"version": 2,
"variants": [
{
"name": "crop",
"sha256": "0a7fea5640c5877026100e65e39848e7b15ee5eebb93897ebbeb95d441e8d652",
"bytes": 675,
"dimensions": [
128,
128
]
},
{
"name": "optimized",
"sha256": "9d4b7431a72adfc10284e4443c4e56b4cd19a67c0729979041fc8b0bc2c6ca34",
"bytes": 116,
"dimensions": [
63,
65
]
},
{
"name": "thumbnail",
"sha256": "529ceb55a8675e5ed870ed7917d7f1a5753c898b689f3e431e403b9b3682a732",
"bytes": 440,
"dimensions": [
63,
65
]
}
]
}
],
"stale_and_duplicate_noop": true
},
"result": "PASS",
"elapsed_seconds": 4.082
}
],
"conditions": {
"host_load": [
5.41796875,
3.67138671875,
2.95068359375
],
"containers": [
"mtusi2026lab5-api-1 d1dcbda23866 running Up 5 minutes (healthy)",
"mtusi2026lab5-api-2 1eaf0fe05de6 running Up 5 minutes (healthy)",
"mtusi2026lab5-broker-1 be1942ad69b4 running Up 5 minutes (healthy)",
"mtusi2026lab5-lb-1 4ddcc75182e6 running Up 5 minutes (healthy)",
"mtusi2026lab5-migrate-1 34901a09ef7b exited Exited (0) 5 minutes ago",
"mtusi2026lab5-postgres-1 a0b9f2742b97 running Up 5 minutes (healthy)",
"mtusi2026lab5-relay-1 63225b5eff8e running Up 5 minutes (healthy)",
"mtusi2026lab5-s3-1 29b3e7c3a260 running Up 5 minutes (healthy)",
"mtusi2026lab5-s3-init-1 6e80e83ed5c5 exited Exited (0) 5 minutes ago",
"mtusi2026lab5-valkey-1 647882797ca2 running Up 5 minutes (healthy)",
"mtusi2026lab5-worker-1 cb275ee9574b running Up 41 seconds (healthy)"
],
"running": "mtusi2026lab5-worker-1 Up 42 seconds (healthy)\nmtusi2026lab5-api-1 Up 5 minutes (healthy)\nmtusi2026lab5-api-2 Up 5 minutes (healthy)\nmtusi2026lab5-relay-1 Up 5 minutes (healthy)\nmtusi2026lab5-broker-1 Up 5 minutes (healthy)\nmtusi2026lab5-postgres-1 Up 5 minutes (healthy)\nmtusi2026lab5-lb-1 Up 5 minutes (healthy)\nmtusi2026lab5-s3-1 Up 5 minutes (healthy)\nmtusi2026lab5-valkey-1 Up 5 minutes (healthy)\n",
"worker_limits": "448\n30\n2\n"
},
"accepted_ids": [
"01a1b327-5f46-7727-8fb2-3d557bcfe643",
"01a1b327-706a-74da-b85b-267a032e36ab",
"01a1b327-807d-7398-8d62-e5d3486f1e9c",
"01a1b327-93fd-7099-8ab3-600c201aa01d",
"01a1b327-a380-7186-b76d-46a83b6ad53f",
"01a1b327-b349-7332-b73c-db1221023da8",
"01a1b327-c34a-7194-8fbd-164055cfd3ef",
"01a1b327-d2e9-72fc-9b7e-a5d8d6e2f8b7"
],
"invariants": [
{
"id": "terminal",
"statement": "every 202 reaches its expected terminal state",
"result": "PASS",
"details": {
"accepted": 8
}
},
{
"id": "single_effect",
"statement": "one variant set per (attachment_id, pipeline_version, variant)",
"result": "PASS",
"details": {
"rows": 0
}
},
{
"id": "no_ready_rollback",
"statement": "ready never returns to queued or processing",
"result": "PASS",
"details": {
"traced": 8
}
},
{
"id": "monotonic_publication",
"statement": "published_version only moves forward",
"result": "PASS",
"details": {
"sampled": 8
}
},
{
"id": "bytes_match_metadata",
"statement": "sha256, size and real dimensions match served bytes",
"result": "PASS",
"details": {
"verified": 8
}
},
{
"id": "bounded_attempts",
"statement": "attempt <= JOB_MAX_ATTEMPTS",
"result": "PASS",
"details": {
"rows": 0
}
},
{
"id": "no_orphans",
"statement": "no pending outbox, messages, dead letters, staging, objects",
"result": "PASS",
"details": {
"queues": {
"attachments.process": [
0,
0
],
"attachments.process.dlq": [
16,
0
]
},
"s3_objects": 56,
"scoped_attachments": 8
}
},
{
"id": "no_lost_202",
"statement": "every 202 is present in PostgreSQL and the API",
"result": "PASS",
"details": {
"accepted": 8
}
}
],
"finished_at": "2026-11-06T23:18:30.279606Z",
"result": "PASS"
}
@@ -1,12 +0,0 @@
date: 2026-11-06T23:17:43Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ .venv/bin/python evidence/final-memory-validation-2026-11-07/targeted.py png-http
gray2 PASS
gray4 PASS
rgb16 PASS
gray1 PASS
gray8 PASS
gray16 PASS
rgb8 PASS
rgb16-adam7-filter4-exif6 PASS
exit_code: 0 elapsed_seconds: 47.160
@@ -1,10 +0,0 @@
date: 2026-11-06T23:15:46Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ docker compose up -d --no-deps --force-recreate --wait worker
Container mtusi2026lab5-worker-1 Recreate
Container mtusi2026lab5-worker-1 Recreated
Container mtusi2026lab5-worker-1 Starting
Container mtusi2026lab5-worker-1 Started
Container mtusi2026lab5-worker-1 Waiting
Container mtusi2026lab5-worker-1 Healthy
exit_code: 0 elapsed_seconds: 3.133
@@ -1,8 +0,0 @@
date: 2026-11-06T23:19:15Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ docker volume ls --filter label=com.docker.compose.project=mtusi2026lab5 --format '{{.Name}}'
mtusi2026lab5_broker-data
mtusi2026lab5_pgdata
mtusi2026lab5_s3data
mtusi2026lab5_valkeydata
exit_code: 0 elapsed_seconds: 0.024
@@ -1,4 +0,0 @@
date: 2026-11-06T23:19:15Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ docker ps --format '{{.Names}} {{.Status}}'
exit_code: 0 elapsed_seconds: 0.024
@@ -1,7 +0,0 @@
date: 2026-11-06T23:15:05Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ docker compose exec -T worker python -c 'import hashlib,platform,importlib.metadata; from pathlib import Path; import messenger.attachments.infrastructure.images as m; print(platform.platform()); print({name:importlib.metadata.version(name) for name in ("sqlalchemy","alembic","Pillow","faststream","aio-pika")}); print("images.py sha256",hashlib.sha256(Path(m.__file__).read_bytes()).hexdigest())'
Linux-6.10.14-linuxkit-aarch64-with-glibc2.41
{'sqlalchemy': '2.0.45', 'alembic': '1.16.5', 'Pillow': '13.0.0', 'faststream': '0.7.8', 'aio-pika': '10.2.1'}
images.py sha256 97c6e9c1ef13c8854be9cd2cc11e4279d1d340ba3ab360dfa3e05772923ea743
exit_code: 0 elapsed_seconds: 0.380
@@ -1,63 +0,0 @@
import json
import re
import subprocess
from pathlib import Path
ROOT = Path.cwd()
files = set(subprocess.check_output(["git", "ls-files", "-z"]).decode().split("\0")) - {""}
files.update(
str(p) for p in Path("evidence/final-memory-validation-2026-11-07").rglob("*") if p.is_file()
)
patterns = {
"private_key": r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----",
"github_token": r"\b(?:gh[pousr]_[A-Za-z0-9]{30,}|github_pat_[A-Za-z0-9_]{50,})\b",
"openai_token": r"\bsk-(?:proj-)?[A-Za-z0-9_-]{35,}\b",
"aws_key": r"\b(?:AKIA|ASIA)[A-Z0-9]{16}\b",
"telegram_token": r"\b[0-9]{8,12}:[A-Za-z0-9_-]{35}\b",
"bearer_literal": r"Bearer [A-Za-z0-9_-]{24,}",
"credential_url": r"\b(?:amqps?|postgres(?:ql)?|redis|https?)://[^\s\"\'{}$<>:/]+:[^\s\"\'{}$<>@/]+@",
"secret_literal": r"""["'](?:password|secret|token|access_token|secret_access_key)["']\s*:\s*["'][A-Za-z0-9_+/=-]{24,}["']""",
}
allow = {
("contracts/examples.json", 5, "secret_literal"): "unchanged skeleton example placeholder",
("contracts/openapi.json", 3096, "secret_literal"): "unchanged skeleton example placeholder",
("examples/requests.http", 20, "secret_literal"): "unchanged skeleton example placeholder",
("scripts/broker-secrets.sh", 17, "credential_url"): "printf format; no credential literal",
(
"student-tests/api/test_operability.py",
131,
"credential_url",
): "synthetic unavailable endpoint u:p",
(
"student-tests/api/test_operability.py",
143,
"credential_url",
): "synthetic unavailable endpoint u:p",
("student-tests/conftest.py", 141, "credential_url"): "ephemeral test database literal",
}
findings = []
for name in sorted(files):
if re.search(r"(^|/)(?:\.env|secrets|volumes)(?:/|$)", name):
findings.append({"file": name, "rule": "forbidden_path"})
try:
text = Path(name).read_text()
except UnicodeDecodeError, OSError:
continue
for rule, pattern in patterns.items():
for match in re.finditer(pattern, text):
line = text.count("\n", 0, match.start()) + 1
item = {"file": name, "line": line, "rule": rule}
if (name, line, rule) in allow:
item["reviewed_non_secret"] = allow[name, line, rule]
findings.append(item)
report = {
"files_scanned": len(files),
"patterns": list(patterns),
"candidates": findings,
"unresolved": sum("reviewed_non_secret" not in f for f in findings),
}
Path("evidence/final-memory-validation-2026-11-07/secret-scan.json").write_text(
json.dumps(report, indent=2) + "\n"
)
print(json.dumps(report, indent=2))
raise SystemExit(bool(report["unresolved"]))
@@ -1,58 +0,0 @@
{
"files_scanned": 804,
"patterns": [
"private_key",
"github_token",
"openai_token",
"aws_key",
"telegram_token",
"bearer_literal",
"credential_url",
"secret_literal"
],
"candidates": [
{
"file": "contracts/examples.json",
"line": 5,
"rule": "secret_literal",
"reviewed_non_secret": "unchanged skeleton example placeholder"
},
{
"file": "contracts/openapi.json",
"line": 3096,
"rule": "secret_literal",
"reviewed_non_secret": "unchanged skeleton example placeholder"
},
{
"file": "examples/requests.http",
"line": 20,
"rule": "secret_literal",
"reviewed_non_secret": "unchanged skeleton example placeholder"
},
{
"file": "scripts/broker-secrets.sh",
"line": 17,
"rule": "credential_url",
"reviewed_non_secret": "printf format; no credential literal"
},
{
"file": "student-tests/api/test_operability.py",
"line": 131,
"rule": "credential_url",
"reviewed_non_secret": "synthetic unavailable endpoint u:p"
},
{
"file": "student-tests/api/test_operability.py",
"line": 143,
"rule": "credential_url",
"reviewed_non_secret": "synthetic unavailable endpoint u:p"
},
{
"file": "student-tests/conftest.py",
"line": 141,
"rule": "credential_url",
"reviewed_non_secret": "ephemeral test database literal"
}
],
"unresolved": 0
}
@@ -1,37 +0,0 @@
{
"evidence/final-memory-validation-2026-11-07/RESULTS.md": "e4dd4dfc47ae8095458028235b1e89373b815edeeceddbd09f116f9ad9f4450b",
"evidence/final-memory-validation-2026-11-07/active-changes.txt": "b817c4f054d652d43cc5b0059aa3808fe18d4a4582a3676c34f0d9418efa992d",
"evidence/final-memory-validation-2026-11-07/audit-files.py": "864e3a7032755da30afeef71431649e87c955cea4dc48021c14575b5ac243ded",
"evidence/final-memory-validation-2026-11-07/audit.json": "68e3f1a5806fe4b65f1e8e8efe0b66c5e55cccd4ce492b7a325d4cc89323ca69",
"evidence/final-memory-validation-2026-11-07/build.txt": "62ca4051248a5ba0c1d5380456dc972572da8c08e27b840992b24f500ab57bb7",
"evidence/final-memory-validation-2026-11-07/checks.txt": "6a1195478b3eb211d21352cb6ed00e690196ccc54e0db3a3fe3ac4e5f6b19336",
"evidence/final-memory-validation-2026-11-07/conditions-after.json": "e72877504561b2e4ab95ffa5ad2e616ff3599ba0887fd10b378f9425cf62376c",
"evidence/final-memory-validation-2026-11-07/conditions.json": "c970c484ad92ada00d33f4df86113928b0ef0321b1b7740441a8c1e10f4b5c30",
"evidence/final-memory-validation-2026-11-07/container-events.json": "bfce3b6295e23ea06f4f29beb1b83b4a5b2ec5e07c4230d3ba68d1dc0c241d39",
"evidence/final-memory-validation-2026-11-07/down.txt": "0b80c02939c9224df3e8f5f98ae6320b6ffc71a8a28c001e7b3ee9ae113a6f39",
"evidence/final-memory-validation-2026-11-07/env.sh": "fa41c289402a362827d3edc88bd0e8301e55d279df09f658f800a94bff96b05d",
"evidence/final-memory-validation-2026-11-07/final-audit.json": "2c6308899417e6da60875de9c262828fe7caccd30bffb223d8ce53635f5845ad",
"evidence/final-memory-validation-2026-11-07/final-checks.txt": "7405e049b92aa01667afe431a64e3c6d5ffc1827c03a4bed480a5158802a08d8",
"evidence/final-memory-validation-2026-11-07/final-containers.txt": "dd5432219d35baeee61392605910c2864b71b83e8972377254eff268670422c5",
"evidence/final-memory-validation-2026-11-07/findings.json": "a2b9cc2a425aa0326fd72c6bdab9a5dc69ea4d90fef2c3ce239fbbb514e978a1",
"evidence/final-memory-validation-2026-11-07/linux-child.py": "c293f49bd273fe7c3c6bb0ac90f2ccb28a02117a4fdae758d04e8c006d4e89ab",
"evidence/final-memory-validation-2026-11-07/linux-runner.py": "680052df464d87d97a9717469d53d27f4f2391f740b0aaa23fa52a77a52c9a83",
"evidence/final-memory-validation-2026-11-07/linux.json": "febd60518391e7d6f479908e48856a6c0a774fd5763f33cf02655e414d6b1fe3",
"evidence/final-memory-validation-2026-11-07/linux.txt": "51f31e560852514bdba1777db1ceade9284ff13610d60afaf7a128f171cd7b29",
"evidence/final-memory-validation-2026-11-07/openspec.txt": "dc82d21ab66012b5960a0f76e82eef10d7df082497119d18efb7aed5009efd3d",
"evidence/final-memory-validation-2026-11-07/png-boundary.json": "a6452fb7b502597058641878464ee383eab334b0d77377bfe54565aeb4279df1",
"evidence/final-memory-validation-2026-11-07/png-boundary.txt": "1aba30dc29b1bae3f7e1d8fb64a7e331effb85fa45a2f3a123ddbe7dd743c6ad",
"evidence/final-memory-validation-2026-11-07/png-http.json": "0dab50ef2d4ed22425d27ccdefaac316ed4b8c7b0a759c2c616eb9239ebba618",
"evidence/final-memory-validation-2026-11-07/png-http.txt": "68ae6e09e67a9df204f0346ebec3b9b185bcf2c67a0a0946aff620408ba867a2",
"evidence/final-memory-validation-2026-11-07/reset-worker-peak.txt": "97c9b5d430649b61f83fb7904414690f50e742b0b53e39c9b4324d60c1b6b2f9",
"evidence/final-memory-validation-2026-11-07/retained-volumes.txt": "27347d997d6f8466d94b0a5bb06350f183c5e30ef9ad2c3858cdc24a316bf5e4",
"evidence/final-memory-validation-2026-11-07/running-containers-after.txt": "44b284098615df613f91d9bc745f236f6ede62f1ea95a9de06e10ed712ffb799",
"evidence/final-memory-validation-2026-11-07/runtime.txt": "1f35a3f37a7e4d4689c8154384b08c9800077d4f77251d576f03921605a28b2e",
"evidence/final-memory-validation-2026-11-07/scan-secrets.py": "27130687bae2d636d9780759559524b347ec3733aa2df722c837378f2d2f39b1",
"evidence/final-memory-validation-2026-11-07/secret-scan.json": "27f7763fd6af6a10d3ddd0c8ae0a3f71dcad4eeb000afa5af5379bb646fb8e30",
"evidence/final-memory-validation-2026-11-07/smoke.txt": "ba0b81113210431ecfe0a085024efd650b4963e1eaf21b2837f6163523a70afc",
"evidence/final-memory-validation-2026-11-07/stopped-containers.txt": "f543710a6a0f639673d62148635409ecec6bea71c8987dc036017d4692505bed",
"evidence/final-memory-validation-2026-11-07/student-test.txt": "1cf669fbca4aee91e87e74089522325f7214ad6406c031013a73c8b935593ac1",
"evidence/final-memory-validation-2026-11-07/targeted.py": "cba7c461fded4eb88247f113a2328780fe985a3c1bdc27a26df47277a2374ffc",
"evidence/final-memory-validation-2026-11-07/up.txt": "1fd736cd42363f734649be3011f92abee46bc9210b4fd513aa2daba9d5f4a666"
}
@@ -1,6 +0,0 @@
date: 2026-11-06T23:15:03Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ make test
python3 tests/smoke.py --base-url "http://127.0.0.1:18085"
PASS: lab5 public API scenario; infrastructure and grading evidence remain separate.
exit_code: 0 elapsed_seconds: 1.706
@@ -1,4 +0,0 @@
date: 2026-11-06T23:19:15Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ docker ps -a --filter label=com.docker.compose.project=mtusi2026lab5 --format '{{.Names}} {{.State}}'
exit_code: 0 elapsed_seconds: 0.028
@@ -1,20 +0,0 @@
date: 2026-11-06T22:54:40Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ make student-test 'PYTEST_ARGS=-q --tb=short'
.venv/bin/pytest -q --tb=short
........................................................................ [ 7%]
........................................................................ [ 14%]
........................................................................ [ 21%]
........................................................................ [ 29%]
........................................................................ [ 36%]
........................................................................ [ 43%]
........................................................................ [ 51%]
........................................................................ [ 58%]
........................................................................ [ 65%]
........................................................................ [ 73%]
........................................s............................... [ 80%]
........................................................................ [ 87%]
......................................................................s. [ 94%]
.................................................. [100%]
984 passed, 2 skipped, 19 deselected in 171.26s (0:02:51)
exit_code: 0 elapsed_seconds: 172.902
@@ -1,310 +0,0 @@
import hashlib
import json
import os
import struct
import sys
import time
from io import BytesIO
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
sys.path[:0] = [str(ROOT / "scripts"), str(ROOT / "student-tests")]
import pipeline_suite as ps
from PIL import Image
from messenger.attachments.infrastructure.images import RenderLimits, render_variants
from png_trns_fixtures import png_with_trns
MODE = sys.argv.pop(1)
OUT = Path(__file__).parent / (MODE + ".json")
if OUT.exists():
raise SystemExit("Result exists; do not repeat completed evidence")
SUITE = ps.Suite(ps.arguments())
STACK = SUITE.stack
REPORT = {
"mode": MODE,
"source_sha": ps.run(["git", "rev-parse", "HEAD"]).strip(),
"started_at": ps.now(),
"cases": [],
}
LIMITS = RenderLimits(10485760, 40000000)
def save():
REPORT["accepted_ids"] = SUITE.api.accepted if SUITE.api else []
OUT.write_text(json.dumps(REPORT, ensure_ascii=False, indent=2) + "\n")
def case(name, function):
row = {"name": name, "started_at": ps.now()}
REPORT["cases"].append(row)
save()
began = time.monotonic()
try:
row["observed"] = function()
row["result"] = "PASS"
except Exception as error:
row.update(result="FAIL", error=str(error))
row["elapsed_seconds"] = round(time.monotonic() - began, 3)
save()
print(name, row["result"], row.get("error", ""), flush=True)
def containers():
return STACK.dc("ps", "-a", "--format", "{{.Name}} {{.ID}} {{.State}} {{.Status}}").splitlines()
def png(name, data, reference):
att = SUITE.api.upload(data, name=name + ".png")
ident = att["id"]
results = []
for version in (1, 2):
if version == 2:
STACK.admin("reprocess", "--attachment-id", ident, "--pipeline-version", "2")
ps.poll(lambda: SUITE.record(ident)["published_version"] == 2, 90, "published v2")
ready = SUITE.api.wait(ident)
SUITE.verify(ready)
assert SUITE.api.content(ident)["bytes"] == data
expected = {r.name: r for r in render_variants(reference, LIMITS, version)}
variants = []
for variant in ready["variants"]:
content = SUITE.api.content(ident, variant["name"])["bytes"]
assert content == expected[variant["name"]].data, (
variant["name"] + " differs from independent white reference"
)
with Image.open(BytesIO(content)) as image:
assert not image.getexif()
assert not any(key in image.info for key in ("icc_profile", "xmp", "transparency"))
variants.append(
{
"name": variant["name"],
"sha256": hashlib.sha256(content).hexdigest(),
"bytes": len(content),
"dimensions": [variant["width"], variant["height"]],
}
)
results.append({"version": version, "variants": variants})
before = SUITE.snapshot(ident)
STACK.publish(SUITE.payload(ident, 1))
STACK.publish(SUITE.payload(ident, 2))
SUITE.drained()
assert SUITE.snapshot(ident) == before
assert len(STACK.sql("variants", id=ident)) == 6
return {
"attachment_id": ident,
"input_sha256": hashlib.sha256(data).hexdigest(),
"input_bytes": len(data),
"versions": results,
"stale_and_duplicate_noop": True,
}
FIXTURES = ROOT / "evidence/final-validation-2026-11-07/fixtures"
ps.SQL["timed_events"] = (
"SELECT attachment_id,job_id,kind,at FROM job_events WHERE attachment_id = ANY(:'ids'::uuid[]) AND kind IN ('claimed','done') ORDER BY id"
)
def without_trns(data):
result, offset = data[:8], 8
while offset < len(data):
size = struct.unpack(">I", data[offset : offset + 4])[0] + 12
if data[offset + 4 : offset + 8] != b"tRNS":
result += data[offset : offset + size]
offset += size
return result
def verify_large(ident, data, transparent):
ready = SUITE.api.wait(ident)
SUITE.verify(ready)
assert SUITE.api.content(ident)["bytes"] == data
variants = []
for variant in ready["variants"]:
content = SUITE.api.content(ident, variant["name"])["bytes"]
with Image.open(BytesIO(content)) as image:
left = image.getpixel((5, image.height // 2))
right = image.getpixel((image.width - 6, image.height // 2))
assert (left == (255, 255, 255)) == transparent
assert right != (255, 255, 255)
assert not image.getexif()
variants.append({**variant, "left_pixel": left, "right_pixel": right})
return variants
def duplicate_check(ident):
before = SUITE.snapshot(ident)
STACK.publish(SUITE.payload(ident, 1))
STACK.publish(SUITE.payload(ident, 2))
SUITE.drained()
assert SUITE.snapshot(ident) == before
assert len(STACK.sql("variants", id=ident)) == 6
def large(name, data, transparent=True):
att = SUITE.api.upload(data, name=name + ".png")
ident = att["id"]
versions = [{"version": 1, "variants": verify_large(ident, data, transparent)}]
STACK.admin("reprocess", "--attachment-id", ident, "--pipeline-version", "2")
ps.poll(lambda: SUITE.record(ident)["published_version"] == 2, 90, "v2 publication")
versions.append({"version": 2, "variants": verify_large(ident, data, transparent)})
duplicate_check(ident)
return {
"attachment_id": ident,
"input_sha256": hashlib.sha256(data).hexdigest(),
"bytes": len(data),
"versions": versions,
"memory_peak": int(STACK.cgroup("worker", "memory.peak")),
"memory_max": int(STACK.cgroup("worker", "memory.max")),
}
def concurrent():
data = (FIXTURES / "rgb16-40mp.png").read_bytes()
SUITE.drained()
STACK.stop("worker")
try:
ids = [SUITE.api.upload(data, name="concurrent-rgb16.png")["id"] for _ in range(2)]
SUITE.queued(2)
finally:
ps.run(["docker", "start", STACK.dc("ps", "-aq", "worker").strip()])
versions = []
for version in (1, 2):
if version == 2:
STACK.stop("worker")
try:
for ident in ids:
STACK.admin("reprocess", "--attachment-id", ident, "--pipeline-version", "2")
SUITE.queued(2)
finally:
ps.run(["docker", "start", STACK.dc("ps", "-aq", "worker").strip()])
for ident in ids:
ps.poll(
lambda: SUITE.record(ident)["published_version"] == version,
90,
"concurrent publication",
)
variants = {ident: verify_large(ident, data, True) for ident in ids}
events = STACK.sql("timed_events", ids="{" + ",".join(ids) + "}")
job_ids = {
row["job_id"]
for ident in ids
for row in STACK.sql("jobs", id=ident)
if row["pipeline_version"] == version
}
selected = [e for e in events if e["job_id"] in job_ids]
claims = [e["at"] for e in selected if e["kind"] == "claimed"]
done = [e["at"] for e in selected if e["kind"] == "done"]
assert len(claims) == len(done) == 2
assert max(claims) < min(done), selected
versions.append(
{
"version": version,
"events": selected,
"processing_overlapped": True,
"variants": variants,
}
)
for ident in ids:
duplicate_check(ident)
peak, limit = (int(STACK.cgroup("worker", name)) for name in ("memory.peak", "memory.max"))
assert peak < limit
return {
"attachment_ids": ids,
"versions": versions,
"memory_peak": peak,
"memory_max": limit,
"memory_events": STACK.cgroup("worker", "memory.events"),
}
def forced(name, **overrides):
data = (FIXTURES / "rgb16-40mp.png").read_bytes()
result = SUITE.limited(data, **overrides)
STACK.restore()
result["next_normal"] = SUITE.smoke_pipeline()
return result
try:
SUITE.detect_api()
SUITE.api = ps.Api(SUITE.args)
dead_before = STACK.queues()[ps.DLQ][0]
REPORT["conditions"] = {
"host_load": os.getloadavg(),
"containers": containers(),
"running": ps.run(["docker", "ps", "--format", "{{.Names}} {{.Status}}"]),
"worker_limits": STACK.dc(
"exec",
"-T",
"worker",
"printenv",
"RENDER_MEMORY_MB",
"RENDER_TIMEOUT_SECONDS",
"WORKER_CONCURRENCY",
),
}
save()
if MODE == "png-http":
folder = ROOT / "evidence/review-trns-depth-2026-11-06/fixtures"
for name in ("gray2", "gray4", "rgb16"):
case(
name,
lambda name=name: png(
name,
(folder / (name + ".png")).read_bytes(),
(folder / (name + "-white.png")).read_bytes(),
),
)
for name, depth, color in [
("gray1", 1, (0,)),
("gray8", 8, (85,)),
("gray16", 16, (12345,)),
("rgb8", 8, (255, 0, 0)),
]:
case(
name,
lambda name=name, depth=depth, color=color: png(name, *png_with_trns(depth, color)),
)
case(
"rgb16-adam7-filter4-exif6",
lambda: png(
"adam7",
*png_with_trns(
16,
(12345, 23456, 34567),
size=(65, 63),
interlace=1,
filtering=4,
orientation=6,
),
),
)
elif MODE == "png-boundary":
for name in ("gray2", "gray4", "gray16", "rgb16"):
case(
name, lambda name=name: large(name, (FIXTURES / (name + "-40mp.png")).read_bytes())
)
case(
"gray16-without-trns",
lambda: large(
"gray16-opaque", without_trns((FIXTURES / "gray16-40mp.png").read_bytes()), False
),
)
case("concurrent-rgb16", concurrent)
case("forced-timeout", lambda: forced("timeout", FAULT_RENDER_TIMEOUT_SECONDS="0.01"))
case("forced-memory", lambda: forced("memory", FAULT_RENDER_MEMORY_MB="200"))
ids = "{" + ",".join(SUITE.api.accepted) + "}"
ps.poll(lambda: not STACK.sql("staging_left", ids=ids), 45, "staging sweep")
SUITE.drained()
REPORT["invariants"] = SUITE.invariants(dead_before)
finally:
STACK.restore()
REPORT["finished_at"] = ps.now()
REPORT["result"] = (
"PASS"
if REPORT["cases"]
and all(x.get("result") == "PASS" for x in REPORT["cases"] + REPORT.get("invariants", []))
else "FAIL"
)
save()
raise SystemExit(0 if REPORT["result"] == "PASS" else 1)
@@ -1,94 +0,0 @@
date: 2026-11-06T23:12:23Z commit: 74c33160ac01618f70e9e1c6a121a5e45c3c7155
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-memory-validation"}
$ docker compose up -d --wait
Network mtusi2026lab5_backend Creating
Network mtusi2026lab5_backend Created
Network mtusi2026lab5_edge Creating
Network mtusi2026lab5_edge Created
Container mtusi2026lab5-valkey-1 Creating
Container mtusi2026lab5-broker-1 Creating
Container mtusi2026lab5-lb-1 Creating
Container mtusi2026lab5-postgres-1 Creating
Container mtusi2026lab5-s3-1 Creating
Container mtusi2026lab5-s3-1 Created
Container mtusi2026lab5-valkey-1 Created
Container mtusi2026lab5-s3-init-1 Creating
Container mtusi2026lab5-lb-1 Created
Container mtusi2026lab5-broker-1 Created
Container mtusi2026lab5-postgres-1 Created
Container mtusi2026lab5-migrate-1 Creating
Container mtusi2026lab5-s3-init-1 Created
Container mtusi2026lab5-migrate-1 Created
Container mtusi2026lab5-api-2 Creating
Container mtusi2026lab5-worker-1 Creating
Container mtusi2026lab5-api-1 Creating
Container mtusi2026lab5-relay-1 Creating
Container mtusi2026lab5-api-2 Created
Container mtusi2026lab5-relay-1 Created
Container mtusi2026lab5-api-1 Created
Container mtusi2026lab5-worker-1 Created
Container mtusi2026lab5-broker-1 Starting
Container mtusi2026lab5-valkey-1 Starting
Container mtusi2026lab5-postgres-1 Starting
Container mtusi2026lab5-s3-1 Starting
Container mtusi2026lab5-lb-1 Starting
Container mtusi2026lab5-s3-1 Started
Container mtusi2026lab5-s3-1 Waiting
Container mtusi2026lab5-postgres-1 Started
Container mtusi2026lab5-postgres-1 Waiting
Container mtusi2026lab5-valkey-1 Started
Container mtusi2026lab5-broker-1 Started
Container mtusi2026lab5-lb-1 Started
Container mtusi2026lab5-postgres-1 Healthy
Container mtusi2026lab5-migrate-1 Starting
Container mtusi2026lab5-migrate-1 Started
Container mtusi2026lab5-s3-1 Healthy
Container mtusi2026lab5-s3-init-1 Starting
Container mtusi2026lab5-s3-init-1 Started
Container mtusi2026lab5-s3-init-1 Waiting
Container mtusi2026lab5-migrate-1 Waiting
Container mtusi2026lab5-broker-1 Waiting
Container mtusi2026lab5-migrate-1 Waiting
Container mtusi2026lab5-s3-init-1 Waiting
Container mtusi2026lab5-broker-1 Waiting
Container mtusi2026lab5-migrate-1 Waiting
Container mtusi2026lab5-s3-init-1 Waiting
Container mtusi2026lab5-s3-init-1 Exited
Container mtusi2026lab5-migrate-1 Exited
Container mtusi2026lab5-s3-init-1 Exited
Container mtusi2026lab5-s3-init-1 Exited
Container mtusi2026lab5-migrate-1 Exited
Container mtusi2026lab5-migrate-1 Exited
Container mtusi2026lab5-api-1 Starting
Container mtusi2026lab5-broker-1 Healthy
Container mtusi2026lab5-relay-1 Starting
Container mtusi2026lab5-broker-1 Healthy
Container mtusi2026lab5-worker-1 Starting
Container mtusi2026lab5-api-1 Started
Container mtusi2026lab5-api-2 Starting
Container mtusi2026lab5-worker-1 Started
Container mtusi2026lab5-relay-1 Started
Container mtusi2026lab5-api-2 Started
Container mtusi2026lab5-relay-1 Waiting
Container mtusi2026lab5-postgres-1 Waiting
Container mtusi2026lab5-migrate-1 Waiting
Container mtusi2026lab5-s3-1 Waiting
Container mtusi2026lab5-lb-1 Waiting
Container mtusi2026lab5-worker-1 Waiting
Container mtusi2026lab5-broker-1 Waiting
Container mtusi2026lab5-valkey-1 Waiting
Container mtusi2026lab5-s3-init-1 Waiting
Container mtusi2026lab5-api-1 Waiting
Container mtusi2026lab5-api-2 Waiting
Container mtusi2026lab5-migrate-1 Exited
Container mtusi2026lab5-s3-1 Healthy
Container mtusi2026lab5-s3-init-1 Exited
Container mtusi2026lab5-broker-1 Healthy
Container mtusi2026lab5-postgres-1 Healthy
Container mtusi2026lab5-valkey-1 Healthy
Container mtusi2026lab5-api-1 Healthy
Container mtusi2026lab5-api-2 Healthy
Container mtusi2026lab5-worker-1 Healthy
Container mtusi2026lab5-relay-1 Healthy
Container mtusi2026lab5-lb-1 Healthy
exit_code: 0 elapsed_seconds: 11.284
@@ -1,60 +0,0 @@
# Финальная validation ЛР5
Измерен HEAD `b30e2f6a84419d1ca64525ffb40df92183cccaef`; runtime совпадает с прошедшим review `42575281b352d35bda17ee7601bae5548a996e91`. Изменяются только документы и evidence. Полные три повтора трека 2 не запускались заново: применимость и первичные хеши проверены в [аудите](../load-validation.md), [audit.json](audit.json) и [historical-hashes.json](historical-hashes.json).
## Условия
Дата каталога — 2026-11-07 Europe/Moscow; сырые UTC-метки начинаются 2026-11-06T21:15. Darwin arm64, Docker 28.3.2, Linux VM 16 CPU / 8 217 190 400 байт RAM. Основной project `mtusi2026lab5` изолирован собственными сетями и четырьмя сохранёнными volumes. HTTP 18085, stats 25085, broker UI 25086, только loopback. Чужие контейнеры не запускались и не останавливались.
Сборка и начальные функциональные проверки частично совпали по времени с собственным `make student-test` (его одноразовые контейнеры отражены в `postgres-outage*.json`). Перед HTTP PNG и проверками памяти student-test завершился, посторонних работающих Docker-контейнеров нет. Нагрузка хоста фиксируется срезами, непрерывного контроля CPU нет; RPS, p95 и новые SLO не заявляются.
Точные лимиты, image IDs и volumes — [runtime-containers.json](runtime-containers.json). Worker 2 CPU / 1 GiB, render 448 MiB / 30 с, concurrency=2; API ×2 по 1 CPU / 512 MiB. Реальные версии и хеш установленного images.py — [image-runtime.txt](image-runtime.txt). Конфигурация запуска — [env.sh](env.sh), без секретов. Fault-suite использует штатный `infra/compose.faults.yaml` и восстанавливает настройки.
## Выполнено
Все `.txt` содержат команду, исходный SHA, UTC и exit code. Команды выполняются из корня после `source evidence/final-validation-2026-11-07/env.sh`.
| Команда / файл | Наблюдение |
| --- | --- |
| `make check validate lint`, [checks.txt](checks.txt) | PASS |
| `make student-test PYTEST_ARGS='-q --tb=short'`, [student-test.txt](student-test.txt) | 980 passed, 1 Linux-only skip, 19 stack deselected; real-broker тесты включены |
| `openspec validate --all --strict`; `openspec list --json`, [openspec.txt](openspec.txt), [active-changes.txt](active-changes.txt) | 22 PASS, активных changes нет; новый change не создаётся для docs/evidence |
| `docker compose build`; `docker compose up -d --wait`, [build.txt](build.txt), [up.txt](up.txt) | PASS; сохранённые volumes, без удаления очередей/данных |
| `make test`, [smoke.txt](smoke.txt) | Публичный HTTP smoke PASS |
| `rabbitmqctl list_policies/list_queues`, [policy.txt](policy.txt), [queues.txt](queues.txt) | Эффективная policy `delivery-limit=-1` на существующей quorum queue; исходная DLQ=15 |
| `.venv/bin/python evidence/final-validation-2026-11-07/targeted.py postgres-outage-retry`, [команда](postgres-outage-retry.txt), [результат](postgres-outage-retry.json), [сырые requeue](postgres-outage-worker.log) | PASS: настоящий PostgreSQL exited, 12 `job.requeued`, затем ready без replay; job_id и attempt=1 сохранены, три варианта, DLQ=15; 8 инвариантов PASS |
| `targeted.py png-http`, [команда](png-http.txt), [результат](png-http.json) | 8 PNG × v1/v2 × 3 варианта = 48 побайтных сравнений с независимым белым эталоном PASS; оригиналы неизменны, metadata/hash/геометрия, удаление EXIF, stale/duplicate noop; 8 инвариантов PASS |
| `pipeline_suite.py --only …`, [команда](delivery-targeted.txt), [результат](delivery-targeted.json) | 12 адресных сценариев и 8 инвариантов PASS; 11 ID, 47 S3-объектов. DLQ 15 → 16 только от ожидаемого malformed message |
| `targeted.py png-limits`, [команда](png-limits.txt), [результат](png-limits.json) | Четыре FAIL для 40 Мп tRNS при штатных пределах; принудительные timeout/memory и последующее обычное задание PASS. Не переименовывать весь запуск в PASS |
PNG HTTP включает grayscale 1/2/4/8/16, RGB8/16, точного непрозрачного соседа 12346 и RGB16 с Adam7/filter=4/EXIF=6. Новые синтетические 40 Мп fixtures сохранены в [fixtures](fixtures); генератор — `large_png` в [targeted.py](targeted.py).
Policy сверена через Context7 `/rabbitmq/rabbitmq-website`; точная семантика 4.1 дополнительно проверена по [RabbitMQ 4.1](https://www.rabbitmq.com/docs/4.1/quorum-queues#poison-message-handling). Динамическое доказательство относится именно к RabbitMQ 4.1.8, а не к изменённой семантике nack в 4.3.
## Непригодные промежуточные проверки
Первый [postgres-outage.txt](postgres-outage.txt) имеет exit 1: `docker compose start worker` запустил зависимости, включая PostgreSQL, поэтому 12 ошибок не возникли. Это ошибка новой проверочной обвязки, а не дефект приложения. Исходная обвязка сохранена в [targeted-initial.py](targeted-initial.py). Повторён только этот сценарий с `docker start` точного ID своего worker; никаких старых пригодных серий не повторялось.
В `png-limits.json` проверка `no_orphans` выполнена до очередного штатного sweep: после последнего ready ещё оставался staging. Остальные семь инвариантов PASS. Read-only [cleanup-recheck](cleanup-recheck.txt) после sweep подтвердил отсутствие pending outbox/staging/nonterminal/дублей и точное совпадение 10 объектов S3 с БД для семи ID. Тяжёлые PNG не повторялись; исходный `no_orphans FAIL` не переписан.
## Незавершённая квалификация
Четыре валидных PNG 8000×5000 с tRNS приняты с 202, но дали failed/processing_failed на первой попытке. Worker жив, последующая обычная загрузка ready. **Major: не выполнен объявленный сценарий «Изображение на пределе пикселей» [OpenSpec](../../openspec/specs/lab5-image-variants/spec.md): PNG 8000×5000 при штатных пределах должен дать ready.** Это вывод о конкретном граничном сценарии, не заявление о потере данных или падении сервиса. Полное подтверждение «5» и готовность к публикации блокируются до отдельного fix/re-review. Реализация не менялась.
Пять прежних minor и документационная неточность `201` в openspec/config.yaml сохраняются в REPORT. Публикация и Telegram не выполнялись.
[resource-probe-worker.txt](resource-probe-worker.txt) воспроизводит `MemoryError` при 448 MiB и загруженных модулях worker: grayscale 2/4 — `flatten` при выделении белого RGB-фона, grayscale16 — преобразование `I → L`, RGB16 — маски/повторное декодирование в `apply_png_transparency`. Без tRNS gray2/gray4/RGB16 обрабатываются, gray16 всё ещё превышает предел. В облегчённом процессе без импортов worker [resource-probe-retry.txt](resource-probe-retry.txt) проходит grayscale, но падает RGB16: такой изолированный результат не заменяет HTTP-проверку реального worker. RSS пробника — наблюдение процесса, а ограничение — виртуальное адресное пространство RLIMIT_AS.
При штатном HTTP-прогоне peak контейнера составил 640 196 608 байт < 1 GiB; `MemoryError` возникает внутри ограниченного дочернего процесса. Увеличение лимита без учёта concurrency=2 и памяти worker не считается готовым решением. После исправления нужны ресурсная проверка новых PNG, отсутствие поломки точного tRNS и целевые регрессии. Полную принятую track2-матрицу повторять без нового основания не требуется.
Первая попытка скопировать диагностические fixtures через `docker compose cp` отклонена read-only rootfs (`probe-copy-*.txt`), последующий запуск отсутствующего файла завершился exit 2 (`resource-probe.txt`). Затем [invoke-probe.py](invoke-probe.py) передал только синтетические fixtures и код через stdin, записал их в разрешённый tmpfs `/tmp` и выполнил пробник. Эти ошибки обвязки не скрыты и не названы тестовыми PASS.
## Завершение и передача
Итог — **1 major, 6 сохранённых minor**, [findings.json](findings.json). Фаза validation заблокирована граничной обработкой PNG; назначенный агент должен исправить реализацию через OpenSpec и пройти review. В этой фазе новых изменений реализации/конфигурации/OpenSpec нет. Check/validate/lint, полный student-test и остальные адресные результаты сохраняются для аудита применимости, не становятся автоматически недействительными после документационного коммита.
`make down` без `-v` выполнен: [down.txt](down.txt), [stopped-containers.txt](stopped-containers.txt); контейнеров своего project нет. Четыре постоянных volume сохранены — [retained-volumes.txt](retained-volumes.txt). Итоговый список работающих Docker-контейнеров пуст — [running-containers-after.txt](running-containers-after.txt). Фоновых агентов и процессов этап не оставил. Push и Telegram не выполнялись.
Финальные `make check validate lint` — PASS ([checks-final.txt](checks-final.txt)); новый код evidence разобран Python AST. Проверены локальные ссылки и совпадение runtime с review, нормативных файлов со skeleton ([final-doc-audit.json](final-doc-audit.json)). Проверка tracked файлов и новых evidence по восьми шаблонам секретов не обнаружила реальных credentials; семь совпадений проверены как skeleton placeholders/тестовые значения ([secret-scan.json](secret-scan.json)). Это проверка содержимого, без чтения `.env` и `secrets/`. Концевые пробелы в новых текстовых логах нормализованы; результаты/значения не изменены, историческое сырьё не редактировалось.
@@ -1,11 +0,0 @@
date: 2026-11-06T21:15:54Z commit: b30e2f6a84419d1ca64525ffb40df92183cccaef
environment: {}
$ openspec list --json
{
"changes": [],
"root": {
"path": "<repo>",
"source": "nearest"
}
}
exit_code: 0 elapsed_seconds: 0.794
@@ -1,142 +0,0 @@
{
"head": "b30e2f6a84419d1ca64525ffb40df92183cccaef",
"review_runtime_equal": true,
"runtime_diff_from_D": [
"infra/rabbitmq/definitions.json",
"src/messenger/attachments/infrastructure/images.py"
],
"platform": "macOS-15.1-arm64-arm-64bit-Mach-O",
"host_load": [
3.61962890625,
2.6328125,
2.22509765625
],
"docker": "28.3.2 16 8217190400",
"retained_volumes": [
"mtusi2026lab5_broker-data",
"mtusi2026lab5_pgdata",
"mtusi2026lab5_s3data",
"mtusi2026lab5_valkeydata"
],
"series": [
{
"name": "level4",
"source_sha": "0e921116610d289716af0a40ec823376eb8bbcbc",
"scenarios": 17,
"invariants": 8,
"accepted": 33,
"config": {
"base_url": "http://127.0.0.1:18085",
"level": 4,
"only": "",
"expect_variants": "thumbnail,crop,optimized",
"max_attempts": 5,
"stall_seconds": 15.0,
"compose": "docker compose",
"compose_faults": "docker compose -f compose.yaml -f infra/compose.faults.yaml"
}
},
{
"name": "track-run1",
"source_sha": "0e921116610d289716af0a40ec823376eb8bbcbc",
"scenarios": 13,
"invariants": 8,
"accepted": 13,
"config": {
"base_url": "http://127.0.0.1:18085",
"level": 5,
"only": "smoke_pipeline,duplicate_after_ready,crash_commit_publish,crash_publish_confirm,crash_after_s3_before_commit,crash_after_commit_before_ack,lease_expiry_fencing,crash_before_side_effect,worker_kill_mid_task,duplicate_in_flight,reorder_attempts,reorder_versions,stale_after_new_version",
"expect_variants": "thumbnail,crop,optimized",
"max_attempts": 5,
"stall_seconds": 15.0,
"compose": "docker compose",
"compose_faults": "docker compose -f compose.yaml -f infra/compose.faults.yaml"
}
},
{
"name": "track-run2",
"source_sha": "0e921116610d289716af0a40ec823376eb8bbcbc",
"scenarios": 13,
"invariants": 8,
"accepted": 13,
"config": {
"base_url": "http://127.0.0.1:18085",
"level": 5,
"only": "smoke_pipeline,duplicate_after_ready,crash_commit_publish,crash_publish_confirm,crash_after_s3_before_commit,crash_after_commit_before_ack,lease_expiry_fencing,crash_before_side_effect,worker_kill_mid_task,duplicate_in_flight,reorder_attempts,reorder_versions,stale_after_new_version",
"expect_variants": "thumbnail,crop,optimized",
"max_attempts": 5,
"stall_seconds": 15.0,
"compose": "docker compose",
"compose_faults": "docker compose -f compose.yaml -f infra/compose.faults.yaml"
}
},
{
"name": "track-run3",
"source_sha": "0e921116610d289716af0a40ec823376eb8bbcbc",
"scenarios": 13,
"invariants": 8,
"accepted": 13,
"config": {
"base_url": "http://127.0.0.1:18085",
"level": 5,
"only": "smoke_pipeline,duplicate_after_ready,crash_commit_publish,crash_publish_confirm,crash_after_s3_before_commit,crash_after_commit_before_ack,lease_expiry_fencing,crash_before_side_effect,worker_kill_mid_task,duplicate_in_flight,reorder_attempts,reorder_versions,stale_after_new_version",
"expect_variants": "thumbnail,crop,optimized",
"max_attempts": 5,
"stall_seconds": 15.0,
"compose": "docker compose",
"compose_faults": "docker compose -f compose.yaml -f infra/compose.faults.yaml"
}
}
],
"evidence_sha256": {
"evidence/forward-final-2026-11-06/upload-negative.txt": "6cb7ad695b0715a2865e81d113bff729bf98d106a29c419e2ff16f7ee067c4b8",
"evidence/forward-final-2026-11-06/baseline.txt": "f6520adb750693e1c7ad31cdad2d1de327a302d15a17a763a035f35ea5b759f5",
"evidence/forward-final-2026-11-06/student-test.txt": "2976f98e550a14ee2c385f8a1b34a37626fdc602dfc6f848fbf81b0f105834fc",
"evidence/forward-final-2026-11-06/summary.json": "406cea849c5a294a7a744e18056ed54a554cccdf3f5b1133d580ea67cdc1a57f",
"evidence/forward-final-2026-11-06/track-run1.json": "5e90e87bcbf43115df39a37b95f50141772d92e3005f445e1a0c9b5d04050ad0",
"evidence/forward-final-2026-11-06/baseline-regressions.txt": "b0525cea37ce47958dc95e02866c1f014239a6cc8500cf76bf448d2b4f1423c9",
"evidence/forward-final-2026-11-06/conditions-level4-middle.json": "677c0e3499a45daf06cc5bc6f46338ceb873f4203ae137b8cb1c9606f13538eb",
"evidence/forward-final-2026-11-06/lint-runtime.txt": "5e3c5752c468b6f747e4dbafd91954660f8b80f820b63ff34704641e7d5db970",
"evidence/forward-final-2026-11-06/final-containers.txt": "e2ea3d831281e9cc6cc9b03ea1f00172e4eda23fb7244b8d9b60f7d211b5ec64",
"evidence/forward-final-2026-11-06/down.txt": "9dbe77c3dc7b1a17798bd117fd1c8b4cb9ab01a6122376ca5ab38d147ac7b9ff",
"evidence/forward-final-2026-11-06/stopped-containers.txt": "f7ac8e8c29b18a83193a089a2dd75aa0dfc103e54173e2421b853c00a0113338",
"evidence/forward-final-2026-11-06/unit-regressions.txt": "26d5cf96bdf415c5b17fe2a596b9228834995418b1b33a83c146f59617debc5a",
"evidence/forward-final-2026-11-06/track-run1.txt": "7966b1f45232223055f58b0031a6eeee987d42d5789798ec17df711f40c207ac",
"evidence/forward-final-2026-11-06/track-run3.txt": "4b8254db242784c6db3a0db7a8e363269a1aa486a1b26caaee254100f1ab0a52",
"evidence/forward-final-2026-11-06/source-inventory.json": "8977c7883b08b4a2ce7385ddf1c8da11bb4344e586dc71ba637118fcf9d8f506",
"evidence/forward-final-2026-11-06/conditions-stopped.json": "5ed4cc9e6c6969cd16b699a24ac747c7b4ee4c8b189be2f57cd610622891f00d",
"evidence/forward-final-2026-11-06/forward-audit.json": "675a9fcf1fb817bb043c8bcde35079617294da9c7832a529c4d7bfa2adc8f52d",
"evidence/forward-final-2026-11-06/conditions-before.json": "4ef9320729dc38a49b7e8dc3338a862fd69946ceaf64b4aed004bc6ac23ab1cc",
"evidence/forward-final-2026-11-06/track-run2.txt": "1ff4b753ab5627ee5393977203d5dc83f59d4c9e990e7171fc63a06061ceecb3",
"evidence/forward-final-2026-11-06/level4-command.txt": "2b9e49afc9e9ec0e8a8f2a3c491906b72c7439b4a2e30ff6ed0a96d9be4c73ed",
"evidence/forward-final-2026-11-06/level4.json": "904e2c40a6737947b1867c9051a36920c007f49ad59de057a168c1dcc345d87c",
"evidence/forward-final-2026-11-06/archive.txt": "54bc70e53d69969580bd0c88f373778f2a029e2329bbccac7844ff16055deceb",
"evidence/forward-final-2026-11-06/conditions-track-after1.json": "65e7d5a4652e102e6ad8ecbf74356b1c468e2d5ddaee492af21432d646aa4bdf",
"evidence/forward-final-2026-11-06/up.txt": "0baaa49727722eaf24ac4d8b3e083c09bbb53e278e557e42551221d0c59e5856",
"evidence/forward-final-2026-11-06/checks-before-commit.txt": "1b8bd42a69e070cffb07e12449ec749066ec92407fa5d94bd8ad766f40d93dff",
"evidence/forward-final-2026-11-06/checks-final.txt": "d0d3a9f7a24914365c772eabceb4ad2238c5922e30c866a26c5274f8fb570e81",
"evidence/forward-final-2026-11-06/failover.txt": "fcd1b67b77aa7936f9f0ae1c48982e9e4ad9f2e8b9705d2b856ff8cf360fdaa9",
"evidence/forward-final-2026-11-06/persistence-containers.txt": "229bf4c9809cf0e6a1d9cfd162949b413d7f990f57a30fbcd5f9c5b6e3f81e92",
"evidence/forward-final-2026-11-06/conditions-track-start.json": "64e99f2110c3ccf029fba7fe30bd1c9f01615cb5b43ffd77fe4a0f09e7030245",
"evidence/forward-final-2026-11-06/final-smoke.txt": "eec6f2507465824cd920fe78bf876dfbf8b905e84ebcd7b9d51e80d16d4c064b",
"evidence/forward-final-2026-11-06/findings.json": "2f0db619bebbc615dc33f78599210201e1d9441139adb33687dea7a46fc1f15b",
"evidence/forward-final-2026-11-06/network-regressions.txt": "30f0735c6d0bd2370cf39ac6855c06750ba18b282b978dcf6dddcf293145e02b",
"evidence/forward-final-2026-11-06/conditions-track-after2.json": "fa3a1808cc215afb3a422fe582c2d4cae5cf480411f4150977be30f9e24bf462",
"evidence/forward-final-2026-11-06/conditions-track-after3.json": "1054464f5526ee719579fbda81b9e7605a97a660fec9bdfc52f6e68ce94cfad1",
"evidence/forward-final-2026-11-06/specs-validation.txt": "79fe9fdb0bb258d6d7788a79d96587b748233001e97f2acd38bf9bfe0376d515",
"evidence/forward-final-2026-11-06/cache-outage.txt": "fbe1eb591256b35a93a8fd3f4fb299caf22f57159f0b68c984f7c75e5e8be9fc",
"evidence/forward-final-2026-11-06/failover-containers.txt": "4468a88f349073027fb07b2c81406e480e1b503b64fb828e58ecee688d7b2091",
"evidence/forward-final-2026-11-06/clock-observation.json": "d7ae11b8c7970cc71775f46fa0a1474e113da4cae9229d8f28bf85312dd8acc4",
"evidence/forward-final-2026-11-06/conditions-level4-start.json": "0cb8a9314b3afd4704c762b1b611f80beb9f30795475642ad2a22452f87993c0",
"evidence/forward-final-2026-11-06/images.txt": "adb50435b1c24cc0deb913c49ed19e3fdd4287b73719d418accad013ebb7489d",
"evidence/forward-final-2026-11-06/retained-volumes.txt": "ef4b62f79ed1cd0dc1e7313d65873c6a6ab35ff933bd8673a3431d3df5691449",
"evidence/forward-final-2026-11-06/openspec-final.txt": "a34016edea7844300b98aaf38e713df8b4b217924b515881b33a599cd1603b24",
"evidence/forward-final-2026-11-06/track-run3.json": "d190b7a1b4962c520d46931dc63601ed2655f1b4063bc7fea01b78b049a0bede",
"evidence/forward-final-2026-11-06/source-name-status.txt": "81abf426ff31f0253a5ced9ae43fdbbf2a56460afe7b5d299a32c6f664196b93",
"evidence/forward-final-2026-11-06/haproxy-errors.txt": "bf3b0b33f997fefc542860389e8a0fcc1856bbb17cd25b6103911610e87806a3",
"evidence/forward-final-2026-11-06/persistence.txt": "06574967e1dab9742841d54cabb7b0109aa063316e08c8db5525234f084f92de",
"evidence/forward-final-2026-11-06/smoke.txt": "709ddeeb3aa1c4e0b516d4b66038b0421929ee519665d79feccc907cc7bc89ac",
"evidence/forward-final-2026-11-06/restore.txt": "96805ba0f8cba25cb90f385809d0a0a6c75b468a25f87006d729fedc51759c7d",
"evidence/forward-final-2026-11-06/track-run2.json": "16fe12bd42f958a0d4962182da65bd4ee69e4b896fbce47daea724b0a1c618e8"
}
}
@@ -1,90 +0,0 @@
date: 2026-11-06T21:16:11Z commit: b30e2f6a84419d1ca64525ffb40df92183cccaef
environment: {"COMPOSE_PROJECT_NAME": "mtusi2026lab5", "COMPOSE_FILE": "compose.yaml", "LB_HTTP_PORT": "18085", "LB_STATS_PORT": "25085", "BASE_URL": "http://127.0.0.1:18085", "NO_PROXY": "localhost,127.0.0.1,::1", "API_IMAGE": "mtusi-messenger:lab5-validation"}
$ docker compose build
#1 [internal] load local bake definitions
#1 reading from stdin 434B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.65kB done
#2 DONE 0.0s
#3 [internal] load metadata for ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142
#3 DONE 0.0s
#4 [internal] load metadata for docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d
#4 DONE 1.7s
#5 [internal] load .dockerignore
#5 transferring context: 453B done
#5 DONE 0.0s
#6 FROM ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142
#6 resolve ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 0.0s done
#6 DONE 0.0s
#7 [builder 1/7] FROM docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d
#7 resolve docker.io/library/python:3.14-slim-trixie@sha256:51dafde81dbdb6ebde285137a295cf18a47ca95234fe388a343719cb97305b3d 0.0s done
#7 DONE 0.0s
#8 [internal] load build context
#8 transferring context: 51.62kB 0.0s done
#8 DONE 0.0s
#9 [builder 2/7] COPY --from=ghcr.io/astral-sh/uv:0.9.7@sha256:ba4857bf2a068e9bc0e64eed8563b065908a4cd6bfb66b531a9c424c8e25e142 /uv /usr/local/bin/uv
#9 CACHED
#10 [builder 4/7] COPY pyproject.toml uv.lock README.md ./
#10 CACHED
#11 [builder 3/7] WORKDIR /build
#11 CACHED
#12 [builder 5/7] RUN uv sync --locked --no-dev --no-install-project --no-editable
#12 CACHED
#13 [builder 6/7] COPY src ./src
#13 DONE 0.1s
#14 [builder 7/7] RUN uv sync --locked --no-dev --no-editable
#14 0.280 Resolved 84 packages in 12ms
#14 0.309 Building mtusi-messenger @ file:///build
#14 1.851 Built mtusi-messenger @ file:///build
#14 1.855 Prepared 1 package in 1.54s
#14 1.859 Installed 1 package in 3ms
#14 2.080 Bytecode compiled 2301 files in 221ms
#14 2.080 + mtusi-messenger==1.5.0 (from file:///build)
#14 DONE 2.1s
#15 [runtime 2/6] RUN groupadd --system --gid 10001 messenger && useradd --system --uid 10001 --gid messenger --no-create-home --shell /usr/sbin/nologin messenger
#15 CACHED
#16 [runtime 3/6] COPY --from=builder --chown=root:root /opt/venv /opt/venv
#16 DONE 0.4s
#17 [runtime 4/6] WORKDIR /app
#17 DONE 0.0s
#18 [runtime 5/6] COPY --chown=root:root alembic.ini ./
#18 DONE 0.0s
#19 [runtime 6/6] COPY --chown=root:root alembic ./alembic
#19 DONE 0.0s
#20 exporting to image
#20 exporting layers
#20 exporting layers 3.0s done
#20 exporting manifest sha256:5fa47645c125c166f4f83308eeda33e4679510f9548cae36141b4bcff1bec3cb
#20 exporting manifest sha256:5fa47645c125c166f4f83308eeda33e4679510f9548cae36141b4bcff1bec3cb done
#20 exporting config sha256:34e5f914cd69e5cff0b7745f9730b4d63d0a62480ef53de7b18914787a51c828 done
#20 exporting attestation manifest sha256:043b67091fa616197172d98abd054f1cd803a4b540ceb0bc4a8e30232d37359d done
#20 exporting manifest list sha256:9293b3138e99506510e947ba0cfc77ed4301f96344dd937d51d03e8c258121f2 done
#20 naming to docker.io/library/mtusi-messenger:lab5-validation done
#20 unpacking to docker.io/library/mtusi-messenger:lab5-validation
#20 unpacking to docker.io/library/mtusi-messenger:lab5-validation 0.6s done
#20 DONE 3.6s
#21 resolving provenance for metadata file
#21 DONE 0.0s
api Built
exit_code: 0 elapsed_seconds: 9.093

Some files were not shown because too many files have changed in this diff Show More