Series a/b/pool/ab/quick in bench/run.py: per-cell fresh stand with the
configuration's replicas and max_connections, open-model mixed profile
(k6/mixed.js), fixed levels and R* search by SLO, a sampler container
for cgroup, VM, HAProxy and PostgreSQL metrics, resume and a stand
lock. Pure helpers in bench/scaling.py, SVG plots in bench/plots.py,
series mode for bench_compare and bench_ab. Hypotheses H1-H3 go to
REPORT.md before the first series.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HAProxy checks get fastinter 500ms and timeout check 2.5s (hung replica
detected within 6.5 s), http-reuse never with uvicorn keep-alive 60 s so
the LB closes idle replica connections, and app connections set
idle_in_transaction_session_timeout so a frozen replica releases its
idempotency key. The bench stand runs behind HAProxy with N replicas and
gains the failover profile (make bench_failover / bench_failover_quick):
constant-rate writes with a key and reads while each replica is stopped,
killed or paused, with per-second timeline, detection and recovery time
and write reconciliation. Driver commands idle-conns and hang-key check
the keep-alive and hung-transaction decisions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HAProxy 3.4 is the only published entry point; api runs as
API_REPLICAS replicas without ports, found through Docker DNS and
checked on /health/ready. Retries only on connection failure, LB errors
in the contract error format, resource limits and per-service secrets.
Adds scripts/stop-one.sh, LB-aware restart.sh, make replicas /
instances / lb_stats and replica tests driven by EXPECT_REPLICAS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- carry the lab 2 solution over and plan lab 3 (PLAN.md, OpenSpec
change instance-id-idempotency)
- rename the compose project to mtusi-lab3 and the image to :lab3
- add X-Instance-Id to every response and instance_id to every log
record ([app] instance_id, hostname by default)
- require Idempotency-Key on POST messages: keys table in Postgres,
key and message in one transaction, replay returns the same 201,
another body gets 409, TTL from [idempotency] ttl_seconds
- send the key from student-tests, k6 scenarios and the skill driver;
add idempotency, X-Instance-Id, key race and fault tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>