Configs may carry a fifth element, the PostgreSQL CPU limit, which goes
into the config key (-pg<CPU>), the budget and the cell stand. Series db:
1, 2, 4 replicas at the tight limit and a 4 x 1 control at 2 CPU; label
summaries show PostgreSQL CPU at R* against its limit and throttling.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
plots.py summary gains per-request CPU by component, PostgreSQL waits and
pool, H1b throttling and memory peaks; compare.py and plots.py headers
name the spec, change and host; errors logs match 5xx by status field and
keep the exception tail; series labels record sampler limits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Series a/b/pool/ab/quick in bench/run.py: per-cell fresh stand with the
configuration's replicas and max_connections, open-model mixed profile
(k6/mixed.js), fixed levels and R* search by SLO, a sampler container
for cgroup, VM, HAProxy and PostgreSQL metrics, resume and a stand
lock. Pure helpers in bench/scaling.py, SVG plots in bench/plots.py,
series mode for bench_compare and bench_ab. Hypotheses H1-H3 go to
REPORT.md before the first series.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/rolling-update.sh (make rolling_update) replaces API replicas one
by one: build, migrations once, new replica UP in HAProxy, old replica
drained and put into maintenance through the LB runtime API on loopback,
then stopped and removed; the slot is reset once DNS drops the address.
Expand migration adds messages.instance_id with a lock timeout. The bench
rollout profile runs the same script under constant load, starting from
the image of an older commit built with git archive.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HAProxy checks get fastinter 500ms and timeout check 2.5s (hung replica
detected within 6.5 s), http-reuse never with uvicorn keep-alive 60 s so
the LB closes idle replica connections, and app connections set
idle_in_transaction_session_timeout so a frozen replica releases its
idempotency key. The bench stand runs behind HAProxy with N replicas and
gains the failover profile (make bench_failover / bench_failover_quick):
constant-rate writes with a key and reads while each replica is stopped,
killed or paused, with per-second timeline, detection and recovery time
and write reconciliation. Driver commands idle-conns and hang-key check
the keep-alive and hung-transaction decisions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- carry the lab 2 solution over and plan lab 3 (PLAN.md, OpenSpec
change instance-id-idempotency)
- rename the compose project to mtusi-lab3 and the image to :lab3
- add X-Instance-Id to every response and instance_id to every log
record ([app] instance_id, hostname by default)
- require Idempotency-Key on POST messages: keys table in Postgres,
key and message in one transaction, replay returns the same 201,
another body gets 409, TTL from [idempotency] ttl_seconds
- send the key from student-tests, k6 scenarios and the skill driver;
add idempotency, X-Instance-Id, key race and fault tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>