- stop-one.sh: an empty TARGET_INSTANCE_ID now exits 2 in any shell,
as the design states (${VAR:?} gave 1 under bash).
- driver.py kill-inflight: reproducible "aborted response is not
retried" scenario (logins in flight, replica killed via stop-one.sh).
- Detection time of a hung replica is fall x (inter + timeout check) =
8 s, not inter x fall + timeout check = 5 s (docker pause: 7.3 s);
fixed in haproxy.cfg, balancer.md and the design.
- The 1 CPU replica limit is kept on purpose; the compose comment and
the design now say that Argon2id uses more and logins are slower.
- load-balancing spec: API_REPLICAS is read by compose and Makefile.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Evidence from 4e0cc4e: stand ports, limits and per-service env,
X-Instance-Id distribution and scaling, LB-generated errors,
test-persistence through the LB, test-failover for each replica in stop
and kill modes, stop-one refusals, test runs and bench self-checks.
REPORT.md gains the LB architecture, timeout chain, limits, criteria
rows and defence answers. Coverage exclusions for restart now point to
the lab 3 persistence evidence; stale lab 2 pointers go to
KNOWN_ISSUES.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without init-state down HAProxy marked a slot UP as soon as Docker DNS
returned the container address, before uvicorn listened, so requests
after LB restart or scale-up hit a replica that was not ready.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HAProxy 3.4 is the only published entry point; api runs as
API_REPLICAS replicas without ports, found through Docker DNS and
checked on /health/ready. Retries only on connection failure, LB errors
in the contract error format, resource limits and per-service secrets.
Adds scripts/stop-one.sh, LB-aware restart.sh, make replicas /
instances / lb_stats and replica tests driven by EXPECT_REPLICAS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HAProxy as the single entry point, API replicas without published
ports, resource limits and per-service secrets, restart and stop-one
hooks, replica tests in student-tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- evidence for replay, conflict, replay through a new API process and
after restarting API and storages, keys table, key race,
X-Instance-Id in responses and logs, explicit and invalid
instance_id and key TTL, test runs
- stand evidence: make wipe_data leaves the lab 2 volume intact, build
from a fresh git clone passes smoke, bench restore stage sends with
Idempotency-Key
- REPORT.md: X-Instance-Id, idempotency design, IDEMPOTENCY_TTL_SECONDS
mapping, limitations and evidence rows
- tasks.md: mark the change tasks done, add follow-ups from verify
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- carry the lab 2 solution over and plan lab 3 (PLAN.md, OpenSpec
change instance-id-idempotency)
- rename the compose project to mtusi-lab3 and the image to :lab3
- add X-Instance-Id to every response and instance_id to every log
record ([app] instance_id, hostname by default)
- require Idempotency-Key on POST messages: keys table in Postgres,
key and message in one transaction, replay returns the same 201,
another body gets 409, TTL from [idempotency] ttl_seconds
- send the key from student-tests, k6 scenarios and the skill driver;
add idempotency, X-Instance-Id, key race and fault tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>