Commit Graph
6 Commits
Author SHA1 Message Date
ArtyomandClaude Opus 5.5 dc2015473f Run the API as replicas behind HAProxy
compose.yaml now runs one api service with API_REPLICAS replicas and
no published ports, container name or hostname, behind lb, HAProxy
3.2.26 LTS pinned by digest and the only service with ports, bound to
127.0.0.1 (LB_HTTP_PORT for HTTP, LB_STATS_PORT for read-only stats).
The balancer finds replicas through Docker DNS with a server template,
checks GET /health/ready every second (fall 2, rise 2), balances round
robin without stickiness, retries connection failures on another
replica and allows a retry after a sent request only for GET and for
message posts carrying Idempotency-Key, never for users, chats or
login. Its own 502/503/504 answers use the contract error body with
the forwarded X-Request-Id.

The network is split into edge and an internal backend, every service
gets CPU and memory limits, the API runs read-only with a tmpfs /tmp,
two workers per replica, an 8 s request limit and a 5 s graceful stop
inside an 8 s stop_grace_period. Only api builds the image, migrate
reuses it once before the replicas start. PostgreSQL loads
pg_stat_statements and allows 200 connections. The short TTL instance
of lab 2 published the API around the balancer and is removed; its
expiry check stays as an in-process test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:43:43 +03:00
ArtyomandClaude Opus 5.5 7bf28fc6ed Make message posts idempotent with Idempotency-Key
POST /api/v1/chats/{chat_id}/messages now requires the header, checked
at the HTTP boundary (1-128 of [A-Za-z0-9._:-], else 400). The handler
keeps the lab 2 order under the chat row lock: membership, then text,
then the receipt for (chat, sender, key) younger than
IDEMPOTENCY_TTL_SECONDS. A receipt with the same fingerprint replays
the stored message, a different one is 409 conflict; without a
receipt one statement inserts the message, the key row and the chat
mark, so both commit or neither does. Concurrent retries wait on the
same row lock and then read the committed receipt. The fingerprint is
SHA-256 of the canonical JSON of the parsed body, so layout does not
matter, and only 201 answers are ever stored.

A lock timeout now counts as a dependency failure and answers 503
instead of 500, so a retry with the same key is safe.

Own tests cover replay, a different layout of the same JSON, conflict,
missing and malformed keys, rejected attempts that leave the key free,
scope by user and chat, revalidation after logout and for outsiders,
expiry, and eight concurrent retries through two app instances
sharing one PostgreSQL. Every earlier message post sends a fresh key.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:36:15 +03:00
ArtyomandClaude Opus 5.5 a18e7b320b Add the idempotency key table as an additive migration
Migration 0002 stores one row per accepted message key: the chat, the
sender, the key, the SHA-256 fingerprint of the request and the
message. The primary key (chat_id, user_id, key) is the key scope of
the contract, the composite foreign key keeps the row inside the
messaging tables, as messages already do, and CHECK constraints repeat
the key format and the fingerprint length. Nothing existing changes,
so the lab 2 code keeps running on the new schema.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:33:11 +03:00
ArtyomandClaude Opus 5.5 bf0f0b4c07 Name the serving replica in every response
The platform settings default the instance id to the container
hostname, computed once per process, so every gunicorn worker of a
replica reports the same value and it matches docker ps. The request
pipeline stamps X-Instance-Id next to X-Request-Id on every response
it sends, and the unhandled error handler adds it itself, because the
500 answer is produced outside the pipeline.

Own tests cover 200, 204, 400, 401, 404, the hidden 405, 413 and 500
with a fixed instance id, and the hostname default. The application
version follows the contract version 1.3.0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:32:36 +03:00
ArtyomandClaude Opus 5.5 9db7a24362 Propose load-balanced replicas and idempotent messages as an openspec change
Plan the grade 3 cycle of lab 3 on top of the carried-over lab 2
solution: run the API as several replicas of one compose service
behind HAProxy as the only published entry point, stamp every API
response with X-Instance-Id taken from the container hostname, require
Idempotency-Key on message posts and keep the key and the message in
one PostgreSQL transaction, and add explicit stop-one, restart and
restore hooks.

The design fixes the balancer settings and retry policy, the
idempotency table, the lookup order under the chat row lock, the
request fingerprint and the shutdown budget. Delta specs add the
lab3-load-balancing and lab3-idempotent-messages capabilities and
modify message posting, request identification, shutdown, container
delivery, migrations, restart and the reproducible stand.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:31:41 +03:00
ArtyomandClaude Opus 5.5 0ec2f9dae3 Carry over the agent context and openspec history from lab 2
Bring the Claude Code settings, the lint hook, the opsx commands and
skills, the architecture notes, the lab 1 and lab 2 specs and the
archived lab 1 and lab 2 changes into the lab 3 skeleton, so lab 3
continues the same spec history instead of starting from scratch.

Rewrite AGENTS.md and the openspec context and rules for lab 3:
HAProxy in front of several stateless API replicas, X-Instance-Id on
every response, a mandatory Idempotency-Key for messages shared by all
replicas through PostgreSQL, explicit stop-one and restart hooks, Out
of scope moved to labs 4-7 plus Swarm and HA storage, and rules on
retry policy, timeout chains and replica-local state.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:23:53 +03:00