Commit Graph
7 Commits
Author SHA1 Message Date
ArtyomandClaude Opus 5.5 6fb4587ef6 Leave the rotation before closing the listener on SIGTERM
On SIGTERM gunicorn closed the listener at once, so a request already
queued on the closing socket reached the balancer as a dropped
connection after sending: a retry for GET and keyed messages, a 502
for user, chat and login posts. Now each worker first raises a
draining flag and keeps serving for DRAIN_SECONDS (2 s by default,
above the 1.5 s worst detection of a 503): readiness answers 503
unavailable, liveness stays 200, and the balancer takes the replica
out while it is still healthy. Only then uvicorn closes the listener
and finishes accepted requests within SHUTDOWN_TIMEOUT_SECONDS minus
the window, so gunicorn graceful_timeout stays 5 s.

The worker runs a uvicorn Server subclass whose main loop adds the
window after the exit signal; UvicornWorker._serve is overridden to
use it. Tests check readiness under the flag and run real gunicorn on
a free port: during the window ready is 503 and live 200, the port
closes after the window, a registration accepted before SIGTERM and
finished after the port closed still gets 201, and the master exits
with code 0 before the shutdown limit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:33:02 +03:00
ArtyomandClaude Opus 5.5 6e4c387e50 Answer readiness within the balancer check timeout
The readiness ping waited up to one second for a pooled connection and
one more second for SELECT 1, while HAProxy gives a check one second.
With a busy pool the balancer saw a check timeout instead of an
explicit 503. The ping now waits 0.4 s for a connection and 0.4 s for
the query, so the worst answer, 0.8 s, still arrives inside the check.
Other queries keep the pool acquire and command timeouts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:33:01 +03:00
ArtyomandClaude Opus 5.5 dc2015473f Run the API as replicas behind HAProxy
compose.yaml now runs one api service with API_REPLICAS replicas and
no published ports, container name or hostname, behind lb, HAProxy
3.2.26 LTS pinned by digest and the only service with ports, bound to
127.0.0.1 (LB_HTTP_PORT for HTTP, LB_STATS_PORT for read-only stats).
The balancer finds replicas through Docker DNS with a server template,
checks GET /health/ready every second (fall 2, rise 2), balances round
robin without stickiness, retries connection failures on another
replica and allows a retry after a sent request only for GET and for
message posts carrying Idempotency-Key, never for users, chats or
login. Its own 502/503/504 answers use the contract error body with
the forwarded X-Request-Id.

The network is split into edge and an internal backend, every service
gets CPU and memory limits, the API runs read-only with a tmpfs /tmp,
two workers per replica, an 8 s request limit and a 5 s graceful stop
inside an 8 s stop_grace_period. Only api builds the image, migrate
reuses it once before the replicas start. PostgreSQL loads
pg_stat_statements and allows 200 connections. The short TTL instance
of lab 2 published the API around the balancer and is removed; its
expiry check stays as an in-process test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:43:43 +03:00
ArtyomandClaude Opus 5.5 7bf28fc6ed Make message posts idempotent with Idempotency-Key
POST /api/v1/chats/{chat_id}/messages now requires the header, checked
at the HTTP boundary (1-128 of [A-Za-z0-9._:-], else 400). The handler
keeps the lab 2 order under the chat row lock: membership, then text,
then the receipt for (chat, sender, key) younger than
IDEMPOTENCY_TTL_SECONDS. A receipt with the same fingerprint replays
the stored message, a different one is 409 conflict; without a
receipt one statement inserts the message, the key row and the chat
mark, so both commit or neither does. Concurrent retries wait on the
same row lock and then read the committed receipt. The fingerprint is
SHA-256 of the canonical JSON of the parsed body, so layout does not
matter, and only 201 answers are ever stored.

A lock timeout now counts as a dependency failure and answers 503
instead of 500, so a retry with the same key is safe.

Own tests cover replay, a different layout of the same JSON, conflict,
missing and malformed keys, rejected attempts that leave the key free,
scope by user and chat, revalidation after logout and for outsiders,
expiry, and eight concurrent retries through two app instances
sharing one PostgreSQL. Every earlier message post sends a fresh key.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:36:15 +03:00
ArtyomandClaude Opus 5.5 a18e7b320b Add the idempotency key table as an additive migration
Migration 0002 stores one row per accepted message key: the chat, the
sender, the key, the SHA-256 fingerprint of the request and the
message. The primary key (chat_id, user_id, key) is the key scope of
the contract, the composite foreign key keeps the row inside the
messaging tables, as messages already do, and CHECK constraints repeat
the key format and the fingerprint length. Nothing existing changes,
so the lab 2 code keeps running on the new schema.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:33:11 +03:00
ArtyomandClaude Opus 5.5 bf0f0b4c07 Name the serving replica in every response
The platform settings default the instance id to the container
hostname, computed once per process, so every gunicorn worker of a
replica reports the same value and it matches docker ps. The request
pipeline stamps X-Instance-Id next to X-Request-Id on every response
it sends, and the unhandled error handler adds it itself, because the
500 answer is produced outside the pipeline.

Own tests cover 200, 204, 400, 401, 404, the hidden 405, 413 and 500
with a fixed instance id, and the hostname default. The application
version follows the contract version 1.3.0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:32:36 +03:00
ArtyomandClaude Opus 5.5 416ce4f9db Carry over the lab 2 solution
Bring the application, own tests, Dockerfile, toolchain, compose
stack, migrations, secret generator, restart hook, evidence scripts and
CI job from the lab 2 solution commit 8cddb09 unchanged, merge the lab
2 Makefile targets into the lab 3 skeleton Makefile without touching
its targets, and append the lab 2 solution variables to the skeleton
.env.example. The package version follows the contract version 1.3.0.

This is the starting point of the lab 3 cycle: lint, own tests and the
offline skeleton check pass before any lab 3 change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 11:26:15 +03:00