lab4: вложения в приватном S3 (SeaweedFS), метаданные в PostgreSQL

- compose: сервис s3 (SeaweedFS 4.46 по digest, volume s3data, внутренняя сеть storage,
  web-интерфейсы только на 127.0.0.1 внутри контейнера), одноразовый s3-init;
- три пары ключей S3: admin (только bootstrap), api (Read+Write своего bucket),
  reconciler (+List); bootstrap идемпотентен и проверяет приватность и права;
- миграция 0004: attachments (pending -> ready, случайный object_key), message_attachments,
  post_message_v2 с проверкой вложений в той же транзакции;
- POST /chats/{id}/attachments: лимиты до чтения/декодирования, потоковая проверка PNG
  (CRC + zlib до точного размера), JPEG через djpeg -strict, семафор загрузок;
- GET метаданных и /content (стриминг из S3, без redirect), readiness с S3,
  HAProxy: деградированный режим без S3 через /health/ready/core;
- janitor незавершённых загрузок с grace period.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Alexey Bodanyuk
2026-10-25 13:26:01 +03:00
co-authored by Claude Opus 5.5
parent d916d09f7b
commit 335df4b2b2
19 changed files with 2586 additions and 69 deletions
+31 -6
View File
@@ -1,4 +1,4 @@
# HAProxy 3.2 LTS — the single public entry point of the lab 3 stand (compose service `lb`).
# HAProxy 3.2 LTS — the single public entry point of the lab 4 stand (compose service `lb`).
#
# Backend discovery: Docker DNS name `api` returns one A record per RUNNING replica of the
# compose service `api` (deploy.replicas / --scale). server-template keeps 8 slots and
@@ -6,10 +6,18 @@
# full stack restart are picked up without restarting the LB. A stopped container drops
# out of DNS and its slot goes to MAINT.
#
# Readiness: an ACTIVE check `GET /health/ready` (PostgreSQL + Valkey reachable, not draining)
# every 1 s; 2 failures -> DOWN, 2 successes -> UP. Passive errors do not decide anything on
# their own, and there are NO backup/fallback servers without checks: a replica that answers
# /health/live but not /health/ready never gets business traffic.
# Readiness: an ACTIVE check `GET /health/ready` (PostgreSQL + Valkey + S3 reachable, not
# draining) every 1 s; 2 failures -> DOWN, 2 successes -> UP. Passive errors do not decide
# anything on their own, and there are NO fallback servers without checks: a replica that
# answers /health/live but not /health/ready never gets traffic while a fully ready one exists.
#
# Degraded mode (lab 4): S3 is a SHARED dependency. If it is down, /health/ready is 503 on
# every replica and backend `api` has no server. Instead of answering 503 to everything, the
# frontend then switches to backend `api_core`: the same replicas, checked by
# /health/ready/core (PostgreSQL + Valkey, not draining). Chats and text messages keep working,
# every attachment request is answered 503 by the API itself (fast, S3_TIMEOUT_SECONDS).
# A replica that alone lost S3 (e.g. its storage network) is DOWN in `api` while the others are
# UP, so it gets no traffic at all.
# Worst case detection of a failed replica (a check answered at once counts at once, a check
# that hangs counts only after `timeout check`):
# fast explicit 503 (drain after SIGTERM): inter 1 s + fastinter 0.5 s ~= 1.5 s;
@@ -72,16 +80,33 @@ frontend public
http-request set-header X-Request-Id %[uuid()] unless { req.hdr(x-request-id) -m reg ^[A-Za-z0-9._:-]{1,128}$ }
http-request set-var(txn.request_id) req.hdr(x-request-id)
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq rid=%[var(txn.request_id)] \"%HM %HPO\""
default_backend api
use_backend api if { nbsrv(api) gt 0 }
default_backend api_core
backend api
balance roundrobin
# Uploads (<= 10 MiB body + verification + S3 PUT): the API's own budget is
# UPLOAD_READ_TIMEOUT 30 s + IMAGE_VERIFY 10 s + S3; the LB must not cut it at 10 s (a 504
# after a successful upload would make the client upload a duplicate).
acl is_upload path_reg ^/api/v1/chats/[^/]+/attachments$
http-request set-timeout server 60s if is_upload METH_POST
option httpchk
http-check send meth GET uri /health/ready ver HTTP/1.1 hdr Host api
http-check expect status 200
default-server inter 1s fastinter 500ms downinter 1s fall 2 rise 2 check resolvers docker init-addr none resolve-prefer ipv4 maxconn 200
server-template api 8 api:8080
# Degraded mode only (see the header): used when NO replica is fully ready.
backend api_core
balance roundrobin
acl is_upload path_reg ^/api/v1/chats/[^/]+/attachments$
http-request set-timeout server 60s if is_upload METH_POST
option httpchk
http-check send meth GET uri /health/ready/core ver HTTP/1.1 hdr Host api
http-check expect status 200
default-server inter 1s fastinter 500ms downinter 1s fall 2 rise 2 check resolvers docker init-addr none resolve-prefer ipv4 maxconn 200
server-template core 8 api:8080
# Local-only status page (127.0.0.1:8404 on the host): which replica is UP/DOWN right now.
frontend stats
bind :8404