Files
mtusi2026lab2/infra/haproxy/haproxy.cfg
T
ArtyomandClaude Opus 5.5 502c7d8ace Keep a returning replica out of rotation until its checks pass
The first failover-each run had no errors, yet the balancer counted
19 and 49 redispatches. The HAProxy log showed why: when a stopped
container comes back into Docker DNS, its server-template slot leaves
MAINT (resolution) as UP/READY at once, before any health check, so
requests went to a gunicorn that was not listening yet and were
redispatched to the other replica.

default-server now sets init-state fully-down (HAProxy 3.1+): at start,
on discovery through DNS and on leaving maintenance a server stays DOWN
until rise 2 checks pass. A rerun under 100 rps shows zero retries and
redispatches for both replicas; rejoining takes about a second longer.
The design and the failover spec of the change record the finding.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 19:51:49 +03:00

68 lines
2.7 KiB
INI

global
log stdout format raw local0 info
maxconn 4096
hard-stop-after 5s
tune.bufsize 32768
resolvers docker
nameserver dns 127.0.0.11:53
accepted_payload_size 8192
resolve_retries 3
timeout resolve 1s
timeout retry 1s
hold nx 2s
hold other 2s
hold refused 2s
hold timeout 2s
defaults
mode http
log global
option dontlognull
option dontlog-normal
log-format "%ci [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq rid=%[var(txn.request_id)] %HM %HPO"
option http-server-close
option redispatch 1
retries 2
retry-on conn-failure empty-response
timeout connect 500ms
timeout client 30s
timeout server 10s
timeout http-request 5s
timeout http-keep-alive 5s
timeout queue 3s
timeout check 1s
http-error status 502 content-type application/json hdr X-Request-Id "%[var(txn.request_id)]" lf-string '{"error":{"code":"unavailable","message":"Upstream replica failed","request_id":"%[var(txn.request_id)]"}}'
http-error status 503 content-type application/json hdr X-Request-Id "%[var(txn.request_id)]" lf-string '{"error":{"code":"unavailable","message":"No ready API replica","request_id":"%[var(txn.request_id)]"}}'
http-error status 504 content-type application/json hdr X-Request-Id "%[var(txn.request_id)]" lf-string '{"error":{"code":"unavailable","message":"Upstream timeout","request_id":"%[var(txn.request_id)]"}}'
frontend public
bind :8080
option forwardfor
http-request set-var(txn.request_id) req.fhdr(X-Request-Id) if { req.fhdr(X-Request-Id) -m len 1:200 }
http-request set-var(txn.request_id) uuid(4) unless { var(txn.request_id) -m found }
http-request set-header X-Request-Id %[var(txn.request_id)]
default_backend api
backend api
balance roundrobin
option httpchk
http-check send meth GET uri /health/ready ver HTTP/1.1 hdr Host lb
http-check expect status 200
acl message_path path_reg ^/api/v1/chats/[^/]+/messages$
acl has_key req.hdr(Idempotency-Key) -m found
http-request set-var(txn.replay_safe) bool(1) if METH_GET
http-request set-var(txn.replay_safe) bool(1) if METH_POST message_path has_key
http-request disable-l7-retry unless { var(txn.replay_safe) -m bool }
default-server check init-state fully-down inter 1s fastinter 500ms downinter 2s fall 2 rise 2 resolvers docker resolve-prefer ipv4 init-addr none maxconn "${API_SERVER_MAXCONN-64}"
server-template api 8 api:8080
frontend stats
bind :8404
monitor-uri /healthz
monitor fail if { nbsrv(api) lt 1 }
http-request use-service prometheus-exporter if { path /metrics }
stats enable
stats uri /stats
stats refresh 5s