date: 2026-10-30T02:45:26Z  commit: e049c26
== before: api replicas (hostname = X-Instance-Id), state, StartedAt, FinishedAt
ff81f93e8b0d running started=2026-10-30T02:45:24.812924046Z finished=2026-10-30T02:45:12.063064055Z
fe75c6c37e5a running started=2026-10-30T02:44:57.104698047Z finished=0001-01-01T00:00:00Z
$ make test-failover ACTION_SCRIPT=scripts/stop-one.sh
python3 tests/resilience.py --mode failover --action-script "scripts/stop-one.sh" --base-url "http://127.0.0.1:18083" 
Executing explicit failover hook; hook output suppressed to avoid leaking secrets.
{"observed_instances_before": 2, "successful_writes": 136, "failed_attempts": 0, "first_recovery_seconds": 2.682}
PASS: failover preserved confirmed data and existing session. Hook scope needs operator evidence.
== after: api replicas
ff81f93e8b0d running started=2026-10-30T02:45:24.812924046Z finished=2026-10-30T02:45:12.063064055Z
fe75c6c37e5a exited started=2026-10-30T02:44:57.104698047Z finished=2026-10-30T02:45:29.956667174Z
== stopped replica: fe75c6c37e5a; shutdown: signal, drain window with readiness 503, close, exit
{"event":"Handling signal: term","level":"info","ts":"2026-10-30T02:45:27.514055Z"}
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T02:45:27.518751Z"}
{"limit":2.0,"event":"service.draining","level":"info","ts":"2026-10-30T02:45:27.528773Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"efd64978c14445e3a460198e8ef66a32","duration_ms":0.358,"event":"request.handled","level":"info","ts":"2026-10-30T02:45:27.962664Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"53b0876587c14b2cbae1936781f79fc0","duration_ms":0.401,"event":"request.handled","level":"info","ts":"2026-10-30T02:45:28.465375Z"}
{"method":"GET","route":"/health/ready","status":503,"request_id":"947ec9729b774819b5de906d12ad0f91","duration_ms":0.306,"event":"request.handled","level":"info","ts":"2026-10-30T02:45:29.005486Z"}
{"event":"Shutting down","level":"info","ts":"2026-10-30T02:45:29.520790Z"}
{"event":"Shutting down","level":"info","ts":"2026-10-30T02:45:29.529825Z"}
{"event":"service.stopped","level":"info","ts":"2026-10-30T02:45:29.623527Z"}
{"event":"Finished server process [14]","level":"info","ts":"2026-10-30T02:45:29.623728Z"}
{"event":"service.stopped","level":"info","ts":"2026-10-30T02:45:29.632649Z"}
{"event":"Finished server process [13]","level":"info","ts":"2026-10-30T02:45:29.632781Z"}
{"event":"Worker (pid:13) was sent SIGTERM!","level":"info","ts":"2026-10-30T02:45:29.635792Z"}
{"event":"Worker (pid:14) was sent SIGTERM!","level":"info","ts":"2026-10-30T02:45:29.635917Z"}
{"event":"Shutting down: Master","level":"info","ts":"2026-10-30T02:45:29.737995Z"}
$ scripts/restore.sh
 Container mtusi2026lab3-api-1  Running
 Container mtusi2026lab3-api-2  Starting
 Container mtusi2026lab3-api-2  Started
 Container mtusi2026lab3-api-1  Waiting
 Container mtusi2026lab3-api-2  Waiting
 Container mtusi2026lab3-api-1  Healthy
 Container mtusi2026lab3-api-2  Healthy
== after restore
ff81f93e8b0d running started=2026-10-30T02:45:24.812924046Z finished=2026-10-30T02:45:12.063064055Z
fe75c6c37e5a running started=2026-10-30T02:45:42.870380597Z finished=2026-10-30T02:45:29.956667174Z
