Put readiness behind a port and log the drain window through Logger

The /health/ready route imported the PostgreSQL adapter and kept the
drain flag in the HTTP layer, which gunicorn_conf imported to raise it.
Readiness is now a port of the shared application layer; its adapter in
shared/infrastructure checks the per-process drain flag and pings
PostgreSQL, and PlatformProvider assembles it. The route only knows the
port, and gunicorn_conf raises the flag through the infrastructure
module. Another critical dependency now changes the adapter and the
provider, not the HTTP layer.

The drain window was logged by a uvicorn logger with a printf string;
it is now the service.draining event of a StructlogLogger with the same
fields as the application, and failover-evidence looks for it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
This commit is contained in:
Artyom
2026-10-30 05:40:16 +03:00
co-authored by Claude Opus 5.5
parent aa1485bd86
commit d99a48a5db
10 changed files with 47 additions and 23 deletions
@@ -2,10 +2,10 @@
- [x] 1.1 `RequestPipeline`: отмена задачи → `request.cancelled`, `503 unavailable` в общем формате, если ответ не начат, затем отмена пробрасывается
- [x] 1.2 `test_drain.py`: прерванный при остановке запрос — `503`, `application/json`, `error.code = unavailable`, `request_id` = `X-Request-Id`, непустой `X-Instance-Id`
- [ ] 1.3 Порт `Readiness`, адаптер `DatabaseReadiness` с флагом `DRAINING` в `shared/infrastructure/readiness.py`, сборка в `PlatformProvider`, маршрут `/health/ready` через порт; тест состава модулей ядра
- [ ] 1.4 `gunicorn_conf.py`: флаг из инфраструктуры, событие `service.draining` через `StructlogLogger`; `failover-evidence.sh` ищет его в логе
- [ ] 1.5 `openspec/config.yaml`: карта модулей (`readiness` за портом)
- [ ] 1.6 Проверка группы: `make lint`, `make student-test`, `make check`
- [x] 1.3 Порт `Readiness`, адаптер `DatabaseReadiness` с флагом `DRAINING` в `shared/infrastructure/readiness.py`, сборка в `PlatformProvider`, маршрут `/health/ready` через порт; тест состава модулей ядра
- [x] 1.4 `gunicorn_conf.py`: флаг из инфраструктуры, событие `service.draining` через `StructlogLogger`; `failover-evidence.sh` ищет его в логе
- [x] 1.5 `openspec/config.yaml`: карта модулей (`readiness` за портом)
- [x] 1.6 Проверка группы: `make lint`, `make student-test`, `make check`
## 2. Стенд и CI
+3 -3
View File
@@ -50,14 +50,14 @@ context: |
main.py сборка приложения: собирает вклады зон в один процесс
migrate.py одноразовая точка входа: применяет SQL-миграции под advisory lock
gunicorn_conf.py конфигурация процесса: воркеры, окно вывода из ротации по SIGTERM
(DrainingServer, MessengerWorker; флаг готовности — shared/entrypoints/health)
(DrainingServer, MessengerWorker; флаг вывода — shared/infrastructure/readiness)
identity/ пользователи, пароли и сессии: ошибки, события, команды, значения, порты,
адаптеры PostgreSQL/Valkey/Argon2 и весь свой HTTP-слой, включая аутентификацию
messaging/ чаты, история и идемпотентная отправка: ошибки, события, команды, агрегат Chat,
курсор, адаптеры PostgreSQL и весь свой HTTP-слой
shared/ только платформа: domain / application / infrastructure (часы, id, журнал,
пул PostgreSQL, UoW-база) и entrypoints/ (конвейер, health, ошибки, настройки,
идентификатор реплики)
пул PostgreSQL, UoW-база, готовность за портом Readiness) и entrypoints/
(конвейер, health, ошибки, настройки, идентификатор реплики)
Стрелки указывают внутрь: `domain/` не импортирует инфраструктуру, `application/` зависит только
от портов, адаптеры подключаются в `ioc.py` зоны. Ядро `shared/` не импортирует веб-фреймворк.
+1 -1
View File
@@ -21,7 +21,7 @@ since=$(date -u +%FT%TZ)
stopped=$(replicas | awk '$2 == "exited" {print $1}')
{
echo "== stopped replica: $stopped; shutdown: signal, drain window with readiness 503, close, exit"
docker logs --since "$since" "$stopped" 2>&1 | grep -E 'Handling signal|Draining|"status":503|Shutting down|Waiting for conn|Finished server|service.stopped|SIGTERM' | tail -16
docker logs --since "$since" "$stopped" 2>&1 | grep -E 'Handling signal|service.draining|"status":503|Shutting down|Waiting for conn|Finished server|service.stopped|SIGTERM' | tail -16
echo "\$ scripts/restore.sh"
scripts/restore.sh
echo "== after restore"
+4 -4
View File
@@ -1,5 +1,4 @@
import asyncio
import logging
import os
import sys
from typing import Any
@@ -10,12 +9,13 @@ from uvicorn.server import Server
from uvicorn_worker import UvicornWorker
from messenger.main import LOG_FIELDS
from messenger.shared.entrypoints.health import DRAINING
from messenger.shared.infrastructure.logging import configure
from messenger.shared.infrastructure.logging import StructlogLogger, configure
from messenger.shared.infrastructure.readiness import DRAINING
LOG_LEVEL = os.environ.get("LOG_LEVEL", "INFO")
SHUTDOWN_SECONDS = int(os.environ.get("SHUTDOWN_TIMEOUT_SECONDS", "6"))
DRAIN_SECONDS = float(os.environ.get("DRAIN_SECONDS", "2"))
LOGGER = StructlogLogger(level=LOG_LEVEL, allowed_fields=LOG_FIELDS)
class StructlogGunicornLogger(GunicornBaseLogger):
@@ -31,7 +31,7 @@ class DrainingServer(Server):
await super().main_loop()
if not self.force_exit:
DRAINING.set()
logging.getLogger("uvicorn.error").info("Draining for %.1f s", DRAIN_SECONDS)
LOGGER.info("service.draining", limit=DRAIN_SECONDS)
await asyncio.sleep(DRAIN_SECONDS)
@@ -26,6 +26,11 @@ class Logger(Protocol):
def error(self, event: str, /, **fields: Any) -> None: ...
@runtime_checkable
class Readiness(Protocol):
async def check(self) -> None: ...
@runtime_checkable
class UnitOfWork(Protocol):
def transaction(self) -> AbstractAsyncContextManager[None]: ...
+3 -9
View File
@@ -1,14 +1,10 @@
from threading import Event
from dishka.integrations.fastapi import FromDishka, inject
from fastapi import APIRouter
from messenger.shared.application.errors import Unavailable
from messenger.shared.application.ports import Readiness
from messenger.shared.entrypoints.schemas import HealthOut
from messenger.shared.infrastructure.postgres import Database
router = APIRouter(tags=["health"])
DRAINING = Event()
@router.get("/health/live")
@@ -18,8 +14,6 @@ async def live() -> HealthOut:
@router.get("/health/ready")
@inject
async def ready(database: FromDishka[Database]) -> HealthOut:
if DRAINING.is_set():
raise Unavailable("Реплика выходит из ротации")
await database.ping()
async def ready(readiness: FromDishka[Readiness]) -> HealthOut:
await readiness.check()
return HealthOut()
+6 -1
View File
@@ -3,11 +3,12 @@ from collections.abc import AsyncIterator
from dishka import Provider, Scope, provide
from messenger.shared.application.messagebus import EventBus, InMemoryEventPublisher
from messenger.shared.application.ports import Clock, EventPublisher, IdGenerator, Logger
from messenger.shared.application.ports import Clock, EventPublisher, IdGenerator, Logger, Readiness
from messenger.shared.config import PlatformSettings
from messenger.shared.infrastructure.clock import MonotonicUtcClock
from messenger.shared.infrastructure.ids import Uuid7Generator
from messenger.shared.infrastructure.postgres import Database
from messenger.shared.infrastructure.readiness import DatabaseReadiness
class PlatformProvider(Provider):
@@ -45,6 +46,10 @@ class PlatformProvider(Provider):
yield database
await database.close()
@provide(scope=Scope.APP)
def readiness(self, database: Database) -> Readiness:
return DatabaseReadiness(database)
@provide(scope=Scope.APP)
def event_bus(self, logger: Logger, settings: PlatformSettings) -> EventBus:
return EventBus(logger, max_queue=settings.event_queue_max)
@@ -0,0 +1,19 @@
from threading import Event
from messenger.shared.application.errors import Unavailable
from messenger.shared.infrastructure.postgres import Database
DRAINING = Event()
class DatabaseReadiness:
__slots__ = ("_database", "_draining")
def __init__(self, database: Database, draining: Event = DRAINING) -> None:
self._database = database
self._draining = draining
async def check(self) -> None:
if self._draining.is_set():
raise Unavailable("Реплика выходит из ротации")
await self._database.ping()
@@ -95,6 +95,7 @@ def test_shared_core_holds_only_platform() -> None:
"ids.py",
"logging.py",
"postgres.py",
"readiness.py",
}
+1 -1
View File
@@ -9,7 +9,7 @@ import time
import httpx
from conftest import DATABASE, new_user
from messenger.shared.entrypoints.health import DRAINING
from messenger.shared.infrastructure.readiness import DRAINING
DRAIN = 1.0
SHUTDOWN = 3