Put readiness behind a port and log the drain window through Logger
The /health/ready route imported the PostgreSQL adapter and kept the drain flag in the HTTP layer, which gunicorn_conf imported to raise it. Readiness is now a port of the shared application layer; its adapter in shared/infrastructure checks the per-process drain flag and pings PostgreSQL, and PlatformProvider assembles it. The route only knows the port, and gunicorn_conf raises the flag through the infrastructure module. Another critical dependency now changes the adapter and the provider, not the HTTP layer. The drain window was logged by a uvicorn logger with a printf string; it is now the service.draining event of a StructlogLogger with the same fields as the application, and failover-evidence looks for it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
aa1485bd86
commit
d99a48a5db
@@ -2,10 +2,10 @@
|
||||
|
||||
- [x] 1.1 `RequestPipeline`: отмена задачи → `request.cancelled`, `503 unavailable` в общем формате, если ответ не начат, затем отмена пробрасывается
|
||||
- [x] 1.2 `test_drain.py`: прерванный при остановке запрос — `503`, `application/json`, `error.code = unavailable`, `request_id` = `X-Request-Id`, непустой `X-Instance-Id`
|
||||
- [ ] 1.3 Порт `Readiness`, адаптер `DatabaseReadiness` с флагом `DRAINING` в `shared/infrastructure/readiness.py`, сборка в `PlatformProvider`, маршрут `/health/ready` через порт; тест состава модулей ядра
|
||||
- [ ] 1.4 `gunicorn_conf.py`: флаг из инфраструктуры, событие `service.draining` через `StructlogLogger`; `failover-evidence.sh` ищет его в логе
|
||||
- [ ] 1.5 `openspec/config.yaml`: карта модулей (`readiness` за портом)
|
||||
- [ ] 1.6 Проверка группы: `make lint`, `make student-test`, `make check`
|
||||
- [x] 1.3 Порт `Readiness`, адаптер `DatabaseReadiness` с флагом `DRAINING` в `shared/infrastructure/readiness.py`, сборка в `PlatformProvider`, маршрут `/health/ready` через порт; тест состава модулей ядра
|
||||
- [x] 1.4 `gunicorn_conf.py`: флаг из инфраструктуры, событие `service.draining` через `StructlogLogger`; `failover-evidence.sh` ищет его в логе
|
||||
- [x] 1.5 `openspec/config.yaml`: карта модулей (`readiness` за портом)
|
||||
- [x] 1.6 Проверка группы: `make lint`, `make student-test`, `make check`
|
||||
|
||||
## 2. Стенд и CI
|
||||
|
||||
|
||||
@@ -50,14 +50,14 @@ context: |
|
||||
main.py сборка приложения: собирает вклады зон в один процесс
|
||||
migrate.py одноразовая точка входа: применяет SQL-миграции под advisory lock
|
||||
gunicorn_conf.py конфигурация процесса: воркеры, окно вывода из ротации по SIGTERM
|
||||
(DrainingServer, MessengerWorker; флаг готовности — shared/entrypoints/health)
|
||||
(DrainingServer, MessengerWorker; флаг вывода — shared/infrastructure/readiness)
|
||||
identity/ пользователи, пароли и сессии: ошибки, события, команды, значения, порты,
|
||||
адаптеры PostgreSQL/Valkey/Argon2 и весь свой HTTP-слой, включая аутентификацию
|
||||
messaging/ чаты, история и идемпотентная отправка: ошибки, события, команды, агрегат Chat,
|
||||
курсор, адаптеры PostgreSQL и весь свой HTTP-слой
|
||||
shared/ только платформа: domain / application / infrastructure (часы, id, журнал,
|
||||
пул PostgreSQL, UoW-база) и entrypoints/ (конвейер, health, ошибки, настройки,
|
||||
идентификатор реплики)
|
||||
пул PostgreSQL, UoW-база, готовность за портом Readiness) и entrypoints/
|
||||
(конвейер, health, ошибки, настройки, идентификатор реплики)
|
||||
|
||||
Стрелки указывают внутрь: `domain/` не импортирует инфраструктуру, `application/` зависит только
|
||||
от портов, адаптеры подключаются в `ioc.py` зоны. Ядро `shared/` не импортирует веб-фреймворк.
|
||||
|
||||
@@ -21,7 +21,7 @@ since=$(date -u +%FT%TZ)
|
||||
stopped=$(replicas | awk '$2 == "exited" {print $1}')
|
||||
{
|
||||
echo "== stopped replica: $stopped; shutdown: signal, drain window with readiness 503, close, exit"
|
||||
docker logs --since "$since" "$stopped" 2>&1 | grep -E 'Handling signal|Draining|"status":503|Shutting down|Waiting for conn|Finished server|service.stopped|SIGTERM' | tail -16
|
||||
docker logs --since "$since" "$stopped" 2>&1 | grep -E 'Handling signal|service.draining|"status":503|Shutting down|Waiting for conn|Finished server|service.stopped|SIGTERM' | tail -16
|
||||
echo "\$ scripts/restore.sh"
|
||||
scripts/restore.sh
|
||||
echo "== after restore"
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from typing import Any
|
||||
@@ -10,12 +9,13 @@ from uvicorn.server import Server
|
||||
from uvicorn_worker import UvicornWorker
|
||||
|
||||
from messenger.main import LOG_FIELDS
|
||||
from messenger.shared.entrypoints.health import DRAINING
|
||||
from messenger.shared.infrastructure.logging import configure
|
||||
from messenger.shared.infrastructure.logging import StructlogLogger, configure
|
||||
from messenger.shared.infrastructure.readiness import DRAINING
|
||||
|
||||
LOG_LEVEL = os.environ.get("LOG_LEVEL", "INFO")
|
||||
SHUTDOWN_SECONDS = int(os.environ.get("SHUTDOWN_TIMEOUT_SECONDS", "6"))
|
||||
DRAIN_SECONDS = float(os.environ.get("DRAIN_SECONDS", "2"))
|
||||
LOGGER = StructlogLogger(level=LOG_LEVEL, allowed_fields=LOG_FIELDS)
|
||||
|
||||
|
||||
class StructlogGunicornLogger(GunicornBaseLogger):
|
||||
@@ -31,7 +31,7 @@ class DrainingServer(Server):
|
||||
await super().main_loop()
|
||||
if not self.force_exit:
|
||||
DRAINING.set()
|
||||
logging.getLogger("uvicorn.error").info("Draining for %.1f s", DRAIN_SECONDS)
|
||||
LOGGER.info("service.draining", limit=DRAIN_SECONDS)
|
||||
await asyncio.sleep(DRAIN_SECONDS)
|
||||
|
||||
|
||||
|
||||
@@ -26,6 +26,11 @@ class Logger(Protocol):
|
||||
def error(self, event: str, /, **fields: Any) -> None: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class Readiness(Protocol):
|
||||
async def check(self) -> None: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class UnitOfWork(Protocol):
|
||||
def transaction(self) -> AbstractAsyncContextManager[None]: ...
|
||||
|
||||
@@ -1,14 +1,10 @@
|
||||
from threading import Event
|
||||
|
||||
from dishka.integrations.fastapi import FromDishka, inject
|
||||
from fastapi import APIRouter
|
||||
|
||||
from messenger.shared.application.errors import Unavailable
|
||||
from messenger.shared.application.ports import Readiness
|
||||
from messenger.shared.entrypoints.schemas import HealthOut
|
||||
from messenger.shared.infrastructure.postgres import Database
|
||||
|
||||
router = APIRouter(tags=["health"])
|
||||
DRAINING = Event()
|
||||
|
||||
|
||||
@router.get("/health/live")
|
||||
@@ -18,8 +14,6 @@ async def live() -> HealthOut:
|
||||
|
||||
@router.get("/health/ready")
|
||||
@inject
|
||||
async def ready(database: FromDishka[Database]) -> HealthOut:
|
||||
if DRAINING.is_set():
|
||||
raise Unavailable("Реплика выходит из ротации")
|
||||
await database.ping()
|
||||
async def ready(readiness: FromDishka[Readiness]) -> HealthOut:
|
||||
await readiness.check()
|
||||
return HealthOut()
|
||||
|
||||
@@ -3,11 +3,12 @@ from collections.abc import AsyncIterator
|
||||
from dishka import Provider, Scope, provide
|
||||
|
||||
from messenger.shared.application.messagebus import EventBus, InMemoryEventPublisher
|
||||
from messenger.shared.application.ports import Clock, EventPublisher, IdGenerator, Logger
|
||||
from messenger.shared.application.ports import Clock, EventPublisher, IdGenerator, Logger, Readiness
|
||||
from messenger.shared.config import PlatformSettings
|
||||
from messenger.shared.infrastructure.clock import MonotonicUtcClock
|
||||
from messenger.shared.infrastructure.ids import Uuid7Generator
|
||||
from messenger.shared.infrastructure.postgres import Database
|
||||
from messenger.shared.infrastructure.readiness import DatabaseReadiness
|
||||
|
||||
|
||||
class PlatformProvider(Provider):
|
||||
@@ -45,6 +46,10 @@ class PlatformProvider(Provider):
|
||||
yield database
|
||||
await database.close()
|
||||
|
||||
@provide(scope=Scope.APP)
|
||||
def readiness(self, database: Database) -> Readiness:
|
||||
return DatabaseReadiness(database)
|
||||
|
||||
@provide(scope=Scope.APP)
|
||||
def event_bus(self, logger: Logger, settings: PlatformSettings) -> EventBus:
|
||||
return EventBus(logger, max_queue=settings.event_queue_max)
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
from threading import Event
|
||||
|
||||
from messenger.shared.application.errors import Unavailable
|
||||
from messenger.shared.infrastructure.postgres import Database
|
||||
|
||||
DRAINING = Event()
|
||||
|
||||
|
||||
class DatabaseReadiness:
|
||||
__slots__ = ("_database", "_draining")
|
||||
|
||||
def __init__(self, database: Database, draining: Event = DRAINING) -> None:
|
||||
self._database = database
|
||||
self._draining = draining
|
||||
|
||||
async def check(self) -> None:
|
||||
if self._draining.is_set():
|
||||
raise Unavailable("Реплика выходит из ротации")
|
||||
await self._database.ping()
|
||||
@@ -95,6 +95,7 @@ def test_shared_core_holds_only_platform() -> None:
|
||||
"ids.py",
|
||||
"logging.py",
|
||||
"postgres.py",
|
||||
"readiness.py",
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ import time
|
||||
import httpx
|
||||
|
||||
from conftest import DATABASE, new_user
|
||||
from messenger.shared.entrypoints.health import DRAINING
|
||||
from messenger.shared.infrastructure.readiness import DRAINING
|
||||
|
||||
DRAIN = 1.0
|
||||
SHUTDOWN = 3
|
||||
|
||||
Reference in New Issue
Block a user