Add the level 3 pipeline scenarios and the stand evidence

Bring the stdlib suite and image generator of the lab 5 preparation:
make suite-preflight checks services, readiness, schema columns, broker
topology and the faults override, and make level3 runs the grade 3
scenarios (smoke through the pipeline, worker down with API, relay and
broker restarts, duplicate delivery after ready, poison image, S3
outage with recovery, retries up to JOB_MAX_ATTEMPTS) and checks the
invariants into evidence/level3.json. make pipeline-evidence shows the
queued job on the broker while the worker is stopped, its properties
and contract payload, the 409 before ready and SIGTERM of the relay and
the worker; make topology dumps queues, consumers with prefetch,
permissions and users.

The lab 4 failover and S3 outage scenarios and the upload fault suite
now expect 202 and wait for ready, the stack evidence lists the broker
and its client packages, and the CI acceptance job runs the pipeline
scenarios and the SIGTERM check of the new processes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Artyom
2026-10-29 01:19:13 +03:00
co-authored by Claude Opus 5.5
parent c55fb1b998
commit fe2693870e
10 changed files with 1497 additions and 20 deletions
+23 -2
View File
@@ -19,7 +19,7 @@ jobs:
- run: make student-test VENV=.venv
acceptance:
name: Стенд Compose с RustFS за HAProxy, smoke, вложения, отказ реплики, рестарт и отказы зависимостей
name: Стенд Compose с RabbitMQ, relay и worker за HAProxy, smoke, pipeline, отказ реплики, рестарт и отказы зависимостей
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
@@ -34,6 +34,17 @@ jobs:
- name: Публичный smoke через балансировщик
run: make test
- name: Топология брокера, consumer с prefetch и права клиентов
run: make topology
- name: Уровень «3» pipeline — worker остановлен, рестарт API и брокера, дубль, S3, исчерпание попыток
run: |
make suite-preflight
make level3
- name: Сообщение задания в очереди, 409 до ready и SIGTERM relay и worker
run: make pipeline-evidence
- name: Вложение после остановки принявшей его реплики и при недоступном S3
run: |
make scenario SCENARIO=failover-upload
@@ -68,12 +79,22 @@ jobs:
- name: Наружу опубликован только балансировщик
run: |
docker compose ps --format '{{.Service}} {{.Ports}}' | awk '$2 != "" && $1 != "lb" {print; bad=1} END {exit bad}'
docker compose ps --format '{{.Service}} {{.Ports}}' | awk '$2 != "" && $1 != "lb" && $1 != "broker" {print; bad=1} END {exit bad}'
docker compose ps --format '{{.Service}} {{.Ports}}' | awk '$1 == "broker" && $2 !~ /^127\.0\.0\.1:/ {print; bad=1} END {exit bad}'
- name: Процессы работают не от root
run: |
docker compose top | awk 'NR>1 && $3=="0" {print; bad=1} END {exit bad}'
- name: Корректная остановка relay и worker по SIGTERM
run: |
start=$(date +%s)
docker compose stop -t 10 relay worker
elapsed=$(( $(date +%s) - start ))
codes=$(docker inspect -f '{{.State.ExitCode}}' $(docker compose ps -aq relay worker) | sort -u)
echo "остановка за ${elapsed} с, коды завершения ${codes}"
test "$elapsed" -le 10 && test "$codes" = "0"
- name: Корректная остановка реплик API по SIGTERM
run: |
start=$(date +%s)