Lab 4 local verification record
Date: 2026-11-15
Workspace: local Docker Compose, two API replicas, PostgreSQL, Valkey, HAProxy, private RustFS bucket

Structure and syntax
- `make check PYTHON=python3`: PASS: offline skeleton structure, local refs and Python syntax; application NOT tested.
- `make validate PYTHON=.venv/bin/python`: PASS: full OpenAPI 3.1 and JSON Schema validation; application NOT tested.
- `python3 -m py_compile server.py scripts/reconcile.py tests/*.py`: PASS (exit 0).
- `docker compose --env-file .env config --quiet`: PASS (exit 0).
- `make test PYTHON=python3 BASE_URL=http://127.0.0.1:18084`: PASS: lab4 public API scenario.
- Lab3 public smoke: fails its strict old `Message` schema on the additive `attachment_ids` response field; Lab4 requires that field, and the Lab4 public smoke plus lab3 idempotency-race test pass. Text-only POST requests remain accepted.

Attachment behavior and security
- API scenario: upload/download SHA-256 and byte count matched; chat participant could download; outsider received 404.
- Negative upload checks: >10 MiB -> 413; corrupt image and MIME spoof -> 415; unsafe filename -> 400; foreign chat/attachment access -> 404.
- Direct RustFS checks outside the API: anonymous ListBucket -> HTTP 403 and anonymous GetObject for a temporary test object -> HTTP 403; temporary object removed in `finally`.
- Injected database metadata failure after S3 PUT: API returned 503; compensating S3 delete left no orphan object.
- RustFS outage: readiness and upload returned 503; after RustFS restart readiness returned 200.

Streaming and resilience
- Upload burst: 10 concurrent PNG uploads x 7,684,003 bytes = 76,840,030 bytes; observed sample 0.768 s / 95.46 MiB/s.
- API cgroup memory.peak: replica 1 = 162,799,616 B (155.3 MiB), replica 2 = 162,144,256 B (154.5 MiB), limit = 512 MiB per replica; no residual spool files found.
- Failover: 115 confirmed writes, 1 transient failed request, first successful response after 1.434 s; existing session and attachment remained available.
- Persistence: restarted API, HAProxy, PostgreSQL, Valkey, and RustFS without deleting volumes; existing data, session, and attachment remained available.

Reconciler track 2
- `reconcile-scenarios.json`: dry-run found missing metadata and untracked object; apply marked missing object failed; fresh orphan and referenced attachment were preserved; second apply produced 0 actions.
- `reconcile-unit.txt`: dry-run preserved grace-expired orphan; apply deleted it; repeated apply was idempotent.

All commands and measurements above were observed on the local lab environment; throughput is a single-run sample, not a production benchmark.
