Commit Graph
7 Commits
Author SHA1 Message Date
ArtyomandClaude Opus 5.5 a774a2fbbd Measure upload memory and bound the buffers along the chain
upload_memory.py recreates the replicas and the balancer for every run
of 10, 20 and 40 concurrent uploads of 1 and 10 MiB with 64 slots
(effectively no limit) and with 3, drives them through lab4_seed.py
from the prepared tooling, and keeps per-run JSON plus a summary with
the cgroup memory.peak of each replica and of the balancer and the
oom_kill counters. The seed script now also records memory.current
before the load and reads X-Instance-Id regardless of case.

The first run showed that slots alone only move a waiting body out of
the worker: with 3 slots and 40 uploads of 10 MiB the kernel grew the
socket buffers of HAProxy to about 110 MB and the 128 MiB balancer was
killed. HAProxy now caps its client and server socket buffers at 64
KiB and the API replicas cap tcp_rmem and tcp_wmem at 256 KiB, which
keeps the balancer at 30 MiB. glibc raised its mmap threshold after
the first large free and kept freed 10 MiB bodies in the heap, so the
API fixes MALLOC_MMAP_THRESHOLD_ at 128 KiB.

With these settings 40 concurrent uploads of 10 MiB peak at 323 MiB of
512 per replica and all 80 succeed, while without the slot limit the
same load reaches 512 MiB, a worker is killed and only 31 of 80
succeed. UPLOAD_CONCURRENCY stays at 3.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-29 00:14:09 +03:00
ArtyomandClaude Opus 5.5 28055a8551 Pass the upload limits to the API and check the console from the network
The API replicas get UPLOAD_CONCURRENCY, UPLOAD_GRACE_SECONDS,
UPLOAD_SWEEP_INTERVAL_SECONDS and UPLOAD_SWEEP_BATCH through the S3
anchor of compose.yaml with the same defaults as the settings, and
.env.example documents them. A value in the shell overrides the
default, which the memory experiment uses to recreate the replicas with
a different number of slots.

s3-check.sh now also asks the console port of the S3 service from a
separate container on the stand network and expects a refused
connection, so 23 checks cover ports, the console inside and outside
the container, the repeated bootstrap, the rights of the API key and
anonymous access.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1d3RLHNBBMA6UqBLLJv27
2026-10-28 22:40:56 +03:00
ArtyomandClaude Opus 5.5 db901d6401 Record level 3 evidence and fill in the report
Evidence comes from the stand started from an empty state and from a
fresh clone: the public smoke, the contract checks, lint and 325 own
tests, the storage layout with an out-of-API comparison of a stored
object against its metadata, failover of each replica under writes, the
upload failover scenario, persistence with StartedAt of every service
including S3, the S3 outage scenario and 22 S3 privacy and bootstrap
checks. The storage evidence script now covers the attachment tables
and the S3 secrets.

REPORT.md follows the lab 4 template: launch from a clean clone,
architecture with the source of truth for each fact, the write order
between PostgreSQL and S3 and what each failure leaves behind, where
each upload limit applies, roles and services, the .env.example
mapping, grade 3 rows and the grade 3 failure table. The design notes
the botocore Expect header that hung uploads behind the port proxy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 22:20:10 +03:00
ArtyomandClaude Opus 5.5 24f9d6195b Add the attachment failure scenarios and S3 checks
lab4_scenarios.py drives the public contract through tests/client.py:
failover-upload stops the replica that answered the upload with the
stop-one hook and downloads through another one comparing SHA-256, and
s3-outage stops S3 and records status and time of upload, download,
metadata, a text message and readiness before bringing it back. Both
print JSON with an ok field. s3-check.sh checks published ports, the
console, two more bootstrap runs without state changes, the allowed
and denied operations of the API key and anonymous access including a
plain HTTP GET from a separate container. evidence-attachments.sh
shows where the bytes live and compares a stored object with its
metadata outside the API.

Makefile gains scenario, s3-check and attachments-evidence, ruff covers
the new script, and the CI acceptance job runs both scenarios and the
S3 check after the smoke test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:50:03 +03:00
ArtyomandClaude Opus 5.5 5aa766acff Run RustFS with a private bucket in the stand
MinIO community images can no longer be pulled on this arm64 host, so
the stand runs RustFS 1.0.4 pinned by digest as non-root with a read
only root, no published port, the console off and data on the s3data
volume. A one-off s3-init with rc v0.1.36 creates the bucket, keeps it
private for anonymous clients, creates the API user with a policy on
the objects and listing of that one bucket, imports a lifecycle rule
for incomplete multipart uploads and checks itself; the API replicas
start only after it succeeded. Root keys reach only s3, s3-init and the
operator s3-cli; the API gets its own key and the S3 settings through
an anchor merged into its environment.

secrets.sh creates hex S3 keys, the restart hook restarts S3 next to
PostgreSQL and Valkey and waits until it is healthy, the limits policy
covers the S3 services, and the stack evidence lists the new packages.
The object storage adapter is tested against a throwaway RustFS of the
same image: a 10 MiB round trip, a missing object, an idempotent delete
and an unreachable endpoint failing within the request budget.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:50:03 +03:00
ArtyomandClaude Opus 5.5 bf8e4e3cd1 Carry over the lab 3 solution
Bring the application, own tests, Dockerfile, toolchain, compose stack,
HAProxy and k6 configuration, migrations, hooks, evidence and level 4
scripts and the CI job from the lab 3 solution commit 61c0c28, merge
the lab 3 Makefile targets into the lab 4 skeleton Makefile without
touching its targets, and append the lab 3 solution variables to the
skeleton .env.example. The package and the application follow the
contract version 1.4.0, and the image and the throwaway test containers
are tagged lab4 so they do not collide with a lab 3 stand on the same
host.

This is the starting point of the lab 4 cycle: lint, own tests and the
offline skeleton check pass before any lab 4 change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 20:17:51 +03:00
Mikhail Verkovykh ec540a1a72 Подготовить skeleton лабораторной 4 по мессенджеру 2026-09-10 18:01:35 +03:00